To improve the multi-speed adaptability of the powered prosthetic knee, this paper presented a speed-adaptive neural network control based on a powered geared five-bar (GFB) prosthetic knee. The GFB prosthetic knee is actuated via a cylindrical cam-based nonlinear series elastic actuator that can provide the desired actuation for level-ground walking, and its attitude measurement is realized by two inertial sensors and one load cell on the prosthetic knee. To improve the performance of the control system, the motor control and the attitude measurement of the GFB prosthetic knee are run in parallel. The BP neural network uses input data from only the GFB prosthetic knee, and is trained by natural and artificially modified various gait patterns of different able-bodied subjects. To realize the speed-adaptive control, the prosthetic knee speed and gait cycle percentage are identified by the Gaussian mixture model-based gait classifier. Specific knee motion control instructions are generated by matching the neural network predicted gait percentage with the ideal walking gait. Habitual and variable speed level-ground walking experiments are conducted via an able-bodied subject, and the experimental results show that the neural network control system can handle both self-selected walking and variable speed walking with high adaptability. The prosthetic knee is a vital apparatus for the rehabilitation of lower limb amputations, which can enable above-knee amputees to regain walkability in daily life. Compared with purely passive prosthetic knee joints, the powered prosthetic knee joints can provide the amputees with active actuation when needed, thus improving the walking gaits of amputees. To achieve satisfactory performance, the mechanical structure, actuating device, and control system of the powered prosthetic knee must be designed appropriately according to their respective requirements and mutual relations.The control system, which works as the “brain” of the powered prosthetic knee, directly determines the response of the prosthetic knee. Therefore, various control methods, such as the adaptive control ,2,3,4, f(1)Use the neural network to learn the underlying relations between inputs and control parameters of the actuating device, thus controlling the prosthetic knee ;(2)Use the neural network to predict motions of the amputated leg, based on which the specific control instructions of the actuating device is generated .Generally, there are two ways to implement the neural network control of the prosthetic knee: Compared with the former efficient application-specific method, the latter one is more versatile.Many different sources of inputs can be used for the neural network control of prosthetic joints. For example, the surface electromyography (EMG) is an electrical activity generated by the muscle cell, which can be detected via surface electrodes and used for limb pattern recognition. S. Li et al. used EMG signals from five muscles of the lower limb as inputs of the trained neural network, which can successfully identify three different level walking speeds with a general recognition rate of 90.48% . L. Liu In addition to EMG signals, signals of mechanical sensors such as accelerometers, gyroscopes, load cells, etc., are also widely used in neural network control of the prosthetic joint. Compared with sEMG-based control, sensor-based control has the advantage of convenience in use, lower cost, and better reliability ,21. For There is also some indirect information that can be used for motion recognition of lower limbs. For example, D. Joshi et al. predicted the motion of the residual limb via the cross-correlation coefficient obtained from sixteen healthy adult males , while H(1)Current neural network control methods did not take into account the past walking gait, which contains the time information that has been proven essential to the improvement of the overall control performance ,32 and t(2)Current neural network control methods directly predicted the prosthetic locomotion, which will make it hard to modify the gait output for the adaptability of different walking conditions without retraining the neural network.(3)Current neural network control methods often used data from the sound limb of the amputee as inputs, which will lose versatility in dealing with asymmetrical walking gaits such as acceleration or deceleration.In this paper, the authors intend to use the neural network to develop a speed-adaptive control of a previously proposed GFB prosthetic knee concept . HoweverTo solve these shortcomings, this paper tries to use the current walking gait as well as the past walking gait of only the prosthetic knee as inputs of the neural network, and then matches the predicted walking gait percentage with the ideal gait database that contains walking gaits corresponding to different walking speeds to generate specific knee motions for the motor control, thereby improving the speed-adaptive and asymmetrical walking effects of the prosthetic knee as well as the modifiability of the neural network control.The rest of this paper is organized as follows: The GFB prosthetic knee consists of a GFB mechanism and a cam-based nonlinear series elastic actuator. Specific theoretical modeling and formula derivation are presented in ,33. Its The dimension of the GFB mechanism is optimized by the improved genetic annealing algorithm  to implerference  with theThe actuating component of the GFB prosthetic knee is a cam-based nonlinear series elastic actuator, whose dimensional parameters are optimized to enable the prosthetic knee to realize natural level-ground walking gait while the energy consumption of the actuator is minimized. The motor drives the conjugate cylindrical cams to rotate via the gear transmission (5:1), which can deform two helical springs that reciprocate along the frame slides to actuate the thigh linkage The specifications of the powered GFB prosthetic knee are listed in The motor system of the GFB prosthetic knee is shown in The hardware of the motor control system consists of a host PC (LabVIEW), a motor controller , and a brushed DC motor  with an integrated reducer . The host PC communicates with the Maxon Epos 2 controller via a USB cable based on the LabVIEW-Maxon EPOS instrument Driver. The DC motor is connected to the Maxon Epos 2 controller via a 3-channel differential incremental encoder  as well as a power cable. The motor control architecture is finalized by the cascade of a PI  current regulation and a PID  position regulation. The sampling frequencies of the PI current regulation and the PID position regulation are 1 kHz and 500 Hz, respectively. Because the prosthetic knee angle is mechanically related to the output angle of the cam-based nonlinear series elastic actuator, the input of the prosthetic control is set as the desired output angle The attitude measurement of the GFB prosthetic knee is to use mechanical sensors to obtain the locomotion data for the neural network control. The specific platform is shown in The overall attitude measurement process of the GFB prosthetic knee is shown in If the motor control function and the attitude data acquisition/processing function are operated in the serial mode, the motor will be unmonitored at some specific period. The attitude data acquisition/processing period must be restricted below millisecond levels to ensure control timeliness, which will increase the design difficulty of both the software and hardware systems, as shown in Different people have different walking gaits, which are strongly related to their own heights, weights, walking speeds, and walking habits, etc. To study the proper gaits of different people, gait data of ten able-bodied subjects with different physical characteristics distributions  as one of the key inputs for the gait feature extraction. The Total Distance, the Variance, the Standard Deviation, and the Coefficient of Variation can be used to describe the degree of divergence of the input signal. The difference between these four methods is that they are sensitive to the magnitude of the signal fluctuation amplitude while the identifications of the signal trend stay completely consistent. Therefore, this paper used the Standard Deviation (SD) for the gait feature extraction due to its moderate sensitivity in systematic testing.The Skewness is a measure of input signal asymmetry while the Kurtosis measures whether the input signal distribution is sharper or flatter than the normal distribution. As the Standard Deviation is able to reflect the magnitude of the input, this paper used the Skewness (SKE) for the subsequent gait feature extraction.The Slope Count (SC) and the Slope Zero Crossing (SZC) can be used to measure the trend change of the input signal, which are essential to the identification of variable speed walking such as acceleration and deceleration. Therefore, these two parameters are all used in the subsequent gait feature extraction.In order for the prosthetic knee to follow the thigh motion of the amputee, this paper proposed a novel neural-network-based gait database matching method to achieve the specific control, as shown in The real-time attitude of the prosthetic knee is transmitted to the host PC via the Arduino board, and the current speed ratio and the gait percentage of the prosthetic knee are calculated by the Gaussian mixture model-based gait classifier, as shown in The workflow of the Gaussian mixture model-based gait classifier (GMMGC) is as follows: the GMMGC first obtains the current knee angle and the last four samples of previous knee angles from the Arduino board and the Previous Gait Register (PGR), respectively. Then the Mean (MN), the Skewness (SKE), the Slope Count (SC), and the Slope Zero Crossing (SZC) features of the total five input samples are extracted. Because the amplitude of different feature differs , each feature is normalized via the Min-Max Scaling method by extracting their previous four samples from the PGR to ensure that each feature will have the same impact on the final classification result. In order to improve the efficiency of the gait classification, this paper used the Principal Component Analysis (PCA) to perform the dimension reduction of input knee gait features. To ensure all the variability of the dimensionally reduced features is larger than 90%, the first three components are adopted for the subsequent gait classification.To perform reliable and fast classification of input gaits, this paper compared some typical multi-label classification methods  and some typical unsupervised classification methods  via a training data  in MATLAB. The results show that the Gaussian kernel Support Vector Machine  achieved >95% classification accuracy, while the K-Nearest Neighbor achieved >99.99% classification accuracy. However, the K-Nearest Neighbor showed a low posterior probability. As for the unsupervised classification methods, the Gaussian mixture model achieved the highest >85% classification accuracy with a high posterior probability. Because the Gaussian kernel Support Vector Machine used the One VS. One coding, there will be After the classification of the input knee gait, the target gait profile can be obtained by matching the classification result with the Ideal Gait Database (IGD). The ideal gait profiles corresponding to different walking speeds in the IGD are normalized for adaptability. Then the gait percentage can be quickly calculated by using the intraclass correlation coefficient (ICC), i.e., compare a small time window of the walked gait with the target gait profile to find the gait cycle percentage, as shown in Equation (1):In Equation (1), Then by using the calculated gait percentages and previous four percentages in the target gait profile as inputs , the neural network will output the predicted gait percentage of the prosthetic knee. Finally, by matching the predicted gait percentage with the target gait profile, the predicted prosthetic knee posture is generated. The host PC will then send corresponding motor instructions to the motor system to actuate the prosthetic knee.For example, in a theoretical control state as shown in To avoid abnormity in practical conditions, measured The prosthetic knee initiation: when the device is turned on, it will automatically home to the state of zero knee flexion. When the flexion angle of the prosthetic knee reaches zero, the prosthetic knee will stand by until the measured speed ratio The prosthetic knee termination: when the measured speed ratio is lower than the termination threshold , as shown in −10, which proves that the designed neural network has the right prediction and correction over the training data.By using 500 separate sets of validation data and another 500 different sets of test data, the regression of training, validation, and test are shown in The overall control flowchart of the GFB prosthetic knee is shown in To study the prediction effect of the proposed BP neural network-based gait detection method, this paper compared it with typical gait detection methods: Adaptive Network-based Fuzzy Inference System (ANFIS), Gaussian process regression (GPR), and Support Vector Machine (SVM). The parameters of ANFIS for comparison are as follows: Numbers of clusters: 5; Radii of clusters: 0.5; Range of influence of the cluster center: 0.5; Squash factor: 1.25; Acceptance ratio: 0.5; Rejection ratio: 0.15. The parameters of GPR for comparison are as follows: Explicit basis in the GPR model: constant; Kernel function: squared exponential kernel. The parameters of SVM for comparison are as follows: Kernel function: linear; Kernel scale: 1.The BP neural network, the ANFIS, the GPR, and the SVM are all trained with the same training dataset. To evaluate the prediction ability of these methods, a test reference gait and its noised one are utilized to check the prediction accuracy. The results are depicted in It can be noticed from To evaluate the prediction reliability of the proposed BP neural network, this paper studied the continuous gait prediction effects of several test subjects. In order to verify the anti-failure performance of the proposed method, the thigh IMU and the shank IMU in the data recording devices are randomly disabled within one or two cycles to simulate the malfunction of the prosthetic attitude measurement system. To experimentally verify the neural network control, an able-bodied subject  participated in the test of the GFB prosthetic knee via a particular connecting apparatus by flexing his knee to 90°, as shown in After multiple times of training, the volunteer becomes familiar with wearing the GFB prosthetic knee to walk. The volunteer is asked to walk at his habitual speed (about 1.2 m/s). It can be obtained from To further demonstrate the effectiveness of the neural network control system, the volunteer is asked to vary his walking speed by acceleration and deceleration. It can be obtained from This paper presented a speed-adaptive neural network control of a powered GFB prosthetic knee. The control system consists of the attitude measurement system and the neural network motor control system, which are run in parallel to reduce hardware/software requirements. To enable speed-adaptive asymmetrical walking gaits such as acceleration or deceleration, this paper only uses sensor data from the prosthetic knee, rather than data from both the sound and unsound legs. The current prosthetic knee gait can be recognized by the Gaussian mixture model-based gait classifier. Unlike conventional prediction method, the neural network in this paper does not output the predicted prosthetic knee motion, but the predicted gait percentage of the prosthetic knee. The specific prosthetic knee motion is obtained by matching the predicted gait percentage with the ideal gait database that stored gait data of different speeds. The benefit of doing this is that the size of the neural network can be reduced, and gait diversity can be increased and modified easily for customization. Level-ground walking experiments are conducted via the able-bodied subject. The results show that the neural network control system can successfully realize the speed-adaptive level-ground walking of the GFB prosthetic knee. Future work of this paper will extend the speed-adaptive control system suitable for ramp and stair walking, as well as the embedded transplantation into the development board.


A robust and accurate aircraft pose estimation method is proposed in this paper. The aircraft pose reflects the flight status of the aircraft and accurate pose measurement is of great importance in many aerospace applications. This work aims to establish a universal framework to estimate the aircraft pose based on generic geometry structure features. In our method, line features are extracted to describe the structure of an aircraft in single images and the generic geometry features are exploited to form line groups for aircraft structure recognition. Parallel line clustering is utilized to detect the fuselage reference line and bilateral symmetry property of aircraft provides an important constraint for the extraction of wing edge lines under weak perspective projection. After identifying the main structure of the aircraft, a planes intersection method is used to obtain the 3D pose parameters based on the established line correspondences. Our proposed method can increase the measuring range of binocular vision sensors and has the advantage of not relying on 3D models, cooperative marks or other feature datasets. Experimental results show that our method can obtain reliable and accurate pose information of different types of aircraft. Aircraft pose estimation is a necessary technology in the field of aeronautical engineering and accurate pose parameters of the aircraft provide the fundamental information needed in many aerospace tasks, such as flight control system testing ,2,3, auxIn general, the visual sensors used for aircraft pose estimation include RGB-D (Red Green Blue-Depth) sensors, binocular vision sensors and monocular cameras. RGB-D sensors provide RGB images and depth information ,10 for pThe RGB-D sensor utilizes an active light source for the close-range sensing of 3D environments. Among visual based pose estimation methods, a simultaneous localization and mapping (SLAM) technique is often used to process the obtained image sequences efficiently ,20,21,22Binocular vision systems estimate the pose of the target through feature correspondence and 3D structure computation ,33,34, aCompared to the state-of-the-art pose estimation systems based on RGB-D and binocular vision sensors, the monocular vision system is relative light in weight, computationally less expensive, and is also suitable for long-range pose estimation. For an aircraft with an on-board monocular camera, its pose can be determined by sensing the 3D environments. An on-board monocular vision system based on cooperative target detection was established in  for aircHowever, for external monocular vision systems which estimate an aircraft’s absolute pose using its 2D projected images, these on-board vision techniques are not applicable because of the unpredictable motion of aircraft and unknown scale factor. It is quite challenging to obtain accurate pose information from a single 2D image of an aircraft, especially for long distance measurements where the size of the aircraft is relatively small compared to its average distance to the monocular camera.Prior knowledge about an aircraft’s structure, such as 3D models or synthetic image datasets, is usually needed to facilitate the estimation of aircraft pose parameters. For monocular model-based methods, the pose parameters are calculated by minimizing the distance between corresponding features extracted from 2D images and the 3D model. The authors in  exploiteTo reduce the complexity of model-based pose estimation, features describing the aircraft’s structure are proposed and 3D pose is estimated by pattern matching. Reference  extracteAlthough a lot of features-based methods have been presented to reduce the complexity of model-based pose estimation, 3D models are still required to obtain high-quality synthetic aircraft image datasets which is necessary for reliable feature matching and accurate pose estimation. The usage of detailed 3D models or feature matching is storage- and time-consuming, and for different types of aircraft, it is necessary to prepare corresponding 3D models and/or feature datasets which also reduces the flexibility and efficiency.In this paper, a vision system using two monocular cameras with a large baseline is presented for pose estimation of model-unknown aircraft. In practical applications, our vision system uses two cameras located on the ground to track a free flight aircraft simultaneously and estimate its real-time 3D pose. For every captured image, the camera orientation parameters are recorded and known. Compared to other vision methods based on detailed 3D models or feature datasets, our approach only utilizes a pair of 2D images captured at the same time to estimate the 3D pose of an aircraft robustly and accurately. Some prior assumptions about the general geometry structure of aircraft are exploited to recognize aircraft’s main structure and obtain 3D/2D feature line correspondences for pose solutions. With a very large baseline distance, our system can perform long range measurement and enhance the accuracy of pose computation. By utilizing common structure features of aircraft, the line correspondences are established without relying on 3D models and there is no need for stereo matching which is very difficult for wide-baseline images.Compared with other systems using binocular stereo vision sensors for aircraft pose estimation, our vision system enables binocular vision sensors to achieve a larger measuring range by adopting a much wider baseline configuration, and our method can estimate the aircraft pose from the wide-baseline image pairs robustly and accurately. Moreover, our visual sensor system is more flexible in estimating the pose of different types of aircraft as it does not need to preload detailed 3D models or feature database.In our pose estimation method, the aircraft’s structure in a single image is represented by line features, and the spatial and geometric relationships between extracted line features are explored for the extraction of aircraft structure. The line features are detected by a line segment detector (LSD) algorithm, and a mean-shift algorithm is used for locating the aircraft’s center. Based on the orientation consistency constraint of line features distributed on the fuselage, a density-based clustering algorithm is used to determine the fuselage reference line by grouping line segments with similar direction. Through the analysis of aircraft’s common wing patterns, bilateral symmetry property of aircraft, coplanar wings, and fuselage reference line are assumed. Under these prior assumptions, the line segments which correspond to the wing leading edges are extracted by vector analysis. After identifying the projected fuselage reference line and wing edges in a pair of images, line correspondences between the aircraft’s 3D structure and its 2D projection are established and the pose information is obtained using planes intersection method. As no detail 3D models and/or feature datasets are required, our vision system provides a common architecture for aircraft pose estimation and can estimate the pose of different aircraft more flexibly and efficiently.The remainder of the article is organized as follows: In this section, we briefly introduce the coordinate systems and pose estimation problem. Four major coordinate systems used for pose estimation are shown in z axis of the camera; the horizontal axis (u) of the image frame is parallel to the x axis of the camera coordinate frame in the right direction, and the vertical axis (v) of the image frame is parallel to the y axis of the camera coordinate frame in the right-handed coordinate system.The camera is calibrated with respect to the world coordinate system and its orientation parameters corresponding to the captured image are recorded, i.e., the interior and exterior orientation parameters of the camera are considered to be known.x axis points along the fuselage reference line; the z axis is perpendicular to the plane containing the fuselage reference line and wing leading edge lines, direct to down side; and the y axis is perpendicular to the bilateral symmetry plane of the aircraft in the right-handed coordinate system. As is shown in The schematic representation of the pose estimation problem is shown in The location and orientation of the body frame with respect to the world frame represent the 3D pose parameters of the aircraft, which are the solutions for the pose estimation problem. For many aircraft, the bilateral symmetry is their inherent property and their wings are coplanar with the fuselage reference line. Moreover, line features distributed along the fuselage are approximately parallel to the fuselage reference line. These geometry structure features are exploited in our method to facilitate aircraft structure recognition and pose estimation.For the absolute pose estimation problem, the orientation and location of the aircraft was specified using a rotation matrix To obtain the solution for the rotation matrix and translation vector, it was necessary to establish the correspondences between features in the body frame and the world frame. In our method, the feature correspondences were found by structure extraction. With some prior assumptions about the geometric relationships between the fuselage reference line and wing leading edges, our pose estimation method first extracted the fuselage reference line and the wing leading edge lines from a given pair of images to establish the line correspondences, then used a planes intersection method to obtain the pose information. The details of the structure extraction and plane intersection algorithm are described in the following sections.A novel structure extraction method is presented to detect the fuselage reference line and wing leading edge lines from a 2D image without using 3D models, cooperative markers, or other feature datasets. For the pose estimation of a free flight aircraft at long distance, it was difficult for feature points to establish reliable correspondences due to the optical blur, ambiguities, and self-occlusions. In these cases, line features were more robust and accurate and less affected by the unpredictable motion of the aircraft. To acquire accurate and robust pose information from 2D images, line segments were used for structure description and recognition.Line features distributed along the fuselage were approximately parallel to the fuselage reference line;Wing leading edge lines were bilaterally symmetrical and coplanar with fuselage reference line.In our algorithm, some assumptions about the generic geometry structure features of the aircraft are explored. The most important assumptions used for structure extraction are as follows:These two assumptions reflect the geometry relationships between line features on the aircraft. Weak perspective projection , which is a first order approximation of the perspective projection, was also assumed. As the size of the aircraft is small compared to its average distance from the monocular camera, weak perspective assumption holds approximately.Based on these assumptions, the geometric configuration of line features, such as geometric consistency  and linear combinations of vectors (directed line segments) were exploited to extract the main structure (fuselage and wing leading edges) of the aircraft and establish line correspondences. Robust and accurate structure extraction can lay a good foundation for pose estimation.The line features were detected in 2D images using line segment detector (LSD) algorithm , which iOur structure extraction method leveraged spatial, length and orientation consistency constraints of the line features to extract the fuselage reference line. Firstly, the spatial consistency constraint was used to identify the centroid of the aircraft and combined with length constraint to remove irrelevant line features; then, orientation consistency-based line clustering analysis was performed to recognize the aircraft’s structure and estimate the direction of fuselage reference line.In actual applications, such as flight test, take-off and landing, our vision system tracked one single aircraft and captured its 2D projected image, so the image area that contained the aircraft was a saliency region in which detected line features are concentrated. Compared to unrelated line segments distributed in the background, line segments of the aircraft were close to each other which formed a high-density region of line features. Based on this spatial consistency constraint, mean shift  algorithMean shift is a non-parametric mode-seeking technique which locates the maxima of a density function iteratively. In our case, it was suitable for indicating the density model of line features and determining the centroid of the aircraft. Given an initial estimation, mean shift algorithm used a kernel function to determine the weight of adjacent elements and re-estimate the weighted mean of the density. The line feature set However, there were still some unrelated line features left in the line feature set In general, the lengths of the aircraft’s main structures (fuselage and wings) were larger than the lengths of other parts such as tail or other external mounts, so it is reasonable to exclude shorter line segments from line feature set. Moreover, the length consistency constraint improved the accuracy of structure extraction. For a line feature detected by the LSD algorithm, the uncertainty of its direction increased with the decreasing length, it means that a small pixel coordinate error of the endpoint causes greater orientation error for shorter line segments. After acquiring the estimation of the aircraft’s center and removing irrelevant line features, the orientation consistency constraint was utilized to estimate the direction of the fuselage reference line. Under weak perspective projection, if the line segments are parallel to each other in 3D space, then this geometric constraint of corresponding line features is invariant after the 3D world to 2D image transformation.For many aircraft, line segments distributed along the fuselage are approximately parallel to the fuselage reference line, and the parallel world lines are transformed to the parallel image lines by the weak perspective camera. In the projected 2D image, angle values of line features along the fuselage were close to each other and concentrated around the orientation of the fuselage reference line . Moreover, compare with other parts of the aircraft, fuselage contains most parallel line segments. By performing parallel line clustering to represent the aircraft structure, the orientation of the fuselage reference line was determined.The input of the parallel line clustering analysis was the directions of line features in (DBSCAN)  algorithCore points . By observing the wing patterns of many aircraft, the validity of the assumption was confirmed. Based on the lateral symmetry of the aircraft, the wing leading edges were expressed as two symmetric vectors of equal length. With the additional condition that the wings were coplanar with the fuselage reference line, the sum of these two vectors was parallel to the fuselage reference line.Since the geometric properties of linear combinations of vectors are invariant under weak perspective projection, after the aircraft in three-space is projected onto the image plane by a weak perspective camera, the addition of two directed wing leading edges in the image is still parallel to the fuselage reference line. The following is a proof of this geometric invariant property.In the world coordinate frame, two vectors representing the wing leading edges were denoted by As By substituting Equation (5) into Equation (4), the geometric invariant property is obtained:Since Based on this geometric invariant property, the wing leading edge lines in the image are extracted using vector calculations. We first vectorized line segments in The parameter The parameter The parameter The parameter The parameter Based on the above analysis, By solving Equation (8), the lines which the wing leading edges belong to were detected in the image and the correspondences between line features in the image and aircraft structure in 3D space are established. The results of wing leading edge extraction are shown in While the wing trailing edge could also be extracted by adjusting the weights in Equation (7), line features of the wing trailing edges were vulnerable to the interference from some other parts of the aircraft. In After extracting the wing leading edge lines and establishing the line correspondences, the planes intersection method was used for aircraft pose estimation. The geometric configuration of the planes intersection method is explained in Although the fuselage reference line was extracted robustly and used to facilitate the wing leading edge extraction, the parameters which identify the fuselage reference line may be less accurate. To perform high-precision pose estimation, only wing leading edge lines in an image pair were utilized in intersection calculation.In Our planes intersection method considered the lines as infinite and the endpoints of line features were not used for pose estimation. Let the 3D line The plane Here, the line x axis of the body frame is parallel to y axis of the body frame is parallel to Let on (SVD) , the rotTo avoid this reflective ambiguity, an initial pose constraint is used in our method. The approximate orientation of the aircraft is specified in the initial frame of the image sequence, and the orientation information of the current frame was used in the next frame. In practice, the approximate value of the roll angle of the aircraft was provided, or the approximate position of one wing tip was marked on one image of the initial image pair. It was easy to provide this pose constraint in the application scenarios of our pose estimation method, for example, take-off, landing, and flight testing.The point of intersection of rization .As the normalized vectors Algorithm 1. Pose estimation based on geometry structure features and line correspondences.Input:A pair of images captured at the same time, the parameter matrices of the two cameras Output:The pose information of the aircraft.Step 1Detect line features in images using the LSD algorithm;Step 2Estimate the location of the aircraft’s center in images and eliminate irrelevant line features based on the spatially and length consistency constraints;Step 3Determine the orientation of fuselage reference line using parallel line clustering;Step 4Extract the wing leading edge line in the image pair based on generic geometry structure features of aircraft and vector analysis;Step 5Acquire the pose information of the aircraft by using the planes intersection method.The pipeline of our pose estimate method is summarized in this section, as is shown in Algorithm 1.The effectiveness and accuracy of our method have been evaluated by experiments on real and synthetic images. The performance of our structure extraction method was verified by real images of various types of aircraft, and synthetic images of different aircraft at different viewpoints and real images captured in ground laboratory experiments were used to measure the accuracy and robustness of the pose estimation algorithm. Our pose estimation method was implemented by MATLAB based on a laptop with an Intel Core i7 CPU with a 2.80 GHz processor and 8.00 GB of RAM.This section provides qualitative evaluation of our structure extraction method by using real images downloaded from the Internet. There were 50 total real images which contain a variety of aircraft, and some of these images were difficult as they contained self-occlusion, cluttered backgrounds, disruption of external mounts, or perspective effects.Since only wing leading edges were used to solve the pose estimation problem, the performance of the structure extraction method was evaluated by the correctness of the extracted wing leading edge lines. Our approach can cope with a wide variety of aircraft flexibly and does not rely on detailed 3D models or cooperative marks. In the experiment, the wing leading edge lines in 42 of the 50 images were correctly extracted. Some of the results are shown in As is shown in The leading edge of the wing is not a straight line see a, i.e., Portions of the wings are not visible in aircraft images from some viewpoints see b. Under Some parts of the aircraft  or other interference factors (changing weather or light conditions) affect the line detection, i.e., some unreliable line features are extracted and affect structure extraction see c.Some incorrect wing extraction results of our approach are shown in Although our algorithm was developed for long-range measurements in which the assumption of weak perspective projection holds, the experimental results show that our algorithm can tolerate a certain perspective effect. The assumptions about the generic geometry structure feature of aircraft were also not expected to hold strictly. Even if the wing leading edge lines were not strictly coplanar with the fuselage reference line, our method is still capable of correctly extracting the wing leading edges.Synthetic images of different aircraft and real images captured in ground laboratory experiments have been used to test the performance of the method. In experiments, our pose estimation algorithm is compared with Li’s method  which isThe algorithms were evaluated in pose estimation errors. For any given ground truth aircraft pose  are visible, the wing leading and trailing edge lines could be detected by modifying the weighted function based on the geometry invariant constraints. Under this condition, if the information about the angle between wing edges and the fuselage (wing patterns) are known, then the perspective-and-line (PnL) method could be used to estimate the aircraft pose and only a monocular camera is required. Developing a more robust structure extraction method and a monocular vision system for aircraft pose estimation will be the focus of our future research.


The fingerprint method has been widely adopted in Wi-Fi indoor positioning because of its advantage in non-line-of-sight channels between access points (APs) and mobile users. However, the received signal strength (RSS) during the fingerprint positioning process generally varies due to the dissimilar hardware configurations of heterogeneous smartphones. This difference may degrade the accuracy of fingerprint matching between fingerprint and test data. Thus, this paper puts forward a fingerprint method based on grey relational analysis (GRA) to approach the challenge of heterogeneous smartphones and to improve positioning accuracy. Initially, the grey relational coefficient (GRC) between the RSS comparability sequence of each reference point (RP) and the RSS reference sequence of the test point (TP) is calculated. Subsequently, the grey relational degree (GRD) between each RP and TP is determined on the basis of GRC, and the K most relational RPs are selected in accordance with the value of GRD. Finally, the user location is determined by weighting the K most relational RPs that correspond to the coordinates. The main advantage of this GRA method is that it does not require device calibration when handling heterogeneous smartphone problems. We further carry out extensive experiments using heterogeneous Android smartphones in an office environment to verify the positioning performance of the proposed method. Experimental results indicate that the proposed method outperforms the existing ones no matter whether heterogeneous smartphones are used. At present, location-based service requirements have rapidly increased. Nowadays, the global navigation satellite system (GNSS) can satisfactorily addressed location service requirements in outdoor environments. However, GNSS cannot meet the indoor positioning requirement because of the fading signal and multipath effect in indoor environments ,2,3. SevSmartphones are becoming highly intelligent with the development of science and technology. Smartphones can help in solving many problems, such as navigation and positioning, which have become a part of people’s lives. The use of a smartphone-based positioning system is convenient because of the popularity and abundance of various embedded sensors in smartphones. On this basis, several studies investigated smartphone-based indoor positioning and various solutions have been presented in the literature ,23,24. CHaeberlen et al.  and KjærA type of device calibration-free method, which does not need to collect training data for device calibration, utilises the RSS difference between APs or locations to mitigate the effect of using heterogeneous smartphones. Shu et al.  proposedAnother type of device calibration-free method uses the similarity between fingerprint and online RSS data instead of the conventional Euclidean distance. Han et al.  and CasoIn addition, other device calibration-free methods were presented. Yang et al.  built RSSince the grey system theory was proposed in 1982, it has been widely applied in social and economic fields because of its advantages in evaluating complex systems with multiple criteria and multiple factors, such as investment returns of various sectors of national economy, analysis of regional economic advantages, and adjustment of industrial structure ,50. GreyIn this method, firstly, average RSS data received from all APs at the TP form a reference sequence and average RSS data received from all APs at each RP form each comparability sequence. Then, the grey relational coefficient (GRC) between the RSS comparability and reference sequence is calculated by the RSS difference between curves, which is used to describe the correlation between them. Next, GRD is calculated by the GRC between reference and comparability sequences. The closer the comparability sequence curve is to the reference sequence curve, the larger the GRD between them is. Subsequently, the K most relational RPs are selected. Finally, the user location is determined by weighting K most relational RPs that correspond to the coordinates. This above-proposed method guarantees the completeness of RSS information. Moreover, the method is applicable to the nonlinear relationship between vectors and has no requirement for typical distribution regulation of vectors. This method avoids the fingerprint matching errors of conventional positioning methods by using similarity of comparability and reference sequences. The proposed method can adapt to the changes of RSS data between two vectors better than other fingerprint methods in terms of similarity because the GRD is determined by the RSS difference between curves, which does not require complete similarity between curves. Another advantage of the method is that it can avoid missing the raw RSS fingerprint information compared with other RSS difference-based fingerprint methods. In addition, the proposed GRA method also has the advantages of less calculation and of a simple principle. The positioning performance of the proposed method is improved whether heterogeneous smartphones are used.The remainder of this work is organised as follows: In this section, the RSS difference of heterogeneous smartphones is analysed and the influence of heterogeneous smartphones on positioning accuracy is discussed. The result indicates that the received RSS data have large differences and that the positioning performance of conventional fingerprint algorithms is degraded when heterogeneous smartphones are used.An experiment is carried out in a conference room to analyse the influence of heterogeneous smartphones on RSS. Two heterogeneous smartphones are used to measure the RSS data from the same AP and location. A total of 300 RSS data are collected within 5 min at a frequency of 1 Hz. Another experiment is designed to further analyse the difference of the collected RSS data from various APs in the same location using heterogeneous smartphones. Four heterogeneous smartphones are used to collect RSS data from 16 APs at the same location. RSS data are collected for 5 min at a frequency of 1 Hz. In the previous experiment, the RSS value has differences when heterogeneous smartphones are used. Thus, we hypothesize that differences between the received RSS in the offline and online phases affect positioning performance. An experiment is designed to verify this conjecture. The experiment is conducted in an office building of the School of Geodesy and Geomatics, Wuhan University. The office area measures 6.4 m × 12.8 m. Nine APs are distributed in the office as transmitters. A total of 28 RPs and 39 TPs are present in the office. RSS data are collected for 1 min at a frequency of 1 Hz at each RP and TP. RSS data are collected by using the same smartphone in the offline and online phases. Conventional algorithms are adopted in the experiment. The positioning performance is tested by adding several constants in the online phase on the basis of RSS data collected from TPs. A new experiment is designed to further verify the influence of heterogeneous smartphones on positioning accuracy. The experimental setting is the same as that in The RSS differences and positioning performances of conventional methods in heterogeneous smartphones are analysed. The results show that there is a difference between the RSS values and that the positioning performance of conventional methods are degraded when heterogeneous smartphones are used. Therefore, we aim to explore a novel fingerprint method to improve the positioning accuracy when heterogeneous smartphones are used. Accordingly, a GRA-based device calibration-free fingerprint method is proposed. The positioning performance of the proposed method will be analysed and compared in detail in the subsequent sections.GRA is a multifactor analysis method in grey system theory, which analyses uncertain relations between one main factor and all the other factors. GRA usually is used to determinate the relational degree according to the similarity of the geometric shape among factors. In the GRA-based fingerprint method, the GRD is used to determine user location through the similarity between the reference sequence and the comparability sequence. The closer the comparability sequence curve is to the reference sequence curve, the larger the GRD between them is.A GRA-based device calibration-free fingerprint method is proposed to mitigate the impact of heterogeneous smartphones on positioning performance. The positioning performance is improved by the proposed method when heterogeneous smartphones are used. In this section, the GRA-based device calibration-free fingerprint method is described with an overall positioning flow.Fingerprint positioning is a widely used method for Wi-Fi indoor positioning because it does not require LOS conditions. Conventional fingerprint methods consist of two phases, namely, offline and online phases. In the offline phase, the location area is divided into grids. The RPs are deployed in the grids. The RSS values at each RP are collected at preset time intervals. The collected RSS data are used as fingerprint data in the offline phase. The RSS data, MAC addresses of Aps, and location information of RPs are used to develop the fingerprint database. In the online phase, the online RSS data measured at unknown user locations are firstly collected to determine user location. Secondly, the online RSS data and fingerprint of the fingerprint database are matched according to the conventional positioning algorithms, such as WKNN and Bayesian, to find the nearby RPs. Finally, the user location is estimated according to the corresponding coordinates of the selected RPs. In this work, the GRA-based method is proposed on the basis of fingerprint positioning. In this work, our approach consists of three phases, namely, offline, online collecting, and online matching phases. Firstly, a fingerprint database is developed by using the RSS data collected at each RP. Secondly, the online RSS data are collected at a TP. Finally, GRC and GRD between reference and comparability sequences are calculated and the user location is determined. Each phase is explained in the following sections.jth AP at the ith RP. n and L represent the total amount of deployed APs and RPs in the environment, respectively.The RSS data are collected from different APs at each RP to develop a fingerprint database. Each RP location is known. A total of 60 RSS data are collected within 1 min at each RP, and the average RSS value of each RP is calculated as fingerprint data. The fingerprint data, MAC addresses of all Aps, and location information of RPs are used to develop the fingerprint database. In Equation (1), In the fingerprint data, the RSS data vector from all APs at each RP is defined as a comparability sequence. The comparability sequences of all RPs form a comparability sequence matrix. In Equation (2), The collected average RSS data from all APs at a TP is used as a reference sequence. As shown in Equation (3), jth AP at the ith RP .In grey relational analysis, reference sequence and comparability sequence cannot be compared directly since they may have different dimensions and magnitudes. Therefore, normalization is needed to translate the original sequence to a comparable sequence, which is generally dimensionless. Data normalization is also necessary when the sequence scatter range is large. According to the characteristics of data sequence, there are some normalized methods for grey relational analysis such as zero-mean normalization, min-max normalization, initialization method, and mean value method. In this paper, the units of data between original reference and comparability sequences are the same. Thus, the original sequences only need to be simply normalized by the most basic methodology to reduce the sequence scatter range, i.e., let the values of original sequence be divided by the mean of the sequence. In Equation (4), ith AP at a TP .In Equation (5), jth AP at the ith RP  and the jth AP at the TP.jth AP at the ith RP  and RSS data of the jth AP at the TP. From the geometry analysis, the GRD is the similarity between the reference and the comparability sequence curves. The closer the comparability sequence curve is to the reference sequence curve, the larger the GRD between them is. Conversely, if the curves are remarkably different, the GRD between them becomes small. Therefore, the RSS difference between curves can be used as a scale standard of GRD. Firstly, the RSS absolute differences between the elements of the comparability sequence of each RP and the elements of the reference sequence of TP are calculated in Equation (6) to determine the GRC. The maximum and minimum RSS differences are then obtained in all RSS differences between comparability and reference sequences (Equations (7) and (8)). Finally, GRC is calculated by using Equation (9).n GRCs, GRD is not easy to compare when the correlation information is scattered. Therefore, it is necessary to centralize the correlation information. Average value is a method of information centralization. That is, the GRD between the comparability and reference sequences can be quantitatively reflected by the average value of the all GRCs of each factor between the comparability sequence and the reference sequence. GRD is calculated by the GRC between reference and comparability sequences.ith RP and TP . The value of The GRD represents the level of correlation between the reference sequence and the comparability sequence. Due to the GRD between each comparability sequence and reference sequence being reflected by ith RP and the TP and ith RP in the K RPs .ith most relational RP that correspond to the coordinates .We sort amongst RPs, select the K most relational RPs by GRD between each RP and TP, and calculate the weight of the K RP with the corresponding GRD in Equation (11) to obtain the user location. In Equation (12), the user location is obtained by weighting K coordinates associated with the K most relational RPs.In this study, the experiments are conducted to evaluate the performance of the proposed fingerprint method. The evaluation metrics of the positioning performance are selected in The positioning performance of the proposed algorithm is tested. The experiment is conducted in an office building of the School of Geodesy and Geomatics, Wuhan University. The office area measures 6.4 m × 12.8 m. The AP and RP locations are shown in On the software side, the RSS data are collected by a smartphone application developed by our research team. We perform the same setting in the process of collecting RSS data at each TP and RP. Four heterogeneous smartphones are used to collect the RSS data at a rate of 1 Hz. The mean absolute error (MAE), standard deviation (STD), and root mean square error (RMSE) are selected as accuracy evaluation metrics to evaluate positioning performance. The absolute errors are calculated by the Euclidean distances between the estimated and true coordinates at TPs. Uncertainty is commonly used in measurements . Uncertan (n < N) most-counted APs.In the proposed method, some parameter settings will affect the positioning accuracy. The positioning results with different parameters, namely, the K value and number of Max AP, are analysed to determine the optimal parameters of the method for achieving the optimal positioning performance. We firstly rank all APs by using the received RSS data from each AP in a descending order at each RP to determine the number of Max AP. Subsequently, we select the Top-N APs from the descending sequence at each RP. Thereafter, the appearance frequency of each AP is counted in Top-N APs of all RPs. Finally, we select the To test the positioning performance of the proposed GRA-based fingerprint method when heterogeneous smartphones are used, we selected RSS data from a pair of heterogeneous smartphones. The positioning performance of the different fingerprint methods are compared under heterogeneous and same smartphones. Our experimental results are shown in In this work, we mainly consider the positioning performance when heterogeneous smartphones are used. Three pairs of heterogeneous smartphones are compared in To further verify the positioning performance of the proposed GRA-based fingerprint method, we selected RSS data from another pair of heterogeneous smartphones. The positioning performance of the different fingerprint methods are compared under heterogeneous and the same smartphones. The experimental results are shown in In the comparative analysis of positioning performance using two pairs of heterogeneous smartphones, the positioning accuracies of the GRA-based fingerprint method are both improved compared with other positioning methods. The positioning performance of the proposed fingerprint method outperforms other methods when heterogeneous smartphones are used. These results indicate that the proposed fingerprint method has good stability to solve the heterogeneity problem. The positioning accuracies of all methods are unsatisfactory when the test data are from the Mi 6 smartphone and when the fingerprint data are from the Honor 9 or Redmi 5plus smartphones. The results suggest that the Wi-Fi module of the Mi 6 smartphone may exhibit certain problems in the collected RSS data.The positioning performance of the proposed GRA-based method is compared with those based on RSS difference when the same and heterogeneous smartphones are used. The RSS data from Redmi 5plus are used as fingerprint data. By contrast, the RSS data from the Honor 9 and Redmi 5plus smartphones are used as the test data. Here, we illustrate the proposed method based on GRA and other existing methods, including DIFF, SSD, and HLF, based on RSS difference. The WKNN algorithm is selected to estimate positioning performance. The RSS method is added for comparison, which only uses raw RSS data. The positioning performance of the proposed method based on GRA is evaluated by comparing the positioning performance of various preprocessing methods based on RSS difference. In addition, the LCS method is compared with other methods, which uses only relative relationship information among RSS values.RSS data from heterogeneous smartphones have differences even if the RSS data are measured from the same AP and location. Thus, the impacts of using heterogeneous smartphones on the fingerprint method are further analysed. We find that the positioning performance of conventional fingerprint algorithms are degraded. GRA theory is introduced to Wi-Fi indoor positioning to address this heterogeneous smartphones problem, and a device calibration-free fingerprint method based on GRA is proposed. In the proposed method, GRC are determined by the RSS absolute difference between the elements of the comparability sequence of each RP and the elements of the reference sequence of TP and GRD is determined by the GRC between reference and comparability sequences. The TP location is subsequently determined by the GRD information between TP and each RP. The proposed method can adapt to the changes in RSS data between two vectors better than other fingerprint methods in terms of similarity. The proposed method can also avoid missing the raw RSS fingerprint information compared with other RSS difference-based fingerprint methods. All experimental results show that the mean positioning error and RMSE of the proposed fingerprint method are reduced relative to other positioning methods when heterogeneous smartphones are used, which indicates that the positioning accuracy and precision of the proposed fingerprint method are improved. The experimental results also show that the STD and STU values of the proposed GRA method is minimal among the all methods in the heterogeneous smartphones, which indicates that the proposed method has good stability and availability. Accordingly, the proposed fingerprint method based on GRA has an optimal positioning performance when heterogeneous smartphones are used. However, the area of the experiment environment is limited. We will further conduct experiments to verify reliability of the proposed GRA method in other scenarios and within large experimental areas in the next step.


Helical microswimmers have been involved in a wide variety of applications, ranging from in vivo tasks such as targeted drug delivery to in vitro tasks such as transporting micro objects. Over the past decades, a number of studies have been established on the swimming performance of helical microswimmers and geometrical factors influencing their swimming performance. However, limited studies have focused on the influence of the hydrophobicity of swimmers’ surface on their swimming performance. In this paper, we first demonstrated through theoretical analysis that the hydrophobicity of swimmer’s surface material of the swimmer does affect its swimming performance: the swimmer with more hydrophobic surface is exerted less friction drag torque, and should therefore exhibit a higher step-out frequency, indicating that the swimmer with more hydrophobic surface should have better swimming performance. Then a series of experiments were conducted to verify the theoretical analysis. As a result, the main contribution of this paper is to demonstrate that one potential approach to improve the helical microswimmers’ swimming performance could be making its surface more hydrophobic. Due to the recent advances in microfabrication brought by the rise of new manufacturing methods ,2,3,4,5,E. coli bacteria. The rotary motor of E. coli bacteria drives its bundle of flagella to rotate in a helical pattern, and in this way generates propulsion. Nowadays helical microswimmers adopt the same swimming mechanism as E. coli bacteria, substituting the internal rotary engine with external magnetic field.Purcell studied on the swimming techniques of natural swimmers in his 1977 work [In the past decades, researchers have designed and fabricated various types of helical microswimmers with different head shapes, magnetic positioning, as well as geometrical parameters of head and tail. In addition, a number of studies have been established on the influence of those variables on the swimming performance of helical microswimmers. In 2015, Xu et al. proposed that the magnetic positioning of helical microswimmers affects the number of possible magnetization directions and further results in different rotational propulsion characteristics , and that the head shape only affects the values of cut-off frequency with no influence on the rotational propulsion characteristics . HoweverThis paper aims to study the influence of surface material hydrophobicity of miniature helical swimmers on their swimming performance at low Reynolds number, especially their step-out frequency values. We first presented a theoretical analysis on how the hydrophobicity of surface material would affect its swimming performance, and then verified it through a series of experiments. Experiments are conducted in glycerine solutions of different fluid viscosity, using two helical swimmers which are identical except for their surface material hydrophobicity.In the following parts of the paper, M denotes the average magnetization of the helical microswimmer, and B denotes the flux density of the applied magnetic field. Note that both equations rely on the assumption that the spatial changes in magnetic field must be small enough so that it can be regarded as uniform throughout the volume of the helical microswimmer.In the experiments, a uniform rotating magnetic field is applied to drive and control the helical microswimmer. Within such a magnetic field, the magnetic torque and the magnetic force exerted on the swimmer are given by :(1)TB=ϑMSince the applied magnetic field generated by the 3D Helmholtz coil system is uniform, the magnetic force The magnetic torque ion drag , which wV. The drag force a being the filament radius of the helical tail. For a microswimmer with a helical tail of length L, radius R, and pitch angle D and the total fluidic drag torque T exerted on it are derived by integrating the forces and torques on infinitely small elements along the length of its helical tail. They are given by [U and U denoting the axial velocity and When modeling the propulsion of a spermatozoon, Gray and Hancock  considergiven by :(7)D=UdF=CAccording to Newton’s law of viscosity , the shegradient , the velAs is discussed in previous sections, a pair of opposing torques are controlling the swimming behavior of the helical microswimmer. The driving torque is the magnetic torque, which tends to drive the swimmer to rotate in sync with the rotating magnetic field; while the resistive torque is the fluidic drag torque, which consists of form drag torque and skin friction drag torque.While swimming in viscous fluid, a helical microswimmer exhibits a step-out frequency, which marks a crucial turning point of its swimming behavior . When thAs is discussed in In the experiments, the uniform rotating magnetic field used to actuate and control the helical microswimmers is generated by a 3D Helmholtz coil system shown in An associative camera  is mounted on the top of the 3D Helmholtz coil system, in order to provide overviews for controlling the microswimmer, and to record videos for further observation, calculation, and analysis. The videos are recorded at a framerate of 50 frames/s.The magnetically actuated miniature helical swimmers studied in the following experiments are made by Titanum. Each swimmer has v/v) HF and the mixture of Dimethyl sulfoxide (DMSO) and ethanal with a volume ratio of 1. The two electrodes were connected with a DC power supply with a voltage of 45 V for 1 5 h. After the reaction, the titanium helical swimmer with prepared surface TiO2 nanotube were washed with water. The dried sample was annealed at The helical swimmers with different hydrophobicity were prepared through a potentiostatic anodization method with a two-electrode cell and followed by surface modification. Briefly, the titanium helical swimmer was washed with piranha solution for several minutes and dried in a drying oven before use. Then, titanium helical swimmer was used as the working electrode and the graphite rod was used as the counter electrode. Both electrodes were immersed in electrolyte solution contains A human spermatozoon moves with Five sets of experiments were undertaken, with the variable among them being fluid viscosity. The glycerin percent weights were set to 85%, 92%, 96%, 98% and 100%, which resulted in an approximate arithmetic progression in fluid viscosity: 109, 310, 624, 939, 1410 , respectively. In each set of experiment, the swimmers S-H1 and S-H2 were separately put into the glycerin solution and actuated by a rotating magnetic field, which was generated by the 3D Helmholtz coils system. They are initially floating in the viscous liquid, and are actuated by a rotating magnetic field, which was generated by the 3D Helmholtz coils system. The propulsion velocity and step-out frequency are calculated before they sink down to the bottom. The rotation frequency of the magnetic field was initially set to 1 Hz, and was discretely increased by 1 Hz at a time. For each frequency value, the swimming behavior of the helical swimmers was recorded by the camera mounted on the top of the 3D Helmholtz coil system.By fixing the recording frame rate to be 50 frames/s throughout the experiments, it is straightforward to establish a one-to-one correspondence between the time and the number of frames. The swimming velocity was measured by counting the number of pixels traveled along by the swimmer during a certain number of frames, followed by a unit conversion from pixels per frame into millimeters per second. In order to reduce experimental errors, measurements were conducted in five randomly selected disjoint time intervals, and the estimated velocity corresponding to a specific rotation frequency was obtained by averaging the five velocity values.The propulsion velocity of S-H1 and S-H2 as a function of rotation frequency in different viscous liquids are presented in The step-out frequency is determined by identifying the frequency value corresponding to the peak of the curve, which is the last point where magnetic torque counterbalances fluidic drag torque. Under each of the five choices of glycerin percent weights, a larger step-out frequency is observed for S-H2, compared with that of S-H1. The difference in step-out frequency between the two helical swimmers verified that the hydrophobicity contributes positively to fluidic drag force (torque), so greater hydrophobicity results in smaller fluidic drag force (torque), and thus higher step-out frequency. Note that this result also matches the theoretical expectation obtained from Equation , which sAs is discussed in To reveal the impact of fluid viscosity on step-out frequency, the step-out frequencies of the two helical swimmers with respect to fluid viscosity are depicted separately in a line graph, as is shown in Equation .In this paper, we studied the swimming performance of miniature helical swimmers with different surface material hydrophobicity, and focused our attention on the difference in step-out frequency. Two millimeter-scaled swimmers were fabricated, with differently hydrophobic surface. Their swimming behavior was in low Reynolds number regime. Based on the balance between driving magnetic torque and resistive fluidic drag torque, as well as the fact that hydrophobicity negatively contributes to the shear stress, we raised a theoretical expectation that the less hydrophobic swimmer would exhibit a smaller step-out frequency. Experimental results obtained from glycerin solutions with different viscosity matched the theoretical analysis: in each set of experiment, the less hydrophobic swimmer showed a smaller step-out frequency compared with the more hydrophobic one. It has also been demonstrated that the influence of fluid viscosity on step-out frequency is negative: greater viscosity results in smaller step-out frequency. In future applications of helical microswimmers, modifying surface material hydrophobicity could be considered as a possible solution to improve the swimming performance.


As one of the most essential technologies, wireless sensor networks (WSNs) integrate sensor technology, embedded computing technology, and modern network and communication technology, which have become research hotspots in recent years. The localization technique, one of the key techniques for WSN research, determines the application prospects of WSNs to a great extent. The positioning errors of wireless sensor networks are mainly caused by the non-line of sight (NLOS) propagation, occurring in complicated channel environments such as the indoor conditions. Traditional techniques such as the extended Kalman filter (EKF) perform unsatisfactorily in the case of NLOS. In contrast, the robust extended Kalman filter (REKF) acquires accurate position estimates by applying the robust techniques to the EKF in NLOS environments while losing efficiency in LOS. Therefore it is very hard to achieve high performance with a single filter in both LOS and NLOS environments. In this paper, a localization method using a robust extended Kalman filter and track-quality-based (REKF-TQ) fusion algorithm is proposed to mitigate the effect of NLOS errors. Firstly, the EKF and REKF are used in parallel to obtain the location estimates of mobile nodes. After that, we regard the position estimates as observation vectors, which can be implemented to calculate the residuals in the Kalman filter (KF) process. Then two KFs with a new observation vector and equation are used to further filter the estimates, respectively. At last, the acquired position estimates are combined by the fusion algorithm based on the track quality to get the final position vector of mobile node, which will serve as the state vector of both KFs at the next time step. Simulation results illustrate that the TQ-REKF algorithm yields better positioning accuracy than the EKF and REKF in the NLOS environment. Moreover, the proposed algorithm achieves higher accuracy than interacting multiple model algorithm (IMM) with EKF and REKF. The most popular worldwide positioning system is the Global Positioning System (GPS), but it is not a viable option in some areas, especially in indoor environments, due to the fact that GPS positioning is based on multiple satellites and the positioner cannot acquire accurate signals in indoors environments because of obstructions such as reinforced concrete. GPS technology also relies too much on terminal performance, that is, satellite scanning, acquisition and positioning operations are integrated into the terminal, which results in low positioning sensitivity and high power consumption of the terminal. Meanwhile, wireless sensor networks (WSN) for indoor localization have been drawing more and more attention in recent years. A WSN is an intelligent measurement and control network system composed of a large number of ubiquitous microsensors with communication and computing capabilities. The sensor nodes are intensively deployed in unmanned monitoring areas and can independently complete assigned tasks according to the environment. Because of these excellent characteristics, WSNs have been widely applied to various applications, such as the military field, emergency rescue, environmental monitoring, health care and industrial applications.As the basic application in WSNs, localization techniques play a key role. Two types of nodes are involved in the positioning process, one are beacon nodes, the other are mobile nodes. Nodes that obtain coordinate information beforehand are called beacon nodes, while the nodes with unknown position coordinates is denoted as mobile nodes. The position of mobile nodes can be estimated by different localization algorithms. According to whether the distance between beacon nodes and mobile nodes is measured during the positioning process, localization algorithms are normally divided into range-based and range-free algorithms. Most of the range-free algorithms are only at the stage of theoretical research and are inadequate to be applied to practice use because they assume uncertainties that cannot be satisfied in reality. The range-based localization algorithms mainly include time difference of arrival (TDOA), time of arrival (TOA), received signal strength (RSS), and angle of arrival (AOA). All the methods mentioned above are widely used in positioning applications.If the distance or angle measurement between a beacon node and a mobile node is acquired from line-of-sight (LOS) propagation, i.e., the radio propagates along a straight line, the position of the unknown node can be estimated by using conventional algorithms. However, when the direct path between nodes is blocked by obstacles, radio can only reach the receiving terminal after reflection and diffraction, leading to a long propagation path. This phenomenon is known as non-line-of-sight (NLOS) propagation, which occurs especially in indoor environments. When the channel is in NLOS, it will result in a positive bias of the measured distance.In order to tackle this issue, many NLOS mitigation algorithms are proposed. How to mitigate the NLOS errors is still a difficult task. In this paper, a fusion algorithm is proposed based on robust extended Kalman filter and track quality method. First to make the extended Kalman filter robust, we need to merge the state and the observation equations into a single linear regression model and then utilize maximum likelihood-type estimator. The REKF with an EKF is used in parallel to attain the initial estimates of mobile nodes. The acquired position estimates are considered as observation vectors of the new observation equation in framework of the KF. Then two KFs, both based on the one-step predicted value of the final state estimate at the previous moment, are used for further processing the NLOS error, respectively. Finally, the position estimates are combined based on the fusion algorithm with weighted track quality to obtain the position vector of the mobile node.1)The proposed algorithm fully combines the advantageous features of the two filters to obtain precise localization result. It achieves both efficiency and robustness and even outperforms the EKF in LOS case and REKF in NLOS environment.2)It only assumes that the measurement noise variance in LOS and the process noise covariance are known in this paper. The prior knowledge of the NLOS errors is not required. Therefore, the proposed algorithm has better latent capacity to reduce localization error.3)The fusion algorithm heavily exploits the data about the state estimate at previous time, which makes it more immediate and dynamic.4)An experiment is conducted under indoor environment. The result shows the proposed algorithm performs better than the standard techniques, which indicates the feasibility of the algorithm in the practical environment.The following contributions have been made in this paper:The paper is organized as follows: A traditional method of positioning estimation is the EKF , which eSome algorithms ,13,14,15Many approaches using multiple filters in parallel are effective to deal with the effect of different types of noise, among which the interacting multiple model (IMM)  is extenm-th beacon nodes is k, k = 1,…,K, where k according to [In this section, some technical details will be introduced. Assume that M beacon nodes are randomly distributed in the field to detect the range signal from the mobile node. The coordinate of rding to :(1)x(k)=k can be expressed as:The true Euclidean distance Then, in LOS condition, the range measurement corresponding to time of flight (TOF) data can be modeled as:The probability density function (PDF) of m-th beacon node at time k is modeled as:When the obstacles prevent the signals from arriving at the beacon nodes, NLOS propagation occurs, leading to bigger distance value than the true. Thus, the distance measurement of the In a NLOS propagation environment, the error of NLOS The PDF of The PDF of The PDF of The robust extended Kalman filter (REKF) ,27 combiThe expression of the score function and the selection of the parameter in it are the key performance factors. One of the most popular score function choices is the soft-limiter  that is The REKF has robustness in NLOS propagation while the EKF has efficiency in LOS propagation. Therefore, this paper addresses a fusion algorithm based on track quality to combine these two algorithms to obtain more accurate localization result. To describe the track quality, first we define a variable called the normalized distance function, whose value represents the quality of the track. It is to be noted that the smaller value corresponds to the better track quality because the function is acquired based on the innovation of state prediction and its covariance. At every moment, the fusion algorithm feeds back the one-step predicted value of the fusion state estimation at the previous moment to the local filter. Thus, the local filter can calculate the normalized distance function based on the feedback information. The fusion center assigns weights through the evaluation of tracking quality of different filters to realize weighted fusion. Because of the real-time and dynamic nature of the algorithm, more accurate coordinates of the mobile node can be obtained.k while the output is position vector As shown in Let The The EKF steps are divided into two phases: the prediction phase and the update phase.Step 1: Kalman PredictionIn this stage, a prior estimate of the next time is derived from the current state and error covariance:Step 2: Kalman UpdateThe updating step is computed to obtain the posterior state estimation In practice, the covariance Step 1: Kalman PredictionThe prediction steps of REKF is similar to the EKF. They are run in parallel:Since the Step 2: Rewrite EKF EquationWhenever possible, converting the nonlinear model into a linear one allows us to apply the robust techniques to the filter. To do so, rewrite the state and observation Equations (1) and (9) as:Applying the least-squares to the model, we can get the solution of the equation as follows, which is equivalent to EKF filtering:Step 3: Robust Regression Algorithmk is omitted for simplicity. At each time step, if the PDF of r-th component of a vector, b is chosen to ensure continuity of function In this step, the time index function ,35, giveAt Step 1, obtain the initial state estimate, the index At Step 2, compute the noise residuals At Step 3, the scale of the noise is estimated, given as:At Step 4, the state estimate is updated by the Newton-Raphson method, which is based on the score function evaluated at the normalized residuals:At step 5, calculate the norm of the previous and actual position estimate as follows:Stop the algorithm when the norm is smaller than the required precision and output the estimate. Otherwise, repeat steps 2–5 until the convergence is achieved.In this step, we consider the filtering results as two relatively accurate measurements , and further process the estimation error in the KF framework.k is expressed as:To do this, the new measurement equation at time step The prediction step of filtering process is the same as conventional KF, given as:k define the normalized distance function as:Firstly, at time step It is used to describe the track quality, which is expressed as:At each time step either of the EKF and REKF which obtains precise state estimate will lead to smaller innovation and higher track quality. Then it should be overweighted to acquire the final result. Here the weights are assigned as follows:To get the state estimate, the weights need to be normalized:Then the update step can be carried out.The Kalman gain is obtained by:The posterior covariance matrix is calculated as:The posterior state estimates are updated as follows:The final position estimate after weighted fusion can be obtained as:k and k for the i-th Monte Carlo run.In this section, simulation results are provided to demonstrate the validity of the proposed algorithm. Beacon nodes are randomly deployed in the 100 × 100 In the following section, the simulation experiments are conducted under different environment, i.e., the NLOS errors obey various distribution and different values of the parameters in the algorithm are chosen in the simulation.The first step is to investigate the impact of historical weight factor The default parameters in the simulation of NLOS errors obeying a Gaussian Distribution are shown in The CDF of the localization error is depicted in In order to further verify the effectiveness and robustness of REKF-TQ, the paper investigates the behavior of the proposed algorithm at changing the mean of NLOS error, the standard deviation of NLOS error and the probability of LOS propagation. Simulation results of changing the probability of LOS propagation are illustrated in The default parameters obeying an exponential distribution are listed in The impact of the number of beacon nodes on the performance of these algorithm is shown in The default parameter values in the simulation are shown in Changing the model of NLOS error distribution leads to similar localization results whereas the positioning accuracy of the REKF-TQ algorithm rather remains stable compared to other methods. This is because the proposed algorithm combines the advantages of the two filtering methods, i.e., the efficiency of EKF in LOS case and the robustness of REKF when NLOS occurs. Whether in the case of LOS or NLOS, one of these two filters has better performance, and can obtain more accurate positioning results. Through the customized standardized distance function, it can be easy to qualitatively determine which filter has better performance, which will be given a larger weight to ensure the accuracy of the estimation in the final fusion process. Moreover, the proposed algorithm fully exploits the information about the location estimation at last moment to carry out the one of the present moment, making it has self-adaptability and real-time, and can effectively correct the positioning results, so it can be more accurate under different NLOS error distributions. Interacting multiple model (IMM) algorithm, which includes multiple filters, realizes location and tracking through the interaction of REK and REKF. The probability of each model can be adjusted adaptively such that the positioning accuracy is improved compared with EKF and REKF. But the models in IMM algorithm may not match the reality well, so it has some limitations and cannot always obtain a more accurate location estimation.To detect the performance of the algorithm in real environment, two real experiments are designed and carried out in indoor environment. The measurements are obtained by the Ultra Wideband technology (UWB), which is a carrier-free communication technology and uses nanosecond non-sinusoidal narrow pulse signal to transmit data.UWB signals are sent in the form of short and intermittent pulses at very high speed (transmission rate can reach 500 megabits per second) over a very wide spectrum (up to 7.5 megahertz). The UBW system can transmit data with a small power . The receiver collects signals from the whole signal bandwidth and reconstructs UWB pulses. In indoor positioning, the UWB can accurately measure the flying time of radio signals, and then calculate the distance between the two devices. Combined with geometric positioning methods such as triangular positioning, the position information of the target can be easily obtained. It has been frequently used for accurate indoor positioning in recent years for its characteristics. The UWB device used in the experiments is shown in In the first experiment, as shown in The cumulative distribution function of estimation error in the first experimental condition is shown in The second experiment is carried out under the condition that beacon nodes are randomly arranged in a 10 × 7 In this paper, an algorithm using a robust extended Kalman filter and a fusion method based on the weighted track quality is proposed to mitigate NLOS errors. It is assumed that only the sensor noise variance is known. The extended Kalman filter (EKF) equations have been reformulated as a linearized regression model, which allows us to apply robust estimation techniques. In particular, this paper considers the Huber’s maximum likelihood-type estimator to robustify the EKF to achieve the robustness in NLOS environment. However, the efficiency in the nominal case and the robustness in the NLOS case are contradictory indicators so that the robust extended Kalman filter (REKF) loses precise positioning performance in the case of LOS. Instead, an EKF with the REKF is concurrently used for preliminary location estimation. The position estimates obtained from the two filters are considered as observation vectors, which can be processed in the framework of Kalman filter (KF). Then two KFs with new observation vector and equation are run in parallel to further filter the estimates using the feedback information based on the location result of the previous moment. Finally, the acquired estimates are combined by the fusion algorithm based on the track quality to get the final position vector of mobile node. Simulation results illustrate that the proposed algorithm generates the accurate location estimation, and it significantly outperforms the EKF, REKF and even the interacting multiple model algorithm (IMM) with EKF and REKF in the LOS case and NLOS environment.


Canis familiaris) is a unique animal and the oldest domesticated species. Dogs have coexisted with humans for more than 30,000 years and are woven into human society as partners bonding with humans. Dogs have acquired human-like communication skills and, likely as a result of the domestication process, the ability to read human emotions; therefore, it is feasible that there may be emotional contagion between human and dogs. However, the higher time-resolution of measurement of emotional contagion between them is yet to be conducted. We assessed the emotional reactions of dogs and humans by heart rate variability (HRV), which reflects emotion, under a psychological stress condition on the owners. The correlation coefficients of heart beat (R-R) intervals (RRI), the standard deviations of all RR intervals (SDNN), and the square root of the mean of the sum of the square of differences between adjacent RR intervals (RMSSD) between dogs and owners were positively correlated with the duration of dog ownership. Dogs’ sex also influenced the correlation coefficients of the RRI, SDNN, and RMSSD in the control condition; female showed stronger values. These results suggest that emotional contagion from owner to dog can occur especially in females and the time sharing the same environment is the key factor in inducing the efficacy of emotional contagion.Emotional contagion is a primitive form of empathy that does not need higher psychological functions. Recent studies reported that emotional contagion exists not only between humans but also among various animal species. The dog ( Emotional contagion is a primitive form of empathy and does not need higher psychological functions such as theory-of-mind and perspective taking . Recent Canis familiaris) dyad is a preferable model for evaluating the factors involved in an emotional contagion that has been established for three reasons. First, dog and owner can form a biological bond that is mediated by the bonding hormone oxytocin  are not stable but contain fluctuations. Heart rate is controlled by both the sympathetic and parasympathetic nerve systems, and the balance of these two systems can determine the RRI. RRIs contain variability, not constancy, and, therefore, some indices are used as parameters for heart rate variability (HRV). The root mean square of the successive differences in RRI (RMSSD) reflects the beat-to-beat variance in HR and is the primary time-domain measure used to estimate the vagally mediated changes reflected in HRV. In contrast, both the sympathetic and the parasympathetic nerve systems contribute to the mean of the standard deviations of RRI (SDNN). Therefore, the parameters of HRV are useful indicators when measuring the autonomic nerve system activity that is caused by emotional state . VariousIn the present study, we used HRV as an index of emotional status that changes in seconds, and compared the changes of HRV indices between the owner and the dog in a stressful situation. We included the factors that can affect the efficacy of emotional contagion, such as the duration of the ownership, attachment behavior (index of a strength of bonding), for analysis. We hypothesized that emotional contagion can be observed in owner–dog dyads and showed high bonding behavior and long-term sharing within the same environment.The experiment was carried out on 34 small-to-large-sized dogs and their owners, who were recruited at animal hospitals and parks . We askeThe experiment took place in an experimental room at Azabu University. An experimental setting is shown in The schematic procedure was illustrated in After the 40-min resting period, the dog (tied with a loose leash) and the owner entered the experimental room. The experiment was conducted in three steps: (1) filling out the questionnaire pretest and resting time for 40 min, (2) participation in the experimental condition for 30 min, and (3) filling out the post-test questionnaire) . All dogThe Trier social stress test (TSST) is frequently adopted as a method of causing social stress in humans . The staThe control experiment was conducted to exclude possibilities that the utterance itself at the time of the explanation of the document or the math exercise were influencing the dog rather than the owner’s emotional states in front of the audience. The owner was asked merely to read the same documents, but the owner did not need to explanation it. Regarding the control condition of arithmetic, the owners were asked to read loud the formulas and answers written on the paper.ECG measurement was conducted according to our previous report . The eleThe ECG data for each dog and owner were recorded on the compact multifunction sensor. At the same time, data for each dog and the data for each owner were sent to a personal computer with a software  in real time to synchronize ECG data with video recorded with cameras .n = 9) or 41 (n = 5) time-bins were obtained in the about 10 min stress/control session. The parameters of the HRV were calculated for each time bin; standard deviation of normal-to-normal R-R intervals (SDNN) that is the index of autonomic nerve systems (ANS), the root mean square of successive heartbeat interval differences (RMSSD) that is the index of parasympathetic nerve system, and mean R-R intervals (mean RRI). If the RRI values were not detected due to mechanical errors for more than 10% of occurrences in each time-bin, the time-bin data were excluded from the analysis.The RRI detection from ECG analysis was the same as in our previous report . BrieflyWe excluded 20 out of 34 data from the further analysis. The reasons for exclusion were as follows. (1) Only one condition could be done. (2) The data was not complete for 2 conditions because of the serious video or ECG artifact was happened under either condition. (3) Certain arrhythmias (Second degree atrioventricular block (2nd degree AV block) was found frequently in one dog. 2nd degree AV block can be happened not only the dog with heart disease but also the healthy dogs with rest. However, it was unclear which was the cause so that the dog-owner’s data was excluded.r = 0.991), and only the duration of the ownership was used for the statistics to avoid multicollinearity. Finally, the effects of these parameters on the correlation coefficients of HRVs were examined by generalized linear model (GLM).For condition comparisons of STAI scores, dog behaviors, and HRV parameters, a Kruskal–Wallis test followed by a Wilcoxson’s signed rank test was conducted. We analyzed these data using statistical software . In addition, to test the hypothesis that sharing the same environment and bonding can increase the efficacy of emotional contagion, expressed as correlation coefficients of HRV parameters between dogs and owners, the following factors were tested; the duration of the ownership, daily duration of time spent with dogs, age of the dogs, dog’s sex, dog’s body weight, dog’s gazing time to the owner, owner’s gaze time at the dog, touching, by Spearman’s rank correlation coefficient , with Benjamini and Hochberg method for false discovery rate of multiple comparison. In these correlation analyses, because the duration of the ownership and dog’s age was highly correlated . Other behavior was not different between the conditions. STAI scores for state anxiety tended to be higher in the stress condition than in the control condition . Other STAI scores were not different between conditions.The behavior of the dogs in each condition was shown in Comparisons of each HRV parameter between conditions did not show any significant difference either in dogs or owners .z = −1.91, p = 0.055, Cohen’s dz = 1.9862). When the correlation coefficients of HRV parameters between dogs and owners were compared with the other variables , there was a significant positive correlations found between the duration of the ownership with the correlation coefficients of RMSSD in the stress condition . GLM revealed that 5 out of 6 correlation coefficients of HRV parameters were positively influenced by the duration of the ownership . In addition, daily duration of time spent with dogs also had two positive influences (but one negative) on the correlation coefficients of HRV parameters . In the control condition, GLM revealed the effect of sex on correlation coefficients of all three HRV parameters; female dogs showed stronger correlation coefficients as compared to male . Owner’s gaze time at the dog and dog’s gaze time at the owner had negative influences on the correlation coefficients of HRV parameters . We have started this experiment of 34 pairs of the owner and dog. However, due to the technical, especially instability of continuous monitoring of HR, 20 pairs were excluded from the analysis. As responding to the owner’s request, we did not shave the hair of the dogs, and this caused the electrical insulation on the pads. More sophisticated methods are needed in future. Even the small sample size, several interesting findings were observed; duration of the ownership were correlated with the correlation coefficient of HRVs between owner and dogs, especially in RMSSD in the stress condition (p < 0.001). RMSSD mainly reflects the activity of the parasympathetic nerve, thus sharing the same environment (duration of ownership) can increased the synchronicity of parasympathetic nerve activity between the owner and dogs under the owner’s stress condition. Another interesting point was that, in the control condition, female dogs showed stronger correlation coefficients of HRV parameters with the owners as compared to male. Females animals, including humans, show higher capability in empathy . However, the gazing behavior directed by dogs toward their owners had negative influences to the correlation of coefficients of HRV between dogs and owners. Measuring oxytocin both in dogs and humans is needed to clarify this unsolved issue.One unsolved question concerns the mechanisms underlying the facilitating effect of sharing environments on the expression of emotional contagion between dogs and owners. Living with humans can bring many social experiences to dogs. For example, dogs can form associative learning between environmental risk/threat and owners’ emotional/behavioral reactions. Therefore, owners’ emotional reactions to the environments could be conditioned stimuli to dogs and dogs can form associative learning, and show the similar emotional reaction by observing the owners’ reactions. The other possibility is that a familiar/affiliative relationship exists. While living together, dogs and their owners can form a bond that can facilitate the emotional contagion. Oxytocin is a candidate molecule for explaining this possibility. We observed that eye-gaze from a dog stimulates oxytocin release in the owner, and owner’s talk and touch also stimulate oxytocin release in a dog . TherefoThe dog is a unique animal, which is the oldest domesticated species. Dogs have coexisted with humans for more than 35,000 years and are woven into human society as partners who bond with humans . Dogs haThe raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.All experimental procedures were approved by the Animal Ethics Committee of Azabu University (#180410-1) and the Ethical Committee for Medical and Health Research Involving Human Subjects of Azabu University (#052).MK involved in all steps of the process, and was the primary conductor of the experiments. MK, MN, KM, and TKi designed the study and wrote the manuscript. MK and MN performed the statistical analysis. TKu, TY, KF, KN, and KI developed the Matlab codes for data processing.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.