Software aging is a phenomenon referring to the performance degradation of a long-running software system. This phenomenon is an accumulative process during execution, which will gradually lead the system from a normal state to a failure-prone state. It is a crucial challenge for system reliability to predict the Aging-Related Failures (ARFs) accurately. In this paper, permutation entropy (PE) is modified to Multidimensional Multi-scale Permutation Entropy (MMPE) as a novel aging indicator to detect performance anomalies, since MMPE is sensitive to dynamic state changes. An experiment is set on the distributed database system Voldemort, and MMPE is calculated based on the collected performance metrics during execution. Finally, based on MMPE, a failure prediction model using the machine learning method to reveal the anomalies is presented, which can predict failures with high accuracy. In modern society, the complexity of software is continuously increasing, which brings convenience but also increases challenges to maintain software reliability. Rapidly developing cloud software based on distributed systems is a typical example and has attracted a lot of attention. It has two characteristics of high complexity and being long-running. Due to the accumulation of errors and garbage, it usually suffers from performance degradation or an increase in failure rate. This phenomenon is also called software aging .Software aging has caused tremendous damage to many complex long-running systems such as web servers , operatisoftware rejuvenation.Software rejuvenation is an effective approach to delay or prevent the occurrence of ARFs . SoftwarHidden failure-prone state: Software aging is a gradual accumulation process. There is no prominent feature indicating the system is in a failure-prone state before ARFs, which is not only related to the current performance but also the previous performance indicators.Fluctuating noises: The system is in a dynamic and long-term running state, and it is inevitably interfered with by some fluctuating noises when collecting performance indicators. The presence of these noises may lead to misinterpretation of instantaneous anomalies as ARFs, leading to an increase in the predicted false alarm rate.Multidimensional factors: ARFs are the result of the simultaneous impact of multidimensional factors from the internal environment and external environment. Single-dimensional performance indicators or linear models are challenging to predict the dynamic and nonlinear software aging process.During the aging process, the system suffers from performance degradation before the failure occurrence. Thus, performance indicators can be used to predict ARFs. However, limited by the complexity of the system, effectively predicting failure is still a challenging task, especially because of the following three challenges:Through the analysis of the challenges above, a novel approach for predicting the ARFs is proposed based on the combination of the dynamical time series analysis method and anomaly detection in machine learning. First, a new software aging indicator, Multidimensional Multi-scale Permutation Entropy (MMPE), is proposed, which is calculated on the time series of performance indicators. Permutation entropy (PE) is a complexity measure for time series analysis . This meTo deal with software aging, quite a lot of research has been developed for predicting the ARFs. The approaches proposed in previous studies are classified into model-based approaches and measurement-based approaches. The basic idea of model-based approaches is to provide state-based models that represent the degradation level of the system. Okamura et al.  combine The measurement-based approaches usually monitor system variables and analyze the data collected during the runtime statistically. Our approach can be regarded as a measurement-based approach. The measurement-based approaches in previous studies mainly include time-series analysis, thresholding, and machine learning.To predict the ARFs, the time-series analysis is used for analyzing the aging trend of software. The work  presentsP. Chen et al.  present Machine learning approaches infer a system state as normal or failure and identify failure-prone anomalies from algorithms such as classifiers and regressors. In the paper , K. Do eThe paper is organized as follows. The details of the proposed software aging indicator are elaborated in The procedure of the ARFs prediction approach is presented in The details of the ARFs prediction approach are introduced in this section, except for the data collection module, which will be presented in the experiment setup.In complex software systems, the performance degradation exhibits nonlinear and dynamic manner during the software aging. Thus, the complexity measure of analyzing time series is introduced as a novel software aging indicator to detect dynamical changes in the software system. The concept of entropy has been an essential measure of the complexity of time series generated from nonlinear dynamical systems. Bandt and Pompe introduced PE as an appropriate method that maps the continuous time-series to a symbolic sequence . PE usesThe specific details of modifying PE to the software aging indicator MMPE can be divided into three parts. The basic principle of permutation entropy is introduced in the first part. In the second part, the coarse-grained process is applied to modify PE to MPE for multi-scale time series. In the third part, MPE is modified to MMPE, which is appropriate for multidimensional and multi-scale time series. Then MMPE is used as a software aging indicator to reveal the dynamic transition of the system from a normal state to a failure-prone state.The initial object of the permutation entropy (PE) algorithm is a one-dimensional time series  theorem . This stAfter the phase space reconstruction, the From the derivation process of To express the structural complexity at different scales, PE is modified to Multi-scale Permutation Entropy (MPE) to contain more hidden characters. MPE is calculated by PE of the coarse-grained time series over different scales . For X =The The process of the coarse-grained sequence corresponding to each scale in the range of [1,The MPE introduced in the previous section is appropriate for single dimensional time series. However, the performance indicators collected during the execution contains multidimensional variables; MPE cannot be directly used as an aging indicator for multidimensional time series. Thus, the MPE is extended to MMPE through the following steps.N. Then The first step is the normalization of performance metrics. The multidimensional performance metrics have multiple ranges and units. The significant changes in the numerical fields between performance indicators with different units  may affect the accuracy of MMPE. To remove the unit limits for performance indicators, the performance indicators are normalized and converted to dimensionless values. For the given The MPE of the The 2-Norm  is often used to calculate the length of the vector . MPE valAlgorithm 1 The algorithm of the Multidimensional Multi-scale Permutation Entropy (MMPE) Calculation Procedure.Algorithm 1. MMPE Calculation ProcedureDuring the software aging process, the multiple time-varying performance indicators are monitored as a multivariate time series. Multivariate time series provide rich information for detecting performance anomalies, but also increase the workload of calculation to a certain extent. More importantly, there may be correlations between these performance indicators, thereby increasing the complexity of feature analysis. If each indicator is analyzed in isolation, the information in the data cannot be fully utilized. Therefore, the principal component analysis (PCA) method is used to reconstruct a set of linearly uncorrelated subsets based on the original high-dimensional performance indicators . The higOur method for detecting performance anomalies is based on the MMPE algorithm combined with the sliding window. A window of length N is designed to slide on the time series and divide the long time series composed of performance indicators into overlapping blocks. Each overlapping block corresponds to a data subset of length N, and MMPE of each data subset is calculated for detecting performance anomalies, as shown in The changes in the MMPE values with time are excepted to accurately indicate the dynamic transformation of the system state during the software aging process. Therefore, before figuring MMPE, it is necessary to consider how to choose the appropriate embedding dimension In the coarse-grained process for different time series scales, the window size N and the scale factor oo large . HoweverWhen reconstructing the phase space, if the embedding dimension nd seven ; To compnal work .Basing on the aging indicator MMPE, the failure-prone anomalies are detected to predict upcoming ARFs in time. Thus, the signature window and anomaly detection techniques are combined to achieve failure prediction. In anomaly detection algorithms, anomalies are also considered outliers and deviations to be detected . This coThe software aging process is a process in which performance gradually decreases with time. The system is in a failure-prone state with performance anomalies before the ARFs occur . TherefoTo avoid false positives and improve the accuracy of failure prediction, an approach which combines the MMPE algorithm and the anomaly detection algorithm is proposed. First, the monitored performance indicators during execution are used as the input of the MMPE algorithm and obtain MMPE that measures the complexity of the system as a novel software aging indicator. The signature window of length In this section, the Isolation Forest and OC-SVM as two different anomaly algorithms are used to choose the appropriate window size in the dataset collected through the experiment. At the same time, they have further verified the applicability of our method on the public data set. Isolation Forest and OC-SVM are the unsupervised algorithms used to detect anomalies. Unsupervised anomaly detection technology assumes that most of the samples are normal, and detects anomalies in the unlabeled test data set by finding the most outlier instances ,35.1. Isolated Forest anomaly detection methodThe principle of Isolated Forest is to isolate anomalies instead of the most commonly used technique for analyzing regular points . Due to X samples of the train set are randomly selected from the training set to construct an “iTree”. Then the steps are iteratively repeated one according to the sample data capacity to create the “iTrees” to construct a binary tree forest.The isolated Forest algorithm uses the expected average length of the binary search tree to estimate the average of “iTrees” length. For the given set of samples of size The binary search tree is recursively traversed and the path length 2. OC-SVM anomaly detection methodρ is the intercept of it. In the case of correctly distinguishing as many target samples as possible, the distance from the origin to the hyperplane is maximized. The optimization of the OC-SVM is the following:The OC-SVM is an unsupervised outlier detection introduced by B. Schölkopf , which eThe anomalies detected by the above anomaly detection algorithms correspond to the abnormal state of the system. The ARFs are predicted by the detection of continuous failure-prone anomalies.To verify the applicability of the prediction approach for the software aging-related failure problem in a real situation, the approach was tested via the case study based on Project Voldemort, which is used at LinkedIn. First of all, we introduce the experiment platform and the experiment setup for proving our ARFs prediction approach in this section. Then, different workloads are designed to simulate the software aging process and monitor the key performance indicators during the running time for detecting the failure-prone anomalies.To explore the effectiveness of our approach for predicting ARFs, we propose the following two research questions:RQ1: Does the size of the signature window impact on the effectiveness of the prediction approach?We set up a set of experiments with different window sizes on the same training set which contains the MMPE and original performance indicators. The suitable size of the signature window was determined by comparing the average effectiveness of the anomaly detection models.RQ2: Does the new software aging indicator MMPE effectively improve the accuracy of predicting failure? Is its effect universal?We set up two sets of experiments, using the original performance indicators and the performance indicators with MMPE as two different training sets. By comparing the average results of the prediction, the effectiveness of MMPE was analyzed. To verify its universality, we not only collected performance datasets through the designed experiment, but also selected public data sets for verification. The experiment designed for the ARFs prediction approach is introduced in the following section.1 was deployed as our test platform. Project Voldemort is the not only SQL (NoSQL) system that provides flexible storage used at LinkedIn by abundant critical services powering a large portion of the site. We set up the database system Project Voldemort 1.10.26 on a single node cluster with two data partitions. We used Yahoo! Cloud Serving Benchmark (YCSB2), an open-source framework for benchmarking cloud data serving [To verify the above two problems experimentally, an open-source distributed key-value storage and serving system, Project Voldemort serving , to geneTo collect the performance indicators during software aging, workloads should be generated to simulate real applications and accelerate the process of software aging-related failures. In our experiment, the YCSB was applied to generate the workloads corresponding to real-world application situations. The workload generated by the YCSB client consists of two parts: the data set of records to be loaded, and the transaction set that defines the operations performed on the data set. The YCSB client will try to perform as many operations as it can possibly perform in each execution. For example, if each operation takes an average of 10 milliseconds, the client will perform about 100 operations per second per worker thread. Besides, these operations are subject to Zipfian distribution, which ensures that specific keys are accessed more frequently than other keys, thus simulating the general access mode of most websites.Update-workload: This workload contains operations containing a mix of 50/50 reads and writes, the corresponding application example is a session store recording recent actions. In this workload pattern, the YCSB Client uses a single worker thread, and the Voldemort server is in a normal operational condition.Write-workload: This workload contains operations containing a mix of 75/25 writes and reads. We change the write rate by changing the time interval  between two consecutive requests generated by the same client. The corresponding application example is that users continually update their information.In this paper, we keep the same data set and change the workload parameters for the transaction set to design two types of workload patterns, such as the following:http://www.project-voldemort.com/voldemort/; 2. YCSB: https://github.com/brianfrankcooper/YCSB)The Update-workload is used to simulate the normal operation, and the Write-workload is used to accelerate the process of software aging-related failures through consuming the resources continuously. We first accelerate the process of software aging and then change the Write-workload pattern to the Update-workload pattern for the normal state after the failure. If the failure persists, it can be ruled out that it is a failure caused by the workload overload. It is worth noting that the maximum number of client threads for Voldemort is 100. The workloads of our application are within the configured range of the server.  to collect system activity information from multiple aspects, including CPU utilization, memory usage, disk I/O, network traffic, etc. Since Project Voldemort is written in Java and runs on the JVM, we also use java memory monitoring tool Update-workload pattern for the normal operations to collect the performance metrics of the normal state, and the Write-workload is applied to accelerate the aging process for aging-related failures caused by the internal software errors (Aging-Related bugs). During the experiments, we collect performance indicators every five seconds, and the experiments will stop when the system crashes, or resources are exhausted. In our experiments, we observe two typical phenomena caused by aging-related failures:To avoid biases related to the workload patterns, we conduct a total of 20 groups of experiments by changing the parameters of the workloads. According to the workload patterns described in the previous section, we apply the Server process killed: When system memory exhaustion occurs, the server process of Voldemort will be killed by Out of Memory (OOM) killer. This situation causes the system to panic and stop; the server cannot be restarted unless a software rejuvenation strategy is initiated. Client connection timeout: Affected by software aging, the server cannot completely respond to the YCSB clients’ requests. The timeout error rate increases with the aging process.Significant performance degradation can also prove the existence of software aging. The performance of Voldemort as a NoSQL system is measured by the average latency of the requests and the throughput. The average latency refers to the delay from the time the user sends the request to the response from the server. This is an indicator that can directly measure whether the software can meet user needs. As shown in Voldemort System Dataset: To eliminate the impact of different units, we normalized the performance indicators to obtain the dimensionless performance metrics. Then, we used the PCA method to convert the performance metrics that may be related to a new set of linearly uncorrelated synthesis through orthogonal transformation [ormation . The perGoogle Cloud trace Dataset (GCD): In addition to the datasets collected in the above experiment, we further applied our approach to the public dataset. However, few public datasets can be directly used for the ARFs prediction. This dataset was filtered from the Google Cloud trace Dataset (GCD) which is very representative in cloud computing. The GCD contains the CPU and memory utilization by summarizing their CPU and Memory utilization every five minutes for the tasks of each job. In this paper, four subsets were selected where the CPU utilization exhibited an increasing trend and the threshold exceeded 80%. Due to their performance being close to the software aging threshold, the corresponding failure reports in the data set are regarded as software aging failures for the ARFs prediction. A similar approach is verified in the references [ferences .To standardize the number of groups in test sets and avoid overfitting, we apply the five-fold Stratified Shuffle Split to split the datasets. This cross-validation returns stratified randomized folds, which preserve the percentage of samples for each class . The proThe prediction of the ARFs is transformed into a binary classification problem in our approach. Since the failure-prone anomalies account for a small percentage of the total samples during software aging running, there is a class imbalance. Thus, the smote method was applied to increase the proportion of anomaly samples to 30%. The confusion matrix was used for representing the four prediction results of the model for the positive class. The positive classes correspond to the predicted conditions that are failure-prone, where the True Positive (TP) presents the correct alarm, and the False Positive (FP) presents the false alarm. The negative classes correspond to the predicted conditions that are not prone to the ARFs, where the False Negative (FN) presents the missed alarm and the True Negative (TN) presents the correct normal alarm.We used three metrics to evaluate the validity of the ARFs prediction results for positive classes. The ratio TP/(TP + FP) represents the precision, where TP is the number of true positives and FP the number of false positives. Intuitively, the precision represents the ability of the predictor not to mark negative samples as positive. This measure can assess the rate of false alarms in which the ARFs predictor misjudges normal conditions as failures .The recall is the ratio TP/(TP + FN). This measure is the ability of the predictor to find all the positive samples, and thus can assess the percentage of failures that can be predicted.The prediction models are expected to perform well in both precision and recall. However, it is hard to take care of the precision and recall in some conditions. For example, if the model expects to predict positive samples as much as possible, it will have a higher recall but a lower precision. If the model is conservative and only predicts positive samples with a high degree of confidence, then its precision will be relatively high, and the recall will be relatively low. Therefore, the f1-score is proposed as an indicator that comprehensively considers recall and precision. The f1-score is defined as the weighted average of precision and recall, and the relative contribution of them to the f1-score is equal. The f1-score reaches its best value at 1 and the worst score at 0. The formula for the f1-score is: The f1-score aims to improve precision and recall as much as possible while training the model.The accuracy score is the ratio (TP + TN)/TP + TN + FP + FN, which presents the ratio of the number of correctly predicted samples to the total number of predicted samples.The effectiveness of our approach for predicting ARFs is verified in this section. First, the effect of different signature window sizes is analyzed by the comparison of prediction results. The most suitable signature window size is determined according to the best prediction results. Then the prediction results obtained by the ARFs prediction approach based on MMPE and original performance time series are compared to verify that MMSE can effectively improve the accuracy of predicting failureTo answer RQ1, the experiments are designed with three window sizes on the same training set. Because the time step of monitored indicators is 5 seconds, we set http://github.com/google/cluster-data).It can be observed that the suitable window size has an obvious positive impact on the prediction results. When the window size is 45 seconds, it corresponds to the worst case. The precision of the OC-SVM is 0.72 and the f1-score is 0.71 in this case. The window size of 45 seconds is too large to reduce the sensitivity of the anomaly detection algorithm to the failure-prone anomalies. When the window size is 30 seconds, the prediction results of all aspects of the two anomaly detection algorithms are the best case. The recall of the Isolated Forest is 0.94 and the f1-score is 0.91 when the window size is 30 seconds. As for the OC-SVM, f1-score is 0.86. Therefore, the window size of 30 is chosen as the optimal choice for the next experiment.The increase in prediction efficiency is due to the that MMPE can reveal hidden system states. Compared with the original performance indicators, the MMPE algorithm reconstructs the time series, including the information in the time dimension. The system state revealed by MMPE not only corresponds to the current performance, but also to the previous values of the performance indicators. The outstanding performance of the ARF approach on the Project Voldemort and GCD dataset verifies its applicability for ARFs in complex software systems.In this paper, an approach for predicting ARFs is proposed based on the dynamical anomaly detection method. We first propose a novel software aging indicator MMPE, calculated on the time series of performance indicators. To our knowledge, we are the first to apply the complexity measure permutation entropy (PE) of a dynamical system to predict the ARFs. Based on the aging indicators, a signature window is designed to label the failure-prone anomalies and use an unsupervised anomaly algorithm Isolation Forest to detect failure-prone anomalies. To verify the applicability of the prediction approach for the ARFs in a real situation, the software aging experiments run on Project Voldemort, a distributed key-value storage system that is used at LinkedIn by numerous critical services powering a large portion of the site. The experimental evaluation results in the distributed database verify that our approach for predicting the ARFs can achieve high accuracy. Compared to the original performance indicators, the addition of MMPE increases the f1-score of the Isolated Forest anomaly detection model from 0.86 to 0.91 on the Voldemort Dateset. Our approach provides a new effective way for the online prediction of the ARFs in complex software systems.


Falls are among the main causes of injuries in elderly individuals. Balance and mobility impairment are major indicators of fall risk in this group. The objective of this research was to develop a fall risk feedback system that operates in real time using an inertial sensor-based instrumented cane. Based on inertial sensor data, the proposed system estimates the kinematics (contact phase and orientation) of the cane. First, the contact phase of the cane was estimated by a convolutional neural network. Next, various algorithms for the cane orientation estimation were compared and validated using an optical motion capture system. The proposed cane contact phase prediction model achieved higher accuracy than the previous models. In the cane orientation estimation, the Madgwick filter yielded the best results overall. Finally, the proposed system was able to estimate both the contact phase and orientation in real time in a single-board computer. Falls are among the most common causes of mobility impairment in the elderly . Falls aFollowing the above trends, the number of fall-related studies has also increased over the past few years. Most of these studies have focused on fall detection with the objective of reducing the adverse effects of falls. A system that detects fall occurrences can generate an alarm to caregivers or doctors, who can provide rapid treatment that supposedly reduces the negative consequences of falls . AlthougHence, our research focuses on the development of an instrumented cane that estimates the risk of fall in real time and provides feedback when there is a high fall risk. In a high-risk situation, the proposed system would alert the user to rest or contact a caregiver for assistance, rather than continuing to move. The sensors are placed on a single-tip cane because cane users are common among the elderly population; moreover, placement on the cane is less burdensome than placement on the body. The proposed system could estimate the fall risk by extracting the gait and balance parameters, which are well-known indicators of fall risk.As an example, balance impairment is considered to greatly increase the fall risk . BalanceAs most falls occur during walking , fall riIn this study, we propose a real-time system that estimates the ground-contact phase and the three-dimensional (3D) orientation of a cane from inertial sensor data. Using the information of both the ground-contact phase and the cane orientation of the cane could be used to extract gait- and balance-related parameters for real-time estimation of fall risk. We first propose a deep learning model that improves the estimation of the ground-contact phase, in combination with a sliding voting window that improves the temporal accuracy of the system. Next, we compare the performances of various 3D cane orientation algorithms in the literature, using the data of an optical motion capture system as reference. Finally, although the proposed system only uses the data from one inertial sensor attached to the cane, during the experiments we employed multiple inertial sensors for analyzing the effect of sensor position on the accuracy of the proposed system.The proposed system consists of two components as shown in By detecting whether the cane is contacting or not contacting the ground, we can analyze when the cane is increasing the BoS area and thus reducing the risk of fall. Moreover, by analyzing the duration of the ground contact, we can extract the cane stride time, cane cadence, and duration ratio of the contact and noncontact phases. These parameters provide useful information on the temporal variability of the gait. In addition, knowing when the cane is contacting the ground, it can help to estimate additional parameters such as the cane orientation and cane displacement. For example, Dang et al.  used theThe cane contact phase has been detected by various methods. One of the most common methods is to use a force sensor installed on the tip of the cane to detect when the reaction force exceeds a specific threshold . HoweverIn contrast, as inertial sensors are small and lightweight, they can be easily attached to any point of the cane. Previous inertial sensor-based cane contact phase estimation algorithms can be categorized into three groups: threshold-based methods , event-bThe following subsections discuss the three categories of contact phase estimation, and present the proposed Deep Learning contact phase estimation model.g denotes gravity.Threshold-based methods are the least complex and fastest methods for contact phase estimation. These methods use simple decision thresholds to estimate whether the cane is contacting the ground or moving in the air. For example, Dang et al.  comparedAs long as both the absolute value of the linear acceleration is at or below the acceleration threshold  matrix containing n samples from m inertial sensor signals.Here, we propose a cane contact phase estimation model using a small size convolutional neural network (CNN), which delivers higher accuracy than the neural network with a single hidden layer. The CNN is a type of artificial neural network inspired by the visual cortex , in whicThe initial part of an inertial sensor-based CNN model is built using 1D convolutional layers followed by a max-pooling layer, which are repeated multiple times sequentially. These convolution layers help the model to extract features from the inertial sensor. Increasing the number of 1D convolution layers increases the detail in the extracted features and the complexity of the patterns that can be modeled. The final part of a CNN model normally consists of fully connected (FC) layers that take the features extracted from the convolutional layers, and learn the relationships among the features for predicting the desired output.n × 6), where the n samples are provided by a 3-axis accelerometer and a 3-axis gyroscope. At each instant, the last n samples are extracted from the inertial sensor data by a sliding window. The input data are passed to a 1D convolutional layer followed by a max-pooling layer. Next, the features extracted from the 1D convolutional layer are passed to an FC layer. Finally, the output of the FC layer is passed to a final FC layer with a Softmax activation function, which estimates the current contact phase. The proposed model outputs one of two labels: “ground contact” (when the cane is contacting the ground) or “no contact” (when the cane is moving in the air).Our proposed model, due to the requirement to perform the estimation in real time even in devices with low computational resources, we adopted a modified CNN model with fewer and smaller-sized layers than most CNN models in the literature. Here, the layer size was reduced to increase the inference speed. The proposed model inputs a 2D matrix  filter gain is selected such that the filter minimizes the drift error while being robust enough to avoid the divergence caused by measurement errors in the accelerometer and magnetometer. Applying a gradient descent algorithm, the Madgwick filter fuses the gyroscope data with the orientation estimated by an accelerometer and magnetometer. Feng et al. [To overcome the limitations of Kalman filters, Madgwick  presenteg et al.  comparedAlthough the Madgwick filter is widely used in several applications, it has been rarely applied to estimate the orientation of a cane. Hassan et al.  applied To the authors’ knowledge, among the many papers that have estimated cane orientation using inertial sensor-based algorithms, only one paper  has numeThe proposed system was validated in two experiments. All participants of both experiments gave their informed consent before participating in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of Kyushu Institute of Technology .In both experiments, participants used a single-tip cane equipped with five wireless inertial sensors , each containing a 3-axis accelerometer with a range of ± 5 g, a 3-axis gyroscope with a range of ± 1500 °/s, and a 3-axis magnetometer (not used in the contact phase detection) with a range of ± 1.3 Gauss. The five inertial sensors were placed throughout the cane at 15-cm intervals. The first sensor was placed below the handle of the cane, as shown in To validate the contact phase estimation, the reaction forces of the cane were measured by a force sensor  placed on the cane tip. The contact phase was labeled as “ground contact” when the reaction force exceeded 1 N, and “no contact” otherwise. The threshold value was decided based so that it had the smallest value above the noise in the force sensor when no force was applied. The force sensor signal was read by a wireless data logger , which was also attached to the cane see . The datThe objective of the first experiment was to validate the proposed cane contact-phase estimation model. Five healthy young participants  performed different daily life activities while using the instrumented cane in a laboratory environment. Ten measurements from each participant were acquired while they performed seven daily life activities in a continuous sequence see . Where pThe cane contact phase was estimated by a small size CNN see . The modj, the output The experimental data were processed through a size-20 sliding window, selected to match the size of the input model. Therefore, for each sample A total of 338,316 samples were collected from each inertial sensor, and a total of 1,691,580 samples were obtained from all IMU sensors. The model was trained and tested using leave-one-out cross validation; at each fold, the data of four participants were used in training and the data of the remaining participant were reserved for testing.The proposed CNN model contained four hidden layers . The firThe model was completed by two FC layers. The first FC layer consisted of eight neurons with a ReLU activation function, whereas the final FC layer consisted of one neuron with a Softmax activation function, which predicted the class of the cane contact phase (“ground-contact” or “no contact”). The proposed model had 745 parameters, significantly fewer than the number of parameters in similar CNN models, enabling faster prediction speed. The model was implemented and trained using the Keras neural network library on a GTX 950M chip , and the prediction time was computed in a Raspberry Pi 3B  single-board computer.In the second experiment, we compared different cane orientation estimation methods. The five participants in Experiment 1 were joined by another five participants. The 10 participants  performed the activities designated in Experiment 1 using the same instrumented cane.To validate the orientation estimated by the inertial sensor, the ground truth orientation was obtained by an optical motion capture system  equipped with eight infrared cameras. The ground truth orientation of the cane was estimated by tracking the positions of five markers placed on the cane, as shown in Both the motion capture system and the sensors in the instrumented cane were sampled at 100 Hz. Moreover, to synchronize the sensors with the motion capture data, participants were requested to raise the cane rapidly before and after each measurement. Thereby, we could compare the acceleration peaks from the inertial sensor with those of the motion capture system. As in Experiment 1, all inertial sensors and the force sensor were calibrated prior to Experiment 2. All calculations and comparisons of the cane orientation estimation were performed in MATLAB 2020a  on the same computer as in Experiment 1.As the measuring range of the optical motion capture system is limited, the activities were completed not in a continuous sequence, but were separately measured 10 times per activity. A total of 700 measurements were collected, yielding 413,419 samples for each inertial sensor.This section presents the estimation results of the cane contact phase and cane orientation, and discusses their importance.First, the cane contact phase estimated by the proposed CNN model was compared with those of previous inertial sensor-based methods, namely, the threshold-  and evenIn the threshold-based method, both threshold values  network . As theyThe cane orientation was estimated by the three methods described in The Acc + Mag method obtained the worst results overall. The significantly higher RMSE values of the Acc + Mag method were expected, as the 3D orientation in this method is estimated only from the accelerometer and magnetometer data. Consequently, this method is highly sensitive to the noise in the accelerometer and magnetometer. The results of the indirect Kalman filter were slightly worse than those of the Madgwick filter, which agreed with the results of the original Madgwick filter .The RMSEs of the stair-climbing activities were similar to those of standing, despite the more significant cane movements during the stairs-up and stairs-down activities. The similar RMSE values, despite that the cane moved during the stairs-up and stairs-down activities, were probably attributable to the short durations of these activities (average duration 3.77 s). In contrast, the gait-related activities with both dynamic movements and longer duration (9.12 s on average) yielded the highest RMSE values.As an example, Finally, To determine which IMU’s RMSE significantly differed from those of the other IMUs, a post-hoc analysis based on Dunn’s test  was perfIn summary, although IMU 2 (placed at approximately 20 cm below the handle) achieved the best orientation results, no clear relationship emerged between the sensor position on the cane and the orientation estimation performance. Meanwhile, the Madgwick filter obtained more accurate estimations than the other tested algorithms. As the Madgwick filter requires fewer computations than Kalman filters, it is more suitable for estimating the orientation in real time. As a verification exercise, the Madgwick filter was implemented in Python and tested on a Raspberry Pi 3B single-board computer. The Madgwick filter processed each sample with a duration of 1.26 μs, confirming that the proposed system can estimate cane movements in real time with an overall runtime (phase and orientation runtimes) of 2.45 ms.A deep learning model is proposed for estimating the cane contact-phase estimation from raw inertial sensor data.The temporal phase estimation is improved using a voting window.The 3D orientation of the cane during multiple daily life activities is estimated by orientation estimation algorithms, which are compared and validated.The position effect of the inertial sensor on the cane on both the contact phase and orientation estimates of the cane are analyzed.The proposed system performs the estimation calculations in real time on devices with low computational power.This study proposed a real-time system for estimating the movements of an instrumented cane. In the proposed system, a single inertial sensor is attached to the tip of a normal cane. As the cane is unmodified apart from the sensor attachment, the system is highly usable. The main contributions of this paper are as follows:A main limitation of the present study was the demographic of the participants: all subjects were young with no mobility impairment. However, as the activities of elderly users with mobility impairments are close to stationary activities, the accuracy of the orientation estimates is expected to increase in this group of users. Nevertheless, similar experiments on elderly patients are required to analyze their gait and balance while using the instrumented cane. The experimental data would also reveal how the different gait patterns of elderly participants with different mobility impairments affect the proposed system, and whether the system can detect these differences.Moreover, the measurements in the second experiment were acquired over short duration because the measurement space of the optical motion capture system was limited. However, a main source of error in inertial sensors is the drift error, which accumulates the orientation estimation error over time. Although the Madgwick filter reduces the drift by fusing the gyroscope data with accelerometer and magnetometer data, longer duration experiments will be conducted in future to analyze the effect of drift on the estimated orientation.Finally, although the gait can be analyzed using the output parameters of our proposed system, as in previous studies , extractIn summary, the proposed system can estimate both the cane contact-phase and cane orientation in real time. Such a system could provide useful information on the gait and balance of users by extracting the gait parameters, such as cane contact duration, the contact versus noncontact phase duration relationship, or the cane orientations during specific gait events. In future, these parameters extracted from the sensors could be combined with patient record information, such as age, gender, or previous history of fall, in order to generate a more accurate fall risk estimation.


The operation efficiency and safety of pressure vessels in the oil and gas industry profits from an accurate knowledge about the inner filling distribution. However, an accurate and reliable estimation of the multi-phase height levels in such objects is a challenging task, especially when considering the high demands in practicability, robustness in harsh environments and safety regulations. Most common systems rely on impractical instrumentation, lack the ability to measure solid phases or require additional safety precautions due to their working principle. In this work, another possibility to determine height levels by attenuation tomography with guided elastic waves is proposed. The method uses a complete instrumentation on the outer vessel shell and is based on the energy conversion rates along the travel path of the guided waves. Noisy data and multiple measurements from sparsely distributed sensor networks are translated into filling levels with accuracies in the centimeter range by solving a constrained optimization problem. It was possible to simultaneously determine sand, water, and oil phases on a mock-up scale experiment, even for artificially created sand slopes. The accuracy was validated by artificial benchmarking for a horizontal vessel, giving references for constructing an affordable prototype system. Separator vessels are generally objects with a cylindrical-shaped shell, vertically or horizontally aligned and consisting of normally high-grade materials. They divide various phases of oil and gas from the injected media by gravity. Depending on the architecture of the purifying process and the specific vessel type, the injected media can vary from a three-phase mixture of hydrocarbons, water and solids directly extracted from the well/reservoir itself, till a pre-separated two-phase media mainly consisting of water, oil and gas of a cascaded process in later stages . Since tAlthough in most operating countries the construction, the quality control and maintenance by Non-Destructive Evaluation (NDE) is regulated by the American Society of Mechanical Engineers (ASME) guidelines , the estIn the context of separator vessels, displacers ,7, pressDisplacers are submerged floating bodies with a defined density between two phases of interest. If the transition level is changing, the resulting buoyant force is captured by a reading device. Multiple phase-levels can be detected, by adding multiple displacers with different densities. However, these devices tend to fail in the unexpected presence of foam or emulsion since their presence effects the pressure gradient along an interface . AdditioOther methods such as radar or ultrasonic pulse-echo devices need to be instrumented at least partly inside a vessel. Ultrasonic devices are generally based on time-of-flight measurements and are commonly used to measure the gas-liquid transition height ,7. HowevHjertaker et al.  providedDespite the achieved success with each single measurement method, the combination of methods and elaborate processing schemes, there is so for no NDE standard that meet all requirements of the industry . EspeciaIn the work at hand, another possibility for continuous level measurements by Guided Elastic Waves (GEW) is investigated. GEW are often also referred as Guided Ultrasonic Waves (GUW) and becoming increasingly popular for certain tasks of NDE. Their propagation in plate-like structures ensues lower geometrical damping rates than conventional bulk waves, which generally results in longer propagation distances and better signal–noise ratios . SeveralAttenuation tomography originates from geophysics ,26 to reAttenuation tomography has already been adopted for a specific NDE scenario with GEW/GUW, namely the reconstruction of localized wall-thinning on pipe-like structures ,30. HereA surrounding media on the other hand is significantly affecting the propagation of GEW by leaking energy into the media as a type of bulk wave or even creating interface waves such as Stonely or Scholte waves . Accordi60kHz·mm . Quasi-S60kHz·mm ,38.l to the receiver. During this propagation, the energy of the wave mode will be degraded by the amount The pressure vessel and the filling is assumed as a linear viscoelastic system, with a vessel wall of finite thickness. The excitation of elastic waves at the outer vessel wall and in an appropriated frequency band will result in various elastic guided wave modes, since the shear and pressure wave velocities of the vessel wall are significantly higher than the one of a potential filling. Nevertheless, the energy of such a mode is apparently damped in the presence of an additional media since its energy is converted into other wave types. The propagation of an elastic guided wave mode for a single measurement is further approximated by a ray propagating from the sender along the travel path Q for arbitrary structures by additionally defining:In general, the attenuation consists of geometrical spreading, internal scattering and the conversion into other wave modes or wave types. For the estimation of the filling distribution particularly the latter one is of interest. As linearity is imposed, the damping from the conversion can be isolated in The measured energy degradation z inside the vessel. Each infinitesimal height section is connected to according sections on the left and right part of the vessel shell, linking a potential propagation path to the height Strictly, Equation  must be N cells given by the index j, Equation  for a certain excitation signal and a possible filling distribution. The excitation signals were Hann-windowed five cycle cosine functions (RC5) of either 50, 70, 100 or 150 kHz. To enhance the signal-to-noise ratio, each measurement was repeated 64 times and mean values were calculated afterwards. For each sample, a standard deviation was calculated beforehand and only signals with all samples within a 3.5 times standard deviation threshold were accepted. This processing step was added to remove undesired electrical noise.Various combinations of sand-water-oil fillings were measured, which can be obtained from For the attenuation tomography, the vessel was represented by two one-dimensional height versus attenuation rate distributions. Each distribution was associated with one sensor ring. Therefore the inner height was sampled, so that the outer circumference is nearly equidistantly sampled by 1 cm cells. The deviation from a perfect equidistance results from the symmetry condition splitting into left and right circumference. The whole system implies only circumferentially acquired measurements for each ring separately, as discussed in E of a certain mode arrival of a measured signal The beginning This approach has the advantage of being simply adaptable to more complex numerical models and acknowledging the attenuation in a broader bandwidth than only picked peak values, to make the method more robust against waveform deformation.w in the equation system. For example, for a sender-sensor combination half the circumference apart from each other The first and second arrival of the fundamental antisymmetric lamb mode were extracted. Values deriving from time sections with interference of both arrivals, meaning overlapping Solutions for the solely first arrival data and combined data from the first and the second arrival were computed. In the combined solution all entries of the second arrival were penalized by additional LSQR) of , which iFor each filling combination, excitation frequency and sensor ring, the solution was calculated for artificially reduced sensor networks, giving equidistant spacings of 5, 10, 20 and 30 cm. All possible combinations of shifting the network were considered likewise for each reduction step. From each single attenuation rate four different height levels were extracted, namely water-, oil-, sand- and top-level. The latter refers to the upper liquid level without distinguishing between water and oil. Based on those results, statistically validated accuracies were calculated and are given in In With increasing filling level, the amplitude of the L first arrival (FA) is significantly decreased. The change for the first 10 cm water and 0 cm sand filling is most significant. It implied the highest increase in travel path which is contact with water for the given sender-sensor combination. The underlying effect of a conversion into a bulk wave is also recognizable by an appearing arrival after the L second arrival (SA). With increasing water level, the amount of converted energy is increasing and therefore the amplitude of the bulk wave. After a filling height of 40 cm and above the amount of converted energy has reached a steady state. Since, both transducers are located below this height and therefore the amount of travel path in contact with a filling does not increase further. Neglectable deviations from this observation possibly originate from the rising load pressure.Above filling heights of 40 cm the attenuation of the SA-L is increasing, whereas below 40 cm the effect is neglectable. For this particular sender-sensor combination, the information regarding the oil filling is resolved only in the SA. The close temporal distance between the SA and the bulk wave reveals the problem of the higher probability of interference in the time frame of the SA. For instance, for the 40 cm water filling, the additional bulk arrival generates a higher energy value than for an unfilled state, which falsifies the measured attenuation value and even switch the algebraic sign.The presence of a sand phase is depicted in the lower row of Regarding the phase velocities all exemplary GEW arrivals show no significant change with increasing filling level in Q, for sensor ring two, comparable acquired and selected as The underlying data of In the following section, various results of the attenuation tomography/tomometry are displayed and discussed regarding their characteristic behavior and their quality. Quantitative results can be found in Exemplary results for 100 kHz excitation frequency, 15 cm sensor spacing and three water filling levels  are given by Due to the possibly low attenuation contrast between water and oil false indications could occur. Parts of the water layer can be estimated as oil, as shown for the measurements without any oil phase in The oil level in Oscillations in the solution in the lower height range are observable for all cases of filling combinations: water , oil-watSand fillings, as displayed by The tables in In the work at hand, the estimation of multi-phase height levels by attenuation tomography with guided elastic waves was realized. The investigated case was a small-scale horizontal separator vessel filled with various distributions of sand, water and oil, whereby piezoelectric transducers were mounted on the outer shell. The results show a reliable and accurate system regarding the top and sand level with accuracies in the centimeter range for a sensor spacing of 15 cm. The ability of exactly distinguishing between water and oil suffers from higher inaccuracies due to low contrast in the attenuation rate. The median error is in the lower centimeter range, whereby the 32nd to 68th inter percentile distance is in the decadic centimeter range. However, already existing practicable solutions for estimating fluid phase transitions could be added to enhance the results further by defining additional constraints in the optimization problem.The proposed technique is based on attenuation amplitude measurements by picking a specified wave mode and its arrivals and set them in relation to a baseline measurement. Therefore, a model-based approach was used which can be adopted to more complex geometries including nozzles or even internals. In the investigated frequency range, the filling showed no significant influence on the wave speed. Based on the analysis of the measurement data the attenuation rate of saturated sand and water were in the same range, whereby a phase transition between both phases added an unexplained mechanism, which caused a significantly decreasing attenuation rate. The proposed algorithm was able to resolve the phase transition. However, the appearance of bulk waves was observed for fluid fillings, whereas the presence of sand apparently damped the direct bulk arrivals. This behavior represents another possibly measurable contrast or addable information to enhance the accuracy of the system further. Basic assumptions regarding the filling distribution and sensor arrangement were made for the attenuation tomography. Those resulted in symmetry conditions which enabled a reduction of the model space to an independent one-dimensional optimization problem per sensor ring. Consequently, the calculated attenuation distributions were one-dimensional cross sections which coincide with the term tomometry introduced by . By combThe investigated frequency-thickness product ranged from 800 to For the investigated vessel geometry with an inner diameter of 1 m and a wall thickness of 16 mm, the 15 cm equidistant sensor spacing along the outer shell quantified as a reliable trade-off between accuracy and minimal instrumentation effort. For higher spacings, the accuracy is decreasing for the given tasks, whereas the pure top-level estimation was reliable even for investigated sensor spacings of up to 30 cm. In the current data processing only first arrivals and the combination of first and second arrivals of a specific guided wave mode were used. The latter gave no significant improvement of the accuracy for most investigated fillings. This circumstance was probably substantiated due to the lower signal–noise ratio of the SA arrivals. The exception was the estimation of the oil level, which possibly originates from the incomplete circumferential instrumentation at the upper 20 cm of the inner height of the test specimen. This results in a higher amount of SA travel paths along the upper height levels, where the oil levels were located for the investigated filling states. Consequently, for a full circumferential instrumentation this exception is expected to converge to an insignificant amount.The system could be adopted to vertical separators, whereby a single vertical line long the vessel wall seems like a logical choice for a sensor arrangement. This would imply a simpler discretization scheme with a purely linear dependence on the filling height, which could possibly lead to a better numerical stability by a lower condition number. On the other hand, the additional information given by multiple arrivals would be not available.Upcoming work should address the stability and influences in case of changes in the process parameters as temperature or pressure. The possibility of estimating residual wax layers or mixed phases as foam or emulsion should be evaluated. Additionally, the possibility of including structural features as internals or nozzles in the propagation paths by further processing such as ray tracing or eikonal solvers and its capable improvements in accuracy could be assessed. A further theoretical question is the underlying wave propagation mechanism along the water-sand phase transition for the investigated geometry and frequency range, which is causing the significant drop in the attenuation rate. The overall observed high data quality is a complied requirement for more elaborate processing schemes as full waveform inversion, which could acknowledge multiple modes at the same time, include complex mechanisms along phase transitions and eventually result in elastic and viscoelastic properties rather than in abstract attenuation rates.


The main factors influencing the shape of the beach, shoreline and seabed include undulation, wind and coastal currents. These phenomena cause continuous and multidimensional changes in the shape of the seabed and the Earth’s surface, and when they occur in an area of intense human activity, they should be constantly monitored. In 2018 and 2019, several measurement campaigns took place in the littoral zone in Sopot, related to the intensive uplift of the seabed and beach caused by the tombolo phenomenon. In this research, a unique combination of bathymetric data obtained from an unmanned surface vessel, photogrammetric data obtained from unmanned aerial vehicles and ground laser scanning were used, along with geodetic data from precision measurements with receivers of global satellite navigation systems. This paper comprehensively presents photogrammetric measurements made from unmanned aerial vehicles during these campaigns. It describes in detail the problems in reconstruction within the water areas, analyses the accuracy of various photogrammetric measurement techniques, proposes a statistical method of data filtration and presents the changes that occurred within the studies area. The work ends with an interpretation of the causes of changes in the land part of the littoral zone and a summary of the obtained results. Unmanned platforms  are devices capable of moving in a specific environment without the presence of an operator on board ,2. TheirThe coastal zone is a transition zone between the land and the water environment . The anaThe main factors influencing the shape of the beach, shoreline and seabed include undulation, wind and coastal currents ,35. BeacSopot, a city located in northern Poland on the Gdansk Bay, is a popular tourist resort . In the The research  presentsThis paper focuses on photogrammetric measurements from unmanned aerial vehicles for the measurement campaign of the unique tombolo  phenomenon in the studied area . The artTombolo  measurement campaigns in Sopot spanned three years. The most important measurements realised with a full range of measurement methods and the use of unmanned systems, took place in November 2018 and November 2019 . The artA photogrammetric flight was performed in 2018 with a type DJI Mavic Pro UAV, while in 2019, a DJI Mavic Pro 2 was used for image acquisition. Aircraft of this type are commercial flying platforms designed and intended mainly for recreational flights and amateur filmmakers. The photogrammetric community soon appreciated the versatility and reliability of these devices. They gained popularity mainly due to their simplicity and intuitive software. The technical data of both platforms are presented in Unmanned aerial vehicles (UAVs) are equipped with stabilized visible light cameras of type F230 and L1D-20C with 12 and 20 million pixels, respectively. The technical data of both cameras are presented in In terms of photogrammetry, the coastal zone combines a land rich in solid textures with water that is extremely variable in terms of images and very luminous. In this case, the UAV measurement focused on the beach, which was considered a priority area, with the water area and the offshore pier being treated as auxiliary areas. It was assumed that a ground sampling distance (GSD) of approximately 2 cm/pixel would be sufficient to generate a numerical model of the terrain and point clouds of the land surface, and it would allow for further analysis of the phenomena with satisfactory accuracy. Based on this value, the height of the flight over the beach was determined. No required minimum GSD values for the auxiliary area were assumed.The data of the planned flight patterns are presented in detail in Flights performed in the 2018 campaign consisted of two different plans. The first plan, based on the double grid scheme , was reaThe photogrammetric flight in the 2019 campaign was planned and executed according to the single grid plan . Due to When analysing the results, note that no tie points can be found on luminous and non-textured surfaces . ExampleThe result-processing stage starts with importing all of the photos to computer software and entering the processing settings. Different commercial photogrammetric software was used for both cases, Pix4D Mapper and Agisoft Metashape, respectively. The project calculation for both cases was done differently with varied user access to the initial processing settings. Each manufacturer also uses its own, different file formats, which are not mutually compatible. This implies the necessity to make data processing and exchange uniform.application of georeferencing directly and/or through the use of ground control points, standard processing settings (as proposed by the software);exporting results in the form of a high-density point cloud to LAS format, a surface model to OBJ format and a numerical coverage model and orthophotomap to TIFF format;B, L and h ;the UAV navigation system records camera position relative to the WGS84 ellipsoid and represents it with geodetic coordinates position of the photogrammetric warp points is expressed in coordinates in the Polish PL-2000 system of flat coordinates, and their altitude is expressed relative to the quasigeoid in the Polish PL-EVRF2007-NH altitude system;p = 0.95);the location of ground control points is measured with an accurate method of differential satellite positioning GNSS RTK  and vertical plane (z) and low standard deviation (σ), which proves that position measurements are very stable. On this basis, it is concluded that georeference for each image was determined correctly, with high accuracy and there were no significant deviations. The values were within the accuracy range typical for single-frequency GPS receivers [In both measurement campaigns, the UAVs used could record images with metadata on the current UAV position. These data are saved in EXIF (exchangeable image file format). The current position, recorded by an onboard GPS receiver and saved in the image metadata, can be compared with the external orientation elements (EOP) determined at the aero-triangulation stage. In this way, for each image, the absolute position error of the central projection position was determined in meters and the standard deviation of the position error for the entire block of images was calculated. The 2018 data presented in eceivers ,56,57,58x, y) are within the range typical for GPS receivers used in commercial UAVs. The absolute error in the vertical plane (z) is significantly greater (65.4 m) and results directly from using different altitude reference systems. The determined ellipse errors for EOP are shown in External orientation elements specified for the 2019 flight have a larger mean absolute camera position error. These values result from the use of ground control points in the photogrammetric process and determining their position in relation to another reference system. The measurement performed by an on-board GPS records the position and altitude relative to the WGS84 ellipsoid. The GCP position was determined relative to the Polish PL-2000 flat coordinate system and the altitude relative to PL-EVRF2007-NH quasigeoid. As presented in The photogrammetric warp points were distributed evenly over the entire area of the study and their position was measured with an accurate GNSS RTK satellite positioning method. Five points were located on stable infrastructure elements, such as concrete sidewalks running along the beach. Two more were placed on the concrete marina breakwater constituting a part of the building. Internal camera orientation elements were taken from the database of the photogrammetric software use and optimized using autocalibration during the preliminary model development process. Detailed internal orientation element values after optimisation and standard deviation are presented in It is very difficult to correctly and precisely reconstruct the water surface and it may even prove impossible in practice. This is because popular algorithms ,60,61,62A rectified image may be fitted and will become part of the orthophotomap, provided that homologous points are found for a given stereo-pair. As shown in Incorrectly located key points affect the accuracy of the high-density point cloud; thus, other photogrammetric products have visible artefacts. In the case of DSM, the resulting ambiguities in the location of key points consist of incorrect altitude reconstruction, which is visible north of the pier in If key points are incorrectly located, high-density point clouds also have a large number of additional points generated outside the mean plane determined by the reconstructed area. The number of outliers increases with a decrease in the density of key points. A fragment of the extracted cloud is shown in This phenomenon is especially pronounced on the borderline between the luminous surface of the sea and the “dry” land. Thus, if there are several wrongly generated points located randomly outside the set of points reconstructing a given object in space, and the number of these points is significantly higher above water, then such an area can be eliminated using a statistical filter ,64. The point cloud is filtered in two phases. The first phase consists of calculating statistical data of the analysed cloud. Let us assume that each point k points in the direct neighbourhood of the point Now, let’s mark the analysed point The average distance to all Mo, without outliers. Outliers are points that are located at a greater distance to k nearest neighbours than a certain threshold value T. This threshold value can be defined as follows:α is a user-defined multiplier and can be determined experimentally for a given point cloud. The resulting point cloud after filtering can thus be written as follows:The first filtration step finishes with the calculation of statistical values. The next filtration stage consists in generating the resultant point cloud The point clouds obtained from photogrammetric flights by iteration have been cleaned according to a procedure adopted and described above, which means that the resulting cloud from each run has been filtered again. In this way, outliers were eliminated. The number of iterations for a given set and the coefficients for each point cloud are shown in A precise fit of the point clouds was performed on previously filtered data. The procedure for aligning the clouds was necessary since the point cloud as of 2018 was developed without GCP. The cloud created in 2019 was taken as a reference cloud. The elements of the rotation matrix and translation vector for the 2018 measurements were calculated using the ICP (iterative closest point) algorithm ,66,67, wThe values representing the fit and filtration results are shown in The M3C2 method described in ,68 was uOver the year, the northern and southern parts of the beach changed significantly. Sand sediment transport due to the influence of wind and undulation on the tombolo   just beiThe figure presented above shows new, forming of accretionary berms, up to 1 m high, formed after early autumn storms. Berm position determines the maximum range of incoming waves and its size represents the storm force. In the Baltic Sea, the berms are up to 3 m high, and on the open Atlantic coast, in places, berms are over 10 m ,70. The sand transmission caused by wind intensifies in the northern and southern parts of the studied area. Erosion dominates in the central part, near the pier entrance. Erosion in this area is caused by intensified human activity, and do not have a natural source. The direct factors include human morphological activities with the use of machines . In highAs shown, the denudation balance of the analysed beach is constant, which means that erosion and accumulation processes counterbalance. This balance is mainly of an anthropogenic origin, and is artificially supported by human activity. Factors indirectly affecting the balance include land-based technical infrastructure and commercial buildings that promote erosion and limit material transmission in the area. Additionally, intensive tourist traffic significantly impacts the balance. This limits the natural ability of the vegetation to grow on the dunes, which intensifies sand transmission in places slightly further from the infrastructure. This artificially maintained balance requires continuous human activity.This article presents the problem of low-level photogrammetry used for the measurement campaign of the unique phenomenon in Sopot. This phenomenon has been monitored for several years , and in Various types of UAVs and other cameras were used in the two analysed cases. These platforms performed a photogrammetric flight according to different plans. Preliminary data validation showed that a direct comparison of the photogrammetric products developed over different years would result in errors and yield unreliable results. These errors are mainly due to differences in data acquisition, software used and other processing settings. Therefore, in the presented research the unification of data processing by adopting common assumptions was proposed. In this procedure, the statistical filtration was applied in order to eliminate outliers and subsequently the ICP algorithm was used to harmonise the content. This operation made possible the comparative analyses and, as a consequence, the identification of changes.High-density point clouds, especially the one developed in 2018, exhibited a significant number of outliers. These points were eliminated by iterative application of the statistical filtration method. This reduced standard deviation, as seen in selected sections, and eliminated incorrectly reconstructed unstable surfaces. This process stage was necessary for the correct and precise mutual fit of the spatial models. In selected, stable data segments located in the studied area, the distance difference was reduced to the mean value of μ = 0.07 mm. The changes taking place on the beach surface were indicated with point clouds fitted in this way.Natural geomorphological processes taking place in the littoral area keep changing its shape and a kind of a specific collision with the infrastructure occurs there. The wind changes the beach surface in annual cycles. This results in covering the infrastructure with sand, creating natural hollows and dunes. Tombolo , which constantly changes the shoreline, also causes the water body near the marina to become shallower . In the


This research investigates the effect of fairness concerns on a sustainable low-carbon supply chain (LCSC) with a carbon quota policy, in which a manufacturer is in charge of manufacturing low-carbon products and sells them to a retailer. The demand is affected by price and the carbon emission reduction rate. The optimal decisions of pricing and carbon emission reduction rate are analyzed under four decision models: (i) centralized decision, (ii) decentralized decision without fairness concern, (iii) decentralized decision with manufacturer’s fairness concern, (iv) decentralized decision with retailer’s fairness concern. The results indicate that the profits in the centralized LCSC are higher than those in the decentralized LCSC with fairness concern. If a manufacturer pays close attention to fairness, the fairness concern coefficient will reduce the carbon emission reduction rate and the profit of the LCSC and increase the wholesale price and the retail price of the product. If a retailer pays close attention to fairness, and the preference of consumers for a low-carbon product is low, the fairness concern coefficient of the retailer increases the total profit of the LCSC and decreases the carbon emission reduction rate and retail price of the product. Otherwise, if the preference of consumers for a low-carbon product is great, the fairness concern coefficient of the retailer would lead to a lower retail price compared with the retail price in the centralized decision and decrease the total profit of the LCSC. The environmental problems caused by the greenhouse effect have become more and more serious . Global In reality, governments guide consumers to buy green products by using carbon labels. For example, China’s energy labeling program was significant in guiding consumers to buy low-carbon and environmentally friendly products, which was launched in 2005 and involved household appliances, automobiles and other industries . ThroughIn order to meet the growing low-carbon demand of consumers, the upstream enterprises of the supply chain have to accelerate the development of emission reduction technologies, and the downstream enterprises have to increase the promotion of low-carbon products. This will increase the operating costs of the supply chain. Thus, low-carbon supply chain (LCSC) management has become an important research topic in recent years.Empirical research shows that the members in a supply chain have fairness preferences, which are often referred to fairness concerns. Specifically, they focus not only on maximizing their own profits, but also on the fair results of profit distribution in the supply chain . For exaTo our best of knowledge, only a few papers have simultaneously studied the impact of carbon emission limit and fairness concern on the optimal prices and carbon emission strategies of members in low-carbon supply chains ,18,19. HThe previous studies related to our study can be classified to three categories, which will be reviewed in the same way like the references ,21,22,23Compared with carbon tax control, total emission control has become a hot topic in academic research. The operational management research based on cap-and-trade systems includes pricing, inventory and emission reduction decisions ,24. Hua The above research on operations and management of LCSCs assumed that decision-makers were completely self-interested. However, in real life, if there is a profit allocation among participants, they are concerned whether they are treated fairly. In the earlier literature, Cui et al.  and Ho eZhao et al.  used gamIn summary, some studies have considered the fairness concerns of the members in the pricing decision-making of a low-carbon supply chain. However, they rarely incorporate carbon quota policies and fairness into the low-carbon supply chain decision-making process at the same time. The study of Liu et al.  was mostWe design a two-echelon low-carbon supply chain (LCSC) which includes a retailer and a manufacturer. The manufacturing process produces carbon emissions. At the beginning of each year, the government gives a free carbon quota to the manufacturer. If the manufacturer’s carbon emissions exceed the quota, it will have to buy carbon credits from a carbon trading market, and when the manufacturer’s carbon emissions are lower than the government’s quota, it can sell the excess carbon credits.A, the manufacturer will buy the carbon emission right in the trading market with a price Firstly, the centralized pricing decision-making and decentralized pricing decision-making processes of the LCSC are studied. Then, the decentralized pricing decision-making process of the LCSC is studied respectively, where dynamic game theory is applied. A flow chart of the decision-making processes in the LCSC is shown in In the studied low-carbon supply chain (LCSC), the manufacturer produces green products and invests in carbon reduction, and the retailer sells green products and has a low-carbon preference. The relationship between the two submits to the master–slave game in which the manufacturer takes the lead.For lucidity and simplicity, the parameters and core variables are expressed in This paper assumes the following:(1) The demand for green products faced by the retailer is not only concerned with the retail price, but also influenced by the low-carbon preference of the consumer. The market demand function can be described by =s−bp+λβ .(2) The manufacturer has to improve its technology with a reduction cost of the carbon emission for reducing the carbon emission. This cost can be described by =1/2⋅kβ2 .β* are set to maximize the profit function of the entire supply chain. The profit function can be expressed asThe LCSC is an idealized organization which aims to maximize the profits in the LCSC. Therefore, the manufacturer and the retailer face the sales market together. The wholesale price determined by the manufacturer is treated as an internal transfer price, which affects the profit of each member but does not affect the total profit. Thus, an appropriate retail price The hessian matrix As the first order principal subformula Next, we substitute If w and the carbon emission reduction rate β, and the retailer can be regarded as a follower who determines the retail price of the product p. Thus, the profit functions can be expressed asIn a decentralized pricing decision-making model, the manufacturer can be regarded as a Stackelberg leader who determines the optimal unit wholesale price of the product w and By adopting the reverse derivation method, the retailer first determines The hessian matrix As the first order principal subformula Next, we substitute The optimal retailer profit Obviously, if The manufacturer has the characteristic of fairness concern, while the retailer has the characteristic of fairness neutrality. As the decision-makers often only focus on the negative unfair utility to themselves, the manufacturer’s fairness concern coefficient u et al. . Then, tThe retailer and the manufacturer also follow the Stackelberg game when the manufacturer is concerned about the fairness. Similarly to Since Through substituting As the first order principal subformula Next, we substitute The optimal manufacturer profit mw, Similarly to Proposition 1.In the decentralized pricing decision, the wholesale price with the manufacturer’s fairness concern is higher than the wholesale price with the manufacturer’s fairness neutrality, while the carbon emission reduction rate with the manufacturer’s fairness concern is lower. The carbon emission reduction rate in the decentralized pricing decision is lower than the rate in the centralized pricing decision.The proof of Proposition 1 is given in Proposition 1 shows that if the manufacturer pays attention to fairness and wants to satisfy its own requirement to fairness, it will not only raise the wholesale price of the product, but also reduce the carbon emission reduction rate.Corollary 1.In the decentralized pricing decision, the retail price with the manufacturer’s fairness concern is higher than the retail price with the manufacturer’s fairness neutrality, and the latter price is greater than the retail price in the centralized pricing decision. Similarly, in the decentralized pricing decision, the order quantity with the manufacturer’s fairness concern is lower than the order quantity with the manufacturer’s fairness neutrality, and the latter quantity is also lower than the order quantity in the centralized pricing decision.The proof of Corollary 1 is given in Corollary 1 indicates that the manufacturer’s fairness concern forces the retailer to raise the retail price of the product and reduce the demand of the product.Corollary 2.In the decentralized pricing decision, the profit of the supply chain in the case of the manufacturer’s fairness concern is smaller than the profit of the supply chain in the case of the manufacturer’s fairness neutrality, and the latter profit is smaller than the profit of the supply chain in the centralized pricing decision.The proof of Corollary 2 is given in Corollary 2 shows that the manufacturer’s fair concern behavior not only reduces its own profit and that of the retailer, but also reduces the profit of the supply chain. This can be explained by the fact that the manufacturer pursues a fair profit distribution by sacrificing part of the profit to punish its rivals.Proposition 2.m and the order quantity In the decentralized pricing decision, the carbon emission reduction rate βThe proof of Proposition 2 is given in Proposition 2 shows that if manufacturer pays attention to fairness, the fairness concern coefficient has an influence on the decision variables.Proposition 3.In the decentralized pricing decision, The proof of Proposition 3 is given in Proposition 3 shows that if manufacturer pays attention to fairness, the fairness concern coefficient has an influence on the profit of all members. This indicates that the manufacturer’s fairness concern coefficient not only hurts its own profit, but also hurts the profit of the downstream retailer.As the retailer has the characteristic of fairness concern and the manufacturer has the characteristic of fairness neutrality, the retailer is concerned with both its own profit and the fairness. By introducing the retailer’s fairness concern coefficient ηr , the utiThe retailer and the manufacturer follow the Stackelberg game where the retailer is concerned with the fairness. Similar to Through substituting As the first order principal subformula Next, we substitute Through substituting Similarly to Proposition 4.The wholesale price is lower than those under the fairness neutrality in the pricing decision.The proof of Proposition 4 is given in Proposition 4 indicates that if the retailer pays attention to fairness, the manufacturer decreases the wholesale price and its carbon emission reduction rates.Corollary 3.(i) The retail price in the decentralized pricing decision with the manufacturer’s fairness concern is lower than the retail price in the decentralized pricing decision with fairness neutrality. (ii) If The proof of Corollary 3 is given in Corollary 3 shows that if retailer cares about fairness, they will reduce the retail price to increase the sales volume.Corollary 4.(i) The profit of the retailer in the decentralized pricing decision with fairness concern is larger than the profit of the retailer in the decentralized pricing decision with fairness neutrality, while the profit of the manufacturer in the decentralized pricing decision with fairness concern is smaller than the profit of the manufacturer in the decentralized pricing decision with fairness neutrality. (ii) Let The proof of Corollary 4 is given in Corollary 4 shows that the retailer’s fairness concern coefficient increases the profit of the retailer and decreases the profit of the manufacturer, and the profit of the supply chain in the decentralized pricing decision is smaller than the profit of the supply chain in the centralized pricing decision. This can be explained by the fact that the retailer pursues the fair profit allocation, which sacrifices the profit of its rival to increase its own profit.Proposition 5.In the decentralized pricing decision, The proof of Proposition 5 is given in Proposition 5 indicates that if the retailer pays attention to fairness, the fairness concern coefficient has an influence on the decision variables.Proposition 6.In the decentralized pricing decision with the retailer’s fairness concern, The proof of Proposition 6 is given in Proposition 6 indicates that if the retailer pays attention to fairness, the fairness concern coefficient has an effect on the profits of the members. Specifically, the retailer’s profit increases with the increase of the retailer’s fairness concern coefficient.s = 300,, Similar to Yang et al. , we giveBy substituting the above parameters into Equation (4), (11), (12), (22), (23), (33) and (34), the relationship among By substituting the above parameters into Equations (3), (5), (13), (14), (24), (25), (35) and (36), the relationship among According to By replacing the above parameters into the profit expressions of the retailer, the manufacturer and the whole LCSC under different decision modes, the relationship among As shown in To verify Corollary 3 and Corollary 4, we increase the consumer’s low carbon preference coefficient by setting As can be seen from In summary, this paper analyzes the impact of the fairness concerns of the supply chain members and the consumers’ low-carbon preferences on supply chain pricing decisions with the carbon quota policy. The research results show that the fairness concerns of members in a supply chain and the low-carbon preferences of consumers have a significant impact on the pricing decisions and sustainability levels of the supply chain. Among them, the fairness concern behavior of the manufacturer harms its own benefit, and the fairness concern behavior of the retailer improves its negotiation ability. Although the manufacturer increased costs due to emission reduction technologies, it is not suitable to consider fairness concerns. The retailer should also control the degree of fairness concerns, otherwise it will reduce the enthusiasm of the manufacturer. Compared with previous studies like Liu et al. (2017), we investigated an optimal pricing decision problem in a low-carbon supply chain from a new perspective. The results of this paper can contribute to understanding the relationship among fairness concern, retail price of the product and carbon emission reduction rate. Thus, this study can help decision-makers of the low-carbon supply chain to select optimal low-carbon and price strategies, which is beneficial to the sustainability of the supply chain.This research studies a LCSC decision-making problem with fairness concern by considering the carbon quota policy. The LCSC includes a manufacturer and a retailer, in which the government gives a free quota of carbon to the manufacturer who applies low-carbon technologies to reduce its emission. The effects of the fairness concern coefficient on the players’ decisions are discussed in the paper.For the studied LCSC, our results show that the total profit of the LCSC under the centralized decision are higher than that under the decentralized decision when fairness is considered. If the manufacturer is concerned with fairness, the fairness concern coefficient further reduces the carbon emission reduction rate and the LCSC’s total profit. If the retailer is concerned with fairness and the consumers have low demand for low-carbon products, the fairness concern coefficient will further boost the LCSC’s total profit. Meanwhile, if the consumers have high preferences for low-carbon products, the fairness concern coefficient of the retailer would result in a lower retail price compared with the retail price of centralized pricing decision-making, and the LCSC’s total profit will be further reduced.Note that our study still has some limitations. First of all, this paper does not consider the double-fairness concern characteristic, in which the manufacturer and the retailer may have fairness concern at the same time. Secondly, the coordination of a low-carbon supply chain achieved by contract can be explored. Finally, cooperative emission reduction may be conducive to improve the sustainable development level of the supply chain. These provide the directions for our future study.