Often enough, social welfare and private benefit do not align for quasi-public goods/services. The inter-basin water transfer (IBWT) project provides a vivid example of this. In this paper, following the game-theoretical approach, we derive an optimal Ramsey pricing scheme to resolve these conflicts. We try to compare traditional supply chain management models with an optimal Ramsey pricing scheme, with an enforcement of coordination among firms. Using simulation techniques, we compute numerical estimates under three regimes: a standard equilibrium decision framework, a coordination decision model and a coordinated Ramsey pricing scheme. Our results show the relative welfare impact of different settings, revealing that the optimal pricing scheme based on the two-part tariff structure cannot only improve social welfare, but also ensure a target profit for participating firms. Lastly, our findings have strong policy implications for the government with profit regulation and the control of water resources. The inter-basin water transfer (IBWT) project involves using artificial methods to divert water from the surplus areas to the deficit regions in order to promote the overall economic and social development of a country. Famous examples in the world include the Central Valley Project (CVP) in the United States ,2, the TWhile water itself is a quasi-public good, its delivery service involves public interests from different stakeholders. Thus, IBWT projects have quasi-public-welfare characteristics: the operations of an IBWT project are inseparable from the government’s intervention and regulation. For example, the CVP is operated under the supervision of the United States Bureau of Reclamation (USBR), the TSWT project is operated under the supervision of Spain Ministry for the Ecological Transition, and the SNWD project is operated under the supervision of the China Ministry of Water Resources. Therefore, IBWT projects should take both private and social welfare into account. Unfortunately, operators typically focus only on profit maximization if there is no government intervention. Then, government regulations such as controls on price, quantity, and quality are critical to keep up the social welfare. Ramsey pricing is believed to offer a suitable perspective for solving the problem. Its idea is to set prices to maximize social welfare subject to a profit constraint . Note thWhen price distortion from the marginal cost is unavoidable, Ramsey pricing provides a second-best solution . VariousAmong the unexplored issues with the Ramsey pricing scheme, we would like to fill the gap by modelling the problem in the framework of coordination games. Our contribution of this paper is to investigate and establish the equilibrium and coordination strategies for an IBWT supply chain with/without Ramsey pricing. We find that an optimal Ramsey pricing scheme through coordination should be pursued to secure those “inherent advantages”. The results are robust.This paper is organized as follows. To maximize social welfare for quasi-public goods/services, Ramsey theory is taken as an optimal pricing scheme for regulated firms with market influence. Conventional applications of Ramsey pricing are well observed. Examples include studies on airports, railroads, ferries, electricity utility industries, health care sectors, pollution controls, and water resource. Morrison 1982), Martin-Cejas (1997), and Zhang et al. (1997) have developed theoretical models to show that Ramsey prices are optimal with cost recovery problems for the uncongested airports 2, Martin,11. A siResearch regarding Ramsey pricing has extended along different technical dimensions. Brock et al. (1985) carried out a dynamic Ramsey pricing analysis under an intertemporal framework and found a strong correlation between marginal and average costs and adjustment costs when spill-over happens . Oum et A new area of research in the context of Ramsey pricing emerges from the monopolist setting. The IBWT projects in China provide excellent opportunities for empirical tests. Especially, the SNWD initiative has inspired a lot of discussions under the lens of game-theory. Researchers may find interesting exercises to integrate the strategic behavior of the players with the Ramsey pricing models. However, the existing literature has rarely touched upon Ramsey pricing in the IBWT supply chain. Currently, the theories, methods and techniques of supply chain management (SCM) have been applied to the study of the operations management of IBWT projects . Ballestero (2004) proposed a decision stochastic approach to determine quantity and price by simulating the recipient’s demand curve and the donor’s supply curve for transferable water . Wang etAlthough the literature regarding Ramsey pricing and the IBWT supply chain exist, none of them explore the strategic operating incentive of the IBWT supply chain under Ramsey pricing. There is also a lack of comparison between operational performance and social welfare among different operation strategies with/without Ramsey pricing in order to determine an optimal policy to maximize social welfare and private benefit. This paper tries to fulfill these urgent needs.An IBWT system can be viewed as an ‘embedded’ supply chain structure, in which a horizontal water supply system is embedded in a vertical water distribution system see . The horith water-intake to the ith distributor is k − 1)th water-intake to the kth water-intake within the horizontal green supply chain is k − 1)th water-intake to the kth water-intake within the horizontal green supply chain is ith water-intake (the water demand for the ith water distributor) is ith water-intake ith water-intake is 3). The bargaining power of the local supplier is In ith distributor with a two-part tariff system, i.e., an entry price (a lump-sum fee) ith consumer is ith water distributor can be expressed in the multiplicative form with an iso-elastic demand curve, i.e., ith water distributor, ith distributor can be derived as: A two-part tariff system is often applied in the water pricing for IBWT projects. In timation ,36, the in price . The invith water distributor, and the IBWT supply chain are as follows: Based on the foregoing parameter settings and model assumptions, the profit functions of the IBWT supplier, the local supplier, the external supplier, the According to the concept of economic surplus , consumeSW for the IBWT supply chain as:On this basis, we can express the social welfare function Based on the modeling notations and assumptions in Under the benchmark equilibrium decision scenario, the detailed decision sequences are as follows: the local supplier and the external supplier will first bargain over the wholesale price ith distributor’s optimal problem is as follows:In the IBWT vertical supply chain equilibrium model, iw as follows:Solving the first-order condition and the second-order derivative of the optimal problem with respect to (w.r.t.) the water retail price Plugging Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the water usage price Plugging Therefore, the equilibrium profits of the IBWT supplier Furthermore, the corresponding consumer surplus and the social welfare are as follows:Plugging w can be expressed as follows: According to the Nash bargaining theory ,42,43,44bw as follows:Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the wholesale price Hereinto,Hence, we can get the bargaining profit of the local supplier and the external supplier in the IBWT horizontal supply chain as follows:ith water distributor an entry price Under the coordination decision scenario without Ramsey pricing, the detailed decision sequences are as follows. The local supplier and the external supplier will first bargain over the wholesale price The optimal pricing problem for the centralized IBWT supply chain can be formulated as follows: ith water distributor as follows:Solving the first-order condition and the second-order derivative of the optimal problem, we can obtain the optimal retail price of the water resources for the ith water distributor is as follows:Thus, the optimal ordering quantity of the water resources for the Plugging the optimal retail price and the optimal ordering quantity of the water resources into the profit function of the IBWT supply chain, we can obtain the target optimal profit of the IBWT supply chain as follows:Furthermore, the corresponding consumer surplus and the social welfare are as follows:ith distributor. The distributors either accept or reject the contract. If the distributors accept, they have to pay an entry price ith distributor’s optimal problem is formulated as follows:In the IBWT vertical supply chain coordination model, the IBWT supplier offers the distributors a two-part tariff contract in which the IBWT supplier charges a usage price iw under the two-part tariff contract as follows:Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the water retail price ith water-intake of the IBWT supplier as follows:Under the two-part tariff contract, to achieve the IBWT supply chain coordination, it is necessary to achieve the coordinated condition: ith distributor, we can get Plugging Therefore, only when the following conditions hold: Hereinto,Plugging w can be expressed as follows: The asymmetric Nash bargaining problem for bargaining over the wholesale price Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the wholesale price Hereinto,Therefore, the coordinated profit of the IBWT supplier On this basis, we can get the bargaining profit of the local supplier and the external supplier in the IBWT horizontal supply chain as follows:ith water distributor an entry price Under the coordination decision scenario with Ramsey pricing, the detailed decision sequences are as follows. The local supplier and the external supplier will first bargain over the wholesale price According to Ramsey pricing theory, the optimal pricing problem for the IBWT supply chain can be formulated as follows: Hereinto, ith water distributor under Ramsey pricing rule as follows:Constructing and solving the Lagrange function of the optimal problem, we can obtain the optimal retail price of the water resources for the Hereinto, ith water distributor is as follows:Thus, the optimal ordering quantity of the water resources for the Plugging the optimal retail price and the optimal ordering quantity of the water resources into the profit function of the IBWT supply chain, we can obtain the target optimal profit of the IBWT supply chain as follows:Furthermore, the corresponding consumer surplus and the social welfare are as follows:ith distributor. The distributors either accept or reject the contract. If the distributors accept, they have to pay an entry price ith distributor’s optimal problem is formulated as follows:In the IBWT vertical supply chain coordination model, the IBWT supplier offers the distributors a two-part tariff contract in which the IBWT supplier charges a usage price iw as follows:Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the water retail price ith water-intake of the IBWT supplier as follows:Under the two-part tariff contract, to achieve the IBWT supply chain coordination, it is necessary to achieve the coordinated condition: ith distributor, we can get Plugging Therefore, only when the following conditions hold: Hereinto,Plugging w can be expressed as follows: The asymmetric Nash bargaining problem for bargaining over the wholesale price Hereinto, rw as follows:Solving the first-order condition and the second-order derivative of the optimal problem w.r.t. the wholesale price Hereinto,Therefore, the coordinated profit of the IBWT supplier On this basis, we can get the bargaining profit of the local supplier and the external supplier in the IBWT horizontal supply chain as follows:The analytical results of the IBWT supply chain coordination and cooperation considering the water delivery loss under the scenario without considering/considering Ramsey pricing, including the optimal/equilibrium solutions and the corresponding profits and social welfare, are summarized in (1) Comparing the analytical results between the equilibrium decision without Ramsey pricing and the coordination decision without Ramsey pricing, (i) the usage prices of water resources under the coordination decision without Ramsey pricing are lower than those under the equilibrium decision without Ramsey pricing; (ii) the retail prices of water resources under the coordination decision without Ramsey pricing are lower than those under the equilibrium decision without Ramsey pricing; (iii) the ordering quantities of water resources under the coordination decision without Ramsey pricing are higher than those under the equilibrium decision without Ramsey pricing; (iv) the profits of the IBWT supply chain and its members under the coordination decision without Ramsey pricing are higher than those under the equilibrium decision without Ramsey pricing; (v) the consumer surplus under the coordination decision without Ramsey pricing is higher than that under the equilibrium decision without Ramsey pricing; (vi) the social welfare under the coordination decision without Ramsey pricing is higher than that under the equilibrium decision without Ramsey pricing.(2) Comparing the analytical results between the equilibrium decision without Ramsey pricing and the coordination decision with Ramsey pricing, (i) the usage prices of water resources under the coordination decision with Ramsey pricing are lower than those under the equilibrium decision without Ramsey pricing; (ii) the retail prices of water resources under the coordination decision with Ramsey pricing are lower than those under the equilibrium decision without Ramsey pricing; (iii) the ordering quantities of water resources under the coordination decision with Ramsey pricing are higher than those under the equilibrium decision without Ramsey pricing; (iv) the profits of the IBWT supply chain and its members under the coordination decision with Ramsey pricing are higher than those under the equilibrium decision without Ramsey pricing; (v) the consumer surplus under the coordination decision with Ramsey pricing is higher than that under the equilibrium decision without Ramsey pricing; (vi) the social welfare under the coordination decision with Ramsey pricing is higher than that under the equilibrium decision without Ramsey pricing.(3) Comparing the analytical results between the coordination decision without Ramsey pricing and the coordination decision with Ramsey pricing, (i) the usage prices of water resources under the coordination decision with Ramsey pricing are lower than those under the coordination decision without Ramsey pricing; (ii) the retail prices of water resources under the coordination decision with Ramsey pricing are lower than those under the coordination decision without Ramsey pricing; (iii) the ordering quantities of water resources under the coordination decision with Ramsey pricing are higher than those under the coordination decision without Ramsey pricing; (iv) the profit of the IBWT supply chain under the coordination decision with Ramsey pricing is lower than that under the coordination decision without Ramsey pricing; (v) the consumer surplus under the coordination decision with Ramsey pricing is higher than that under the coordination decision without Ramsey pricing; (vi) the social welfare under the coordination decision with Ramsey pricing is higher than that under the coordination decision without Ramsey pricing.(4) No matter whether under the equilibrium decision without Ramsey pricing, or under the coordination decision without/with Ramsey pricing, as the water delivery loss rate increases, the profits of IBWT supply chain and its members, the consumer surplus and the social welfare decreases, respectively.(5) Under the coordination decision with Ramsey pricing, as the Ramsey coefficient increases, the profits of IBWT supply chain and its members increases, the consumer surplus decreases, and the social welfare decreases. As this study focuses on exploring the pricing regulation policies and operational strategies for a generic IBWT supply chain, the corresponding numerical and sensitivity analyses are conducted to validate and supplement the foregoing modeling analysis results and derive the general pricing regulation policies and operational strategies for a generic IBWT supply chain. Thus, based on the real characteristics of IBWT projects, e.g., the SNWD project in China , the supply chain structure, the relationships among stakeholders , and the values of the parameters and their relationships in the IBWT supply chain are set to mimic the real-world case.n = 6. We assume that three water distributors are supplied by the local supplier  and three water distributors are supplied by the external supplier . i − 1)th water-intake to the ith water-intake and the water transferring cost from the ith water-intake to ith water distributor are roughly set as Without any loss of generality, an IBWT supply chain with one local supplier, one external supplier, and six water distributors is developed for numerical analysis. Since there are six water-intakes and six water distributors in the IBWT supply chain, i.e., The numerical analysis assesses and compares the pricing and quantity decisions and the resulting profits of the IBWT supply chain and its members, the corresponding consumer surplus and social welfare for the IBWT supply chain equilibrium, and coordination models considering Ramsey pricing or not. The benchmark numerical analysis results of IBWT supply chain equilibrium is shown in The findings from the numerical analysis results are summarized below:(1) Comparing the numerical analysis results between the equilibrium decision without Ramsey pricing  and the (2) Comparing the numerical analysis results between the equilibrium decision without Ramsey pricing  and the (3) Comparing the numerical analysis results between the coordination decision without Ramsey pricing  and the The sensitivity analysis assesses and compares the impacts of the changes of the water delivery loss rate and Ramsey coefficient on the profits of the IBWT supply chain and its members, the corresponding consumer surplus and social welfare for the IBWT supply chain equilibrium and coordination models considering Ramsey pricing or not.The sensitivity analysis results of the water delivery loss rate for the IBWT supply chain equilibrium and coordination decision without/with Ramsey pricing are shown in The sensitivity analysis results of Ramsey coefficient for the IBWT supply chain coordination models with Ramsey pricing is shown in According to the modelling and numerical analysis results and research findings of First, the coordination decision outperforms the equilibrium decision regarding the profits of the IBWT supply chain, the consumer surplus and the social welfare under the scenario without the government’s Ramsey pricing. Hence, the coordination strategy based on the two-part tariff contract could effectively coordinate the IBWT supply chain without the government’s Ramsey pricing and improve the profits of the IBWT supply chain members and is recommended for the optimal operation management of the IBWT supply chain.Second, coordination decision with Ramsey pricing outperforms that without Ramsey pricing regarding the consumer surplus and the social welfare, however, coordination decision without Ramsey pricing outperforms that with Ramsey pricing regarding the profits of the IBWT supply chain and its members. Hence, for the government, it would hope to make pricing decision to achieve social welfare maximization according to Ramsey pricing rule; while, for the IBWT supply chain, they would hope to make pricing decision by themselves to optimize their profits. Owing to the quasi-public-goods characteristics of the water resources and the quasi-public-welfare characteristics of the IBWT projects, the goals of social welfare maximization should be given priority in the operations management of the IBWT supply chain. Therefore, Ramsey pricing regulation is recommended to improve the consumer surplus and the social welfare, and also to guarantee a target profit for the IBWT supply chain and its members in the optimal operations management of IBWT supply chain.Third, the value of Ramsey coefficient hinges on the extent to which the IBWT supply chain profit constraint is binding. The larger the Ramsey coefficient is, the more profits are gained by the IBWT supply chain and its members, and the less consumer surplus and social welfare are achieved by the government. Therefore, a proper Ramsey coefficient is beneficial for balancing the conflict between the IBWT supply chain profits and the consumer surplus and the social welfare.Finally, the water delivery loss rate plays an important role on the profits of all stakeholders in the IBWT supply chain, the consumer surplus and the social welfare. The less the water delivery loss rate is, the more profits are gained by the IBWT supply chain and its members, and the bigger consumer surplus and social welfare are achieved by the government. Therefore, reducing the water delivery loss rate could effectively benefit each stakeholder in the IBWT supply chain.In summary, from the perspective of actual governance and operations, the government should regulate the pricing decision of an IBWT supply chain via Ramsey pricing rule, and an IBWT supply chain would be better off adopting coordination strategy based on two-part tariff contract under the government’s Ramsey pricing. In this way, the consumer surplus and social welfare could be improved. Meanwhile, the IBWT supply chain and its members could achieve the corresponding target profits.Owing to the quasi-public-goods characteristics of the water resources and the quasi-public-welfare characteristics of IBWT projects, both social welfare and economic benefits should be taken into account in the operations management of the IBWT supply chain. Ramsey pricing regulation is introduced to the operations management of the IBWT supply chain to achieve social welfare maximization and, in the meantime, guarantee a target profit for the IBWT supply chain. On this basis, a benchmark equilibrium decision model without Ramsey pricing, a coordination decision model without Ramsey pricing and a coordination decision model with Ramsey pricing for the IBWT supply chain are developed, analyzed, and compared through the game-theoretic and coordination research approaches, and the corresponding numerical and sensitivity analysis for all models are conducted and compared. Finally, the corresponding management insights and policy implications are summarized in this article. The research results indicate that: (1) For the IBWT supply chain, the coordination strategy, based on the two-part tariff contract, could effectively coordinate the IBWT supply chain and improve the profits of IBWT supply chain members, no matter whether the government adopts Ramsey pricing regulation or not. (2) Ramsey pricing regulation could effectively improve the consumer surplus and social welfare, but reduce the profits of the IBWT supply chain and its members. Hence, the government would hope to adopt Ramsey pricing regulation to improve the consumer surplus and achieve social welfare maximization, while, the IBWT supply chain would hope to make price of water resources themselves to optimize their profits without government’s pricing regulation. (3) Owing to the quasi-public-goods characteristics of water resources and the quasi-public-welfare characteristics of IBWT projects, the goals of social welfare maximization should be given priority in the operations management of IBWT supply chain. Therefore, for the government, Ramsey pricing regulation is recommended to improve the consumer surplus, achieve social welfare maximization, and guarantee a target profit in the optimal operations management of IBWT supply chain. (4) The value of the Ramsey coefficient depends on the extent to which the IBWT supply chain profit constraint is binding. A proper Ramsey coefficient is beneficial for balancing the conflict between the IBWT supply chain profits and the consumer surplus and the social welfare. (5) Reducing the water delivery loss rate could effectively improve the IBWT supply chain profit, the consumer surplus and the social welfare.In terms of theoretical work, the existing literature barely covers the pricing regulation policies and operational strategies of the IBWT supply chain. This study provides a novel and useful approach for these issues via Ramsey pricing theory to enhance the optimality for the IBWT projects. In practice, the formulation and numerical analysis provide a solid base for governments to design appropriate pricing and regulatory policies and for IBWT stakeholders to come up with efficient strategies for operations.With the resource constraints and technical difficulties on empirical data collection, this study can only proceed with a game-theoretical decision model of Ramsey pricing using a simulated real-world numerical and sensitivity analyses. All results are robust enough to derive general pricing regulation policies and operational strategies for a generic IBWT supply chain. Admittedly, there are still many important possible extensions for future research. First, supply capacity constraint and water shortage issues can be addressed. Second, the uncertainty factors of random precipitation, water quality, and water environment can be modeled. Third, other types of coordination contracts, such as revenue sharing contract and options contracts, may also be considered. Fourth, water use efficiency may be explored and compared to investigate the impact of the Ramsey price optimization methodology. Finally, the empirical data may be collected, or the actual parameters may be obtained from a real-world case, in order to conduct the corresponding numerical and sensitivity analyses to show the efficiency gain for each party involved.


Heart rate has been measured comfortably using a camera without the skin-contact by the development of vision-based measurement. Despite the potential of the vision-based measurement, it has still presented limited ability due to the noise of illumination variance and motion artifacts. Remote ballistocardiography (BCG) was used to estimate heart rate from the ballistocardiographic head movements generated by the flow of blood through the carotid arteries. It was robust to illumination variance but still limited in the motion artifacts such as facial expressions and voluntary head motions. Recent studies on remote BCG focus on the improvement of signal extraction by minimizing the motion artifacts. They simply estimated the heart rate from the cardiac signal using peak detection and fast fourier transform (FFT). However, the heart rate estimation based on peak detection and FFT depend on the robust signal estimation. Thus, if the cardiac signal is contaminated with some noise, the heart rate cannot be estimated accurately. This study aimed to develop a novel method to improve heart rate estimation from ballistocardiographic head movements using the unsupervised clustering. First, the ballistocardiographic head movements were measured from facial video by detecting facial points using the good-feature-to-track (GFTT) algorithm and by tracking using the Kanade–Lucas–Tomasi (KLT) tracker. Second, the cardiac signal was extracted from the ballistocardiographic head movements by bandpass filter and principal component analysis (PCA). The relative power density (RPD) was extracted from its power spectrum between 0.75 Hz and 2.5 Hz. Third, the unsupervised clustering was performed to construct a model to estimate the heart rate from the RPD using the dataset consisting of the RPD and the heart rate measured from electrocardiogram (ECG). Finally, the heart rate was estimated from the RPD using the model. The proposed method was verified by comparing it with previous methods using the peak detection and the FFT. As a result, the proposed method estimated a more accurate heart rate than previous methods in three experiments by levels of the motion artifacts consisting of facial expressions and voluntary head motions. The four main contributions are as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts ; (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life. Heart rate is an important and popular indicator to monitor cardiac activity. It was traditionally measured by skin-contact sensors such as electrocardiography (ECG), photoplethysmography (PPG), and ballistocardiography (BCG). Although they are accurate methods of standardized measurement, it is often difficult to wear for a long period of time in order to avoid inconvenience, measurement burden, and skin damages . Since tDespite the potential of the vision-based measurement, it has still presented limited ability to be applied in daily life. The main issues regarding the vision-based measurement are to improve signal-to-noise ratio (SNR) and to overcome the noise of illumination variance and motion artifacts . The visRecent studies have investigated remote BCG measured by the ballistocardiographic head movements generated by the contraction of the heart and the ejection of the blood from ventricles into the vasculature . They caTo overcome the limitation of the motion artifacts, previous studies have focused on how to improve the signal extraction and the signal estimation. Shan et al.  extract As shown in Machine learning approaches have been developed to improve the traditional signal processing such as the peak detection and the FFT. They learn by finding hidden features from the training data. Recently, deep learning approaches have been studied by corresponding the structure of the human brain. Especially, convolutional neural networks (CNNs) have achieved state-of-the-art performance in visual classification and time-series problems ,15,16,17This study aimed to develop a vision-based method to measure the heart rate from ballistocardiographic head movements using the unsupervised clustering. Facial videos were taken from three experiments by levels of the motion artifacts as follows: (1) normal, (2) facial expressions, and (3) facial expressions and voluntary head motions. The contribution of this study can be summarized as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts ; (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life.The cardiac signal was estimated from facial videos by measuring the ballistocardiographic head movements and filtering the noise. The signal extraction and the signal estimation of the mentioned remote BCG ,10,11,12To perform the unsupervised clustering, it is necessary to extract the features from the cardiac signal. This study extracted the features from the frequency domain by considering the harmonic component of the heartbeat. First, the cardiac signal was transformed to the frequency domain from the time domain by the FFT. Then, the power density was calculated from the frequency domain between 0.75 Hz and 2.5 Hz corresponding to the normal heartbeat. Finally, the relative power density (RPD) was extracted by the normalization as:The labels corresponding to the features were extracted from the synchronized ECG signals. The 2nd order Butterworth bandpass filter with the cut-off frequencies of 0.75–2.5 Hz was applied on the ECG signals. The heart rate was calculated from the signals by the QRS detection algorithm, which was implemented by Pan and Tompkins to detect the peaks by considering the QRS complex in ECG signal . The datd-dimension, d-dimensional real vector, n) was not determined initially, the candidates were extracted by performing the clustering while increasing n from 3 to N, where N is the number of non-duplicate labels in the dataset. If the kurtosis of each cluster was larger than the one of the gaussian distribution, the candidate was registered into the model.The unsupervised clustering was performed to train a model to estimate the heart rate using the dataset consisting of the features and its labels, as shown in k candidates were extracted in descending order of the distance. The heart rate was finally estimated by averaging the k candidates. Since the optimized k parameter was not known, it was determined by minimizing the differences between labels and estimates in experiments as:Heart rate was estimated from the RPD using the model, as shown in This study evaluated the proposed method from three experiments according to measurement conditions. The measurement conditions were determined by levels of the motion artifacts as follows: (1) normal, (2) facial expressions, and (3) facial expressions and voluntary head motions.Experiment 1: normal. This experiment was to collect the normal dataset without facial expressions and voluntary head motions. The participants consisted of twenty persons . They were asked to sit in front of a camera for 3 min with a stationary state and to keep a neutral facial expression. The facial video was recorded by an RGB webcam (Logitech Webcam C270) at 30 fps and 640 × 360 pixels. The webcam was installed 1 m away from the participant. The ECG signal was simultaneously measured at a sampling rate of 500 Hz by an ECG measurement system with Lead-I . It served as a ground-truth for the evaluation of our method. This experiment was approved by the Institutional Review Board of the Sangmyung University, Seoul, Korea (BE2018-35).Experiment 2: facial expressions. This experiment was to collect the dataset including the motion artifacts by facial expressions. The participants consisted of twenty persons  who also participated in experiment 1. They were asked to sit in front of a camera for 3 min with a stationary state and to follow the six facial expressions  on the monitor. Each facial expression was displayed on the monitor for 30 s in random order to minimize the ordering effect. The facial video was recorded by the webcam used in experiment 1. The ECG signal was simultaneously measured at a sampling rate of 500 Hz by an ECG measurement system with Lead-I  for the ground-truth. This experiment was approved by the Institutional Review Board of the Sangmyung University, Seoul, Korea (BE2018-35).Experiment 3: facial expressions and voluntary head motions. This experiment was to collect the dataset including the motion artifacts by facial expressions and voluntary head motions. This study employed the publicly available MAHNOB-HCI database [database . The parThe facial video and corresponding ECG signal were segmented as a sliding window , as shown in k parameter about the number of candidates for heart rate estimation should be optimized by the experiments. A small k parameter can track instantaneous changes of heart rate, but it is also sensitive to slight variations and can be unstable. On the other hand, a large k parameter is smoothed and stable, similar to the effect of the moving averaging, but it cannot reflect instantaneous changes of heart rate. Thus, it is important to determine the k parameter by optimization of this trade-off. This study tested the differences between labels and estimates according to the k parameter from 1 to 10 in each dataset. The k parameter was determined when the difference was minimum.As described in p-value was less than 0.05. All statistical values were calculated with beat-to-beat heart rate, not average heart rate, for a more rigorous evaluation. The Bland–Altman plot was to interpret the differences between other methods graphically and statistically. The plot was represented by assigning the mean (x-axis) and difference (y-axis) between the two measurements. The 95% limits of an agreement were calculated by mean difference and the 1.96 standard deviation of the difference was represented as lines on the plot.The contribution of the proposed method was to develop an enhanced heart rate estimation by using unsupervised clustering. This study verified the enhancement by comparing the proposed method with previous methods using peak detection  and FFT k parameter for heart rate estimation in the normal dataset without facial expressions and voluntary head motions. The differences decrease until the k parameter is 1 , and then increase continuously until the k parameter is 10 . Thus, the k parameter was defined as 1, which shows the lowest difference.The heart rates estimated using peak detection, FFT, and clustering were evaluated with respected analyses. The Bland–Altman plots of estimated heart rates from the normal dataset without facial expressions and voluntary head motions using peak detection, FFT, and clustering are shown in k parameter for heart rate estimation in the dataset with facial expressions. Similar to the result of the normal dataset, the difference was lowest when the k parameter was 1 . Therefore, it was determined as 1.The heart rates were estimated from the dataset including the motion artifacts by facial expressions using peak detection, FFT, and clustering, as shown in k parameter for heart rate estimation in the dataset with facial expressions and voluntary head motions. The differences decrease until the k parameter is 2 , and then increase continuously until the k parameter is 10 . Thus, the k parameter was determined as 2 that shows the lowest difference.The Bland–Altman plots of estimated heart rates from the dataset including the motion artifacts by facial expressions and voluntary head motions using each estimation method are shown in In this study, the clustering-based method was developed to enhance the heart rate estimation by replacing the traditional peak detection and FFT-based methods. This study evaluated the proposed method on three datasets according to levels of the motion artifacts as follows: (1) normal, (2) facial expressions, and (3) facial expressions and voluntary head motions. The proposed method was better than the peak detection and FFT-based methods in all datasets.Overall, this study has drawn three significant findings. First, this study focused on the development of the heart rate estimation from the cardiac signal estimated from facial videos. The proposed method was demonstrated by comparing it with the peak detection and FFT-based heart rate estimation. In this demonstration, the cardiac signal was estimated using the remote BCG proposed by . This meSecond, the proposed method is a machine learning ML-based method using the unsupervised clustering. It is important to collect the dataset for training the model in the ML-based method. Its performance can be improved if the invariant features are extracted from the dataset, including various noise. In vision-based measurement studies, the illumination variance and motion artifacts are considered as major noise sources. Thus, it is necessary to collect more datasets, including the noise due to the illumination variance and motion artifacts. If the model is constructed with more collected datasets, the accuracy can be improved in a noisy environment.Third, This study explored the issues regarding the remote BCG in the measurement conditions according to the motion artifacts as follows: (1) normal, (2) facial expressions, and (3) facial expressions and voluntary head motions. First, the possibility of overcoming the motion artifacts for the remote BCG was shown by the development of the clustering-based method. Second, the proposed method was demonstrated to better reduce the motion artifacts than traditional peak detection and FFT-based methods and to more accurately estimate the heart rate. Third, the possibility of combining it with existing vision-based measurements and improving their performance. Consequently, our findings represent a significant step towards ensuring the enhanced development of remote BCG.This study estimated the heart rate from facial videos using remote BCG based on unsupervised clustering. The proposed method using the clustering was compared with the previous heart rate estimation using peak detection and FFT. As a result, the proposed method estimated a more accurate heart rate than peak detection and FFT in three experiment datasets with difference levels of motion artifacts. Specifically, it demonstrated the ability to overcome the motion artifacts problem for remote BCG in both stable and unstable environments. Although it showed higher errors than some other remote BCG methods which improve the signal extraction and signal estimation, it is expected to overcome this issue if the heart rate is estimated by the proposed heart rate estimation from the cardiac signal extracted by their signal extraction and signal estimation. The findings are a significant step toward ensuring the enhanced development of remote BCG. This study is expected to help more accurately estimate the heart rate by overcoming the motion artifacts problem and consequently improving the measurement environment in daily life.


Beekeeping is one of the widespread and traditional fields in agriculture, where Internet of Things (IoT)-based solutions and machine learning approaches can ease and improve beehive management significantly. A particularly important activity is bee swarming. A beehive monitoring system can be applied for digital farming to alert the user via a service about the beginning of swarming, which requires a response. An IoT-based bee activity acoustic classification system is proposed in this paper. The audio data needed for acoustic training was collected from the Open Source Beehives Project. The input audio signal was converted into feature vectors, using the Mel-Frequency Cepstral Coefficients  and Linear Predictive Coding. The influence of the acoustic background noise and denoising procedure was evaluated in an additional step. Different Hidden Markov Models’ and Gaussian Mixture Models’ topologies were developed for acoustic modeling, with the objective being to determine the most suitable one for the proposed IoT-based solution. The evaluation was carried out with a separate test set, in order to successfully classify sound between the normal and swarming conditions in a beehive. The evaluation results showed that good acoustic classification performance can be achieved with the proposed system. Digital farming is one of the key technologies which has been gaining support through different research initiatives in the last decade. The fundamental digital farming setup consists of the central management platform , human–cIn the case of mobile networks, the 5G, and the already existing LTE-M1/NB-IoT, enable ubiquitous connectivity, which is important when the system gets distributed over more than one location. The increased number of connected devices, many of them originating from wireless-distributed sensor networks, were also responsible for the last major changes in network protocol standardization, where IPv6 was defined and deployed to communication networks, with the objective to cope with the new situation.If IoT and intelligent ambiance present one of the upfront sectors, agriculture is still a more traditional one. ICT technologies were introduced widely  to agricSwarming has always been seen by beekeepers as an extremely important event, which requires an immediate response. Beekeepers were used to observing the beehive visually to detect swarming. When the beehives were distributed over a larger area, this resulted in increased costs associated with transportation and accessibility. Swarming occurs when specific conditions are met in the beehive. The old queen bee is confronted with a new queen bee, which takes over the hive. The old queen, accompanied by part of the worker bees, leaves the hive and starts a new colony in a new hive. These specific hive conditions can be monitored with an IoT-based module, and the processed information can be used for alarming the beekeeper via a digital farm alarming service. Furthermore, this solution can be integrated into a smart farming system, to establish a complete overview of animals and their conditions.The presented paper has three main objectives. The first one is to propose a concept of an IoT-based bee swarm monitoring system, which can operate with audio signals captured in a beehive. Swarming bees generate a specific audio signal, which can be classified with machine learning approaches. The second objective is to define the most suitable classification configuration for the acoustic monitoring of a beehive, which results from the proposed IoT-based architecture. The basic concept, how to detect swarming from the audio signal, was presented by Zgank . The appThe paper is organized as follows: The review of relevant literature is given in The topic of bee swarm acoustic activity detection using an intelligent ambiance system addresses interdisciplinary knowledge. There were already in 1950 the first attempts to analyze bees’ sounds with simple approaches . New solThe next step was to define the appropriate feature extraction algorithm, which can be used thereafter for the classification task. The authors in  used speMachine learning algorithms can be used for pattern recognition and classification in the case of bee activities’ acoustic monitoring. The Kohonen Self-Organizing Map (SOM) artificial neural networks (ANNs) were applied successfully for the classification task in . MachineBesides the acoustic monitoring, the swarming activity can be detected by monitoring the environmental parameters of a beehive ,23. SomeThe IoT-based concept of the bee swarm activity acoustic classification system is given in There are two main architecture possibilities of how an IoT-based monitoring service can be designed. The crucial point is the IoT module and its properties, namely, processing power and memory. In the case when the IoT module has low processing power and memory, its only function is to capture the audio signal in the beehive and transmit it over a wireless connection to a backend server in a cloud (or on the edge), where acoustic classification takes part. When swarm activity is detected, the beekeeper is alarmed via the service, preferably on a smartphone or wearable device. A variant of such a simple IoT module can carry out feature extraction see  and tranThe opposite design approach is when the IoT module has enough processing power and memory to run acoustic classification locally in the beehive. Here, only the data about the occurring activity is transmitted to the monitoring service, which then alarms the beekeeper. Thus, the amount of transmitted data is as small as possible. The two proposed IoT-based concepts differ from the point of view of how complex processing takes place in the beehive. This issue will be addressed with different models’ topologies and complexities during the acoustic training.The captured beehive audio data contain different bees’ activities, which were studied in detail by behavioral science . Four tyThe bees’ activity acoustic monitoring solution presented in this paper combines all swarm activity indicating sounds into a straightforward solution, where only two classification categories were included in the design. The first category was the normal activity; the second one was swarm activity, despite the original subcategory to which the recording belonged. This design simplification reduces the system’s complexity, which is important with IoT-based solutions.The intelligent ambiance systems, and particularly, the machine learning approaches, are based on the availability of a large amount of data. The area of spoken language technologies has already shown that openly accessible data contribute significantly to the research work in general. To extend this policy to the area of digital agriculture, the open audio data recorded as part of the Open Source Beehives Project  were useThe collected data were either stored locally or transmitted over the IEEE 802.11 Wi-Fi  or a mobThe preliminary study  used onlBeehives can be placed in various geographic locations with a different acoustic background. In the case of a rural environment, the level of background noise can be minimal. Moreover, the audio capturing device is usually positioned inside the beehive, which also reduces the influence of acoustic background on the audio signal. On the other hand, with urban beekeeping, the acoustic background can present an important degradation factor. This parameter was simulated with a city street ambiance noise recording, which was added to original audio data for the last part of the evaluation. The noisy test set was prepared in two versions, one with low-level noise (SNR 42.74) and one with high-level noise (SNR 27.30). All other properties of this augmented noisy test set were identical to the primary test set.The experimental setup was constructed in a way to include architectures and algorithms, which originate from the areas of speech recognition and speech classification ,27. ThusBoth architectures were using an identical first part, which carried out the audio signal preprocessing and feature extraction. The second part performed the actual classification  of the data, and resulted in the final decision on the bees’ activity. All modules will be described in the following subsections.is was converted into a feature vector. The Mel-Frequency Cepstral Coefficients (MFCC) and Linear Predictive Coding (LPC) were selected for this task.The first step needed to perform our acoustic classification was focused on feature extraction, where the input digital acoustic signal The block scheme of Mel-Frequency Cepstral Coefficients feature extraction, which is also used frequently for human speech recognition, is presented in The input audio signal was first preprocessed, which started with the preemphasizing approach to flatten the input signal spectrally. The acoustic modeling procedure needed a signal which was stationary in time, thus, framing was applied. Each frame had 25 ms’ length, and was shifted by 10 ms. The Hamming window was applied to the frames. The signal energy in each frequency band was calculated applying the Mel-scale filter bank see , which, The last step of the MFCC feature extraction procedure was to include a Discrete Cosine Transform, which was applied to convert the log Mel spectrum into the time domain, and in parallel, to reduce the degree of correlation.The included acoustic feature extraction generated 12 Mel-Frequency Cepstral Coefficients. An energy coefficient was added, which is standard practice for similar audio/speech classification tasks . The firThe second, separate, set of feature vectors was again generated with the MFCC algorithm, but, this time, the cepstral mean normalization was applied. The normalization is a frequently used approach, which improves the robustness of the feature extraction procedure, particularly to changing conditions in a recording environment. This could be important, as recordings captured over the longtime period were also part of the audio database.The Linear Predictive Coding was used for comparison as the second feature extraction method. The LPC features are similar to the MFCC features also used frequently in the area of human spoken language technologies. The input audio signal was first preprocessed and windowed using the Hamming window. The error signal  was defined between the original and its model’s representation. The signal was predicted from previous values by a weighted sum, which is called a linear predictor. The resulting system of equations can be solved efficiently with Durbin recursion.Spectral subtraction was included as an additional preprocessing module for the evaluation of background noise. It was only applied to the noisy test set. The objective was to reduce the additive background noise, which was used to simulate the location of a beehive in an acoustically adverse environment. Spectral subtraction functions in the frequency domain, where the spectrum of a noisy signal is computed with Fast Fourier Transform (FFT) . The aveTwo different acoustic models’ topologies were included in the training process in the case of the HMM approach . The HMMThe HMM acoustic modeling training procedure followed the common recipe for both acoustic model topology types. The left–right continuous density Hidden Markov Models with one Gaussian probability density function per state were chosen for the initial acoustic models. Manual check-up of audio data revealed that part of the recordings also included silence. Therefore, a special silence model with a left–right topology was also added to the setup. This modeled the silence part of the signal explicitly. Thus, the acoustic model set included silence, normal and swarm activity. Similar end results could be also achieved if the signal preprocessing stage would be enhanced with a sound activity detection algorithm, which would exclude silence from training. Here, only two acoustic models would be needed. The disadvantage of sound activity detection is the increased complexity of an additional algorithm.The training process was divided into two parts, as it was necessary to improve the audio labels, which originated from the OSPB open audio database. The acoustic training started on the full training set with 1800 balanced recordings. First, the acoustic models’ parameters were initialized. The flat-start approach was used for the parameters’ initialization, where identical global values were assigned to all three models. Two iterations of the Baum–Welch reestimation algorithm were applied to acoustic models to train the models’ parameters. Afterward, the number of mixtures of Gaussian probability density functions was increased by a factor of two. This stepwise procedure was repeated to prevent drastic changes of values. The acoustic models in the first stage were trained until reaching 16 Gaussian mixtures per state. This was sufficient to carry out forced realigning, where training labels were matched to acoustic data. As a result of realigning, the training set contained 1.4% of outliers for 1-state acoustic models, and 1.2% of outliers for 15-state acoustic models. The outliers were excluded from the forthcoming training set, which was done for each type of HMM acoustic model separately.The HMM acoustic models’ training continued with the second stage, where the parameter values of each acoustic model were first initialized with the local values of each class. Thereafter, the acoustic models’ training procedure started with the Baum–Welch algorithm, where three training iterations were carried out. After each training cycle, the number of Gaussian probability density functions was increased by a factor of two. The stepwise training procedure ended when 32 Gaussian mixtures per state were achieved. The final acoustic models of different types and topologies were then used for the acoustic classification task on the test set data.The GMM training procedure was simpler compared to the HMM training procedure, as only one step of model training was applied, with no improvement of the labels’ quality using the forced realigning procedure. The training of the GMM acoustic models started with flat-start parameter initialization, where global values were estimated. The acoustic model training was continued with the Baum–Welch reestimation, which was repeated twice per each training cycle. Afterwards, the number of mixtures of Gaussian probability density functions was increased by a factor of two. The stepwise procedure combining training with Baum–Welch reestimation and increasing the number of mixtures, was repeated until 32 Gaussian PDF mixtures per state were reached. It would be possible to increase the number of Gaussian mixtures per state even further if more acoustic training material would be available, although it could not be guaranteed that the increased amount of training material would always improve the audio classification accuracy.The final GMM acoustic models were included in the GMM classifier, which was fed with the input feature vectors. The GMM classification was carried out on the frame level. The final decision about the class of input acoustic signal resulted from the majority vote, which took place in the post-processing algorithm.In comparison with the HMM procedure, the GMM procedure presented a less complex task, but the necessary GMM classification modules would not always be available in human–computer interfaces of an intelligent ambiance. Therefore, the final design decision should be a compromise between the availability of digital signal processing modules and the complexity of the algorithms.The complexity and number of acoustic models’ parameters are usually reflected directly in the resources  needed by the IoT module, which should carry out the classification task.Acc) was applied as a metric for the evaluation. It is defined as:H is the number of correctly recognized bee activity events, and N is the number of all activity events in the particular test set. The test set results were also evaluated using Precision (P), Recall (R) and F1-score as metrics.The assessment of the proposed IoT-based acoustic classification system was done on a test set with 643 recordings, which were held out from the acoustic models’ training set. The offline evaluation was carried out for different acoustic models prepared during the training phase. Classification Accuracy  than MFCC and MFCC_CMN, but at the cost of lowering the F1-score because of reduced recall.The same configuration of acoustic models and the MFCC feature extraction procedure was used to analyze the computer power and memory needed to carry out the evaluation . The claThe memory size needed for the acoustic models increased proportionally with the number of mixtures of Gaussian probability density functions per state. The size of the smallest model was 25 kB, and the size of the model with 32 mixtures was 355 kB. The real-time factor measured on the test set was between 0.003 and 0.009. The fast RTF was mainly the result of the topology applied for the classification. The computation ratio between feature extraction and classification procedure depended strongly upon the number of mixtures of Gaussian probability density functions which defined the needed time for classification.On the other hand, the time needed for feature extraction was constant, with RTF of 0.002. The classification performance analysis indicated that the proposed system could also be used in an on-line mode of operation.The third part of the evaluation addressed the influence of background noise and denoising procedure on bee activity acoustic classification. The evaluation on the noisy test set was carried out with the MFCC feature extraction and 15-state HMM acoustic models with eight Gaussian PDF mixtures per state. The results are given in The added urban background noise had a significant impact on bee acoustic classification results. Already, the low level of noise degraded the accuracy to 70.76% and the F1-score to 0.83. In the case when a high level of noise was added, the classification degraded severely. The accuracy was 51.17% and the F1-score 0.68. The denoising with spectral subtraction could reduce the degradation in a case of low-level noise, where the accuracy reduced to 71.85%. With the high-level noise, the denoising further degraded the results, achieving an accuracy of 48.83% and an F1-score of 0.66. A possible cause could be the spectral similarity of the input signal and noise at the high level of degradation. The classification results with background noise indicate that the proposed system can, to some degree, handle the background noise.The overall comparison of bee activity acoustic classification accuracy is presented in In general, it can be observed that the proposed IoT-based system copes successfully with the task. If IoT architecture should be considered, the HMM models of high or middle complexity with MFCC features and without cepstral mean normalization are the most suitable choice. Different other machine learning approaches , such asIoT technologies have evolved rapidly over the last decade, and are moving successfully to agriculture, where beekeeping presents one of the important disciplines. The proposed IoT-based acoustic classification system is capable of monitoring bee swarm activity from the captured input audio signal. The system is based on MFCC and LPC feature extraction. Three different types of acoustic models were included in the setup. The best performance was achieved with the system based on HMM acoustic models and MFCC features. This could be important in the case if the proposed system would be part of a larger intelligent ambiance system encompassing various IoT modules. The difference between the GMM and HMM approach in the case of medium topology needs to be analyzed further. The evaluation shows promising results. If some limitations connected with the setup would be taken into account, the system could be used in an IoT environment. The future work will be oriented on the topic of how to improve the accuracy at the level of additional classification categories.


We propose an automatic method to identify people who are potentially-infected by droplet-transmitted diseases. This high-risk group of infection was previously identified by conducting large-scale visits/interviews, or manually screening among tons of recorded surveillance videos. Both are time-intensive and most likely to delay the control of communicable diseases like influenza. In this paper, we address this challenge by solving a multi-tasking problem from the captured surveillance videos. This multi-tasking framework aims to model the principle of Close Proximity Interaction and thus infer the infection risk of individuals. The complete workflow includes three essential sub-tasks: (1) person re-identification (REID), to identify the diagnosed patient and infected individuals across different cameras, (2) depth estimation, to provide a spatial knowledge of the captured environment, (3) pose estimation, to evaluate the distance between the diagnosed and potentially-infected subjects. Our method significantly reduces the time and labor costs. We demonstrate the advantages of high accuracy and efficiency of our method. Our method is expected to be effective in accelerating the process of identifying the potentially infected group and ultimately contribute to the well-being of public health. The most frequent infectious diseases in humans—and those with the highest potential for rapid pandemic spread—are usually transmitted via droplets during close proximity interactions  systems over human beings in solving multiple tasks simultaneously. The accurate identification of this potentially-infected group can only be achieved with an integrative understanding of personal identity, spatial and temporal contexts from the video sequences. Such a wide range of information is processed by individual sub-tasks, including person detection, re-identification, depth and pose estimation. The user study shows that our method is effective in reducing the time and labor costs, and produces consistent results as human screening.We propose a novel framework to automatically evaluate the infection risk based on the principle of To this end, we made the following contributions:We propose a novel framework to evaluate the infection risk of identified individuals. This framework is constructed upon multi-tasking capabilities of modern techniques of computer vision. Our method effectively addresses the problem of infectious disease prevention, greatly reducing labor and time costs.Close Proximity Interaction for assessing and ranking the infection risk. This is achieved by robustly reconstructing the 3D joint trajectories, based on 3D depth and pose estimation. The proposed metric takes distance as well as mutual contact between subjects into account.We quantitatively model the principle of We evaluate our method in real-world environments including indoor office, and other scenarios with massive human traffic . The results show that our automatic method is not only time-efficient but also produces consistent prediction results as human observers.Close Proximity Interaction, including the cornerstones to build this framework. Section 4 presents the results from our experiments, and section 5 discusses the failure cases and limitations of our method. Section 7 concludes this work and points out the directions for future efforts.The rest of this paper is structured as follows. Section 2 summarizes the related works, and section 3 describes the proposed framework to model the principle of Monitoring the spread of infectious disease is critical for taking prompt actions to control the expansion. The contact in close distance between an infectious individual and the population leads to the spread of respiratory infections   diverse visual appearance changes caused by variations in view angle, lighting, background clutter, and occlusion. (2) difficulties in producing discriminating feature representation invariant to background clutter. (3) over-fitting problem due to the limited scale of a tagged dataset.Two types of solutions are proposed to address these problems. One is to learn a more distinctive feature representation to make a trade-off between recognition accuracy and generalization ability. The other is to leverage the Siamese neural network and triplet loss to minimize the loss of images with the same identity and maximize that with different identities. We briefly survey the person re-identification literature from these two aspects in this paper.Improvements in feature representation mainly achieved by leveraging local parts of the person. Representative methods applied part-informed features such as segmentation mask, pose, gait, etc. Pose sensitive model proposed by Saquib Sarfraz et al.  incorporSiamese neural network architecture is also adopted to tackle the problem of person re-identification by taking image pairs or triplets  and motion analysis. Single person pose estimation is based on the assumption that the person dominates the image content. Deep learning methods perform well when the assumption is satisfied. However, for our specific problem in this paper, the case of a single person in one captured image seldom happens. Thus, we focus on the survey of multiple people pose estimation problem here. Cases such as occluded or invisible key points and background clutter lead to significant difficulties for multi-person pose estimation. State-of-the-art approaches built on CNN can be mainly divided into two categories: bottom-up approaches and top-down approaches.Bottom-up approaches  strategy transforms the search space of a simple task to the search space similar to its constitutive complex task  cameras. These cameras are potentially non-overlapping and installed at different locations. The search starts with a diagnosed patient P*, who is confirmed in the clinic with the pandemic disease. The goal of this work is to identify the contact graph (CG) and quantitatively evaluate their potential infection risk (PR) with the principle of close proximity interaction. The workflow of our method is presented in The key contribution of our method is to quantitatively model the principle of Close Proximity Interaction  identifying the potentially-infected group, (2) modeling close proximity interaction.P* across different cameras, (3) constructing the contact graph (CG) by adding the individuals who appear in the same image with the patient P*.The first step of our method is to identify the potentially-infected group of subjects. This includes a couple of sub-tasks: (1) segmenting the persons from the image, (2) re-identifying the diagnosed patient Faster R-CNN . This is based on the assumption that the infection transmitted via droplet is critically related to the physical distance between individuals.We model the principle of close proximity interaction by extracting contextual knowledge from the surveillance videos. The knowledge includes personal identity (acquired from the previous stage), spatial and temporal information. The latter two components describe the movement trajectories of individual persons in the 3D space. These are used o evaluate the extent of interaction proximity among subjects in the contact graph  depth estimation, (2) posture estimation, and (3) risk evaluation.For each edge For the task of depth estimation, we use the existing method . The algorithm is implemented in the Python environment. The deep learning models are implemented with the open-source framework, TensorFlow.The model is trained on COCO dataset for 160k iterations, starting from a learning rate of 0.02 and reducing it by 10 at 60k and 80k iterations. In RPN network, we use 5 scales with box area of the square of 32, 64, 128, 256 and 512 pixels for anchors and 3 ratios of 0.5, 1, 2. There are 256 anchors per image to use for training in total. The Faster R-CNN outputs the individual detection results. The average time cost for this task is 0.011 s.We use the database of CASIA  with an average age of 21 and SD of 3.5 were recruited in this study. They are all undergraduate and graduate students in the department of information science. Written agreement to participate in this study was obtained from individual participant after explanations of this study. They all agreed to join this study for free.Participants were invited to the lab and conducted this experiment. After explaining the task details, they signed the agreement of participation. They were instructed to rank the infection risk of all detected persons in each video, given the diagnosed subject. They were not aware of the purpose of this study, as the comparison baseline of our proposed algorithm.We used all three scenarios  in the previous section. Participants were presented with a short sequence of videos, They were instructed to sort the infection risk of all detected individuals in the image based on common sense or intuition. Starting from the candidate with the highest perceived risk, they associated with the candidate with the rank number from 1 to N (N is the number of candidates in each image). No judging criteria were given. We started the timer when the participant sees the image and started marking it without explicitly informing the user of timekeeping. Interviews were conducted after participants finished the previous procedure by asking open questions and collecting their subjective feedback on how they perceived and ranked the risk. Each participant spent around 20 min to complete the study.We compared the ranking result from our method and the user experiments . The barIn terms of time cost, our method requires far less time (3 s) to process one image, than the time cost required by our participants (2 min). During the decision flow, when the participants ranked the risk order, we observed that it requires significantly less time to identify the person of the highest risk than the rest choices. This is consistent with the high degree of consensus in the candidate selection. We believe this shows the advantages of our method in accuracy and efficiency. The reasoning behind the decision process of human participants is to be explained in the following paragraph.We interviewed the participants and collected their feedback and comments. We asked about how they decided the ranking order, and all participants mentioned the factor of the distance between the candidate and the diagnosed patient. This confirms the principle of close proximity interaction. Six participants explicitly pointed out that the fact that the top candidate is conversing with the diagnosed patient in the examples of the bus station and hospital critically shapes their decision. This is also consistent with the transmission route of the droplet. When people are having a face-to-face conversation, the droplets are more likely to spread out to the person in the conversational group. The carried virus in the droplet causes the infection. For the ranking decision with lower possibility, participants agreed that it is more difficult to decide since more than one candidate is located at a similar distance with the diagnosed patient. However, they also mentioned that since the rest of the candidates are not exposed to the high infection risk, their significance to infection control deserves less attention.In this section, we discuss the insights we learned from our experience, in particular typical failure cases and limitations in our experiments.The building blocks critically determine the success of inferring the close proximity interaction in the upstream workflow. Here we identify the failure cases caused by two components: person re-identification and distance estimation.The state-of-the-art methods in person re-identification still face significant challenges in a complicated environment. The current accuracy of re-identification in our method is 88%. In selected scenarios, the method in our work fails to identify the same person in two different camera views. This is caused by the relative perspective between the person in the view and the camera perspective. Improving the method of re-identification is the solution to this problem. Reliable reconstruction of 3D information from the 2D image is still an open question in the domain of computer vision. Although we propose an efficient method to infer the depth information and integrate with the 2D posture, failure cases still arise due to occlusion and viewpoint perspective. For the former case, if the two persons are standing in line with the camera (shown in the right image of First, only direct infection is considered, while the indirect infection is neglected. Some bacteria or viruses will remain on objects such as escalator rail, doorknob, shopping cart, etc. handled by infected patients. Though their infection may be weakened to varying degrees, it still poses potential threats to indirect infection. We did not take these contaminated objects into account yet. Object detection and tracking techniques will help to locate these objects. It is challenging to accurately determine whether a person is in direct contact with an object rather than just being close to it due to the factors of occlusion and overlapping. When the contaminated object is sheltered from persons or multiple objects overlap each other, the visible part of the object is insufficient to provide sufficient information for making a judgment.Second, formulating the infection risk assessment criteria based on vision-level rather than chemical analysis also presents a unique set of challenges. Obtaining the exact distance between people in practical circumstances is necessary for verifying the estimated distance by our method. Besides, the potential risk of infection varies according to different environmental set-ups and transmission routes of infectious diseases. A confined space like a room may lead to a higher risk than an open space. The cumulative effect of continuously contact over a while rather than a particular moment is difficult to measure. Besides, it is worth pointing out that we do not take the intra-person variations of immunity into account since it cannot be measured at the vision-level.This paper proposes a novel method to represent the potentially-infected group of people as a graph structure. We also model the principle of close proximity interaction by robustly analyzing the physical distance between subjects in the 3D world. This vision-based approach can re-identify diagnosed patients with infectious diseases and evaluate the infection risk of people who have contacted them. We evaluated our method in various scenarios, including indoor office, bus station, bus compartment, hospital. The comparison with the process of manual analysis shows that our method achieves consistent results but significantly reduces the time cost.There are a few directions for our future work. Our current work focuses on the direct contact between the subjects and neglects the indirect contact between subjects via objects. It is highly likely that the objects in close contact with the diagnosed subject contain the virus and thus lead to disease spread. Investigating the indirect infection caused by contaminated objects is in line with our future work. Besides, deploying our method in an in-the-wild study could validate the effectiveness of our method in the real world. One potential scenario is to predict the absentee statistics of the childcare center, given the surveillance camera videos. This could offer advice to parents and administrators concerning the status of the disease infection on both individual and group levels.http://vislab.isr.ist.utl.pt/hda-dataset/.Publicly available datasets were analyzed in this study. This data can be found here: The studies involving human participants were reviewed and approved by School of Informatics, Xiamen University. The patients/participants provided their written informed consent to participate in this study.SG conceived, designed the analysis, and wrote the majority of this manuscript. MJ and XG conceived, designed the analysis, and revised the manuscript. XS conceived the analysis and participated in the writing of the manuscript. JY and HW collected and performed the analysis. FX read and revised the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The present study aimed at investigating invariance of a diagnostic classification model (DCM) for reading comprehension across gender. In contrast to models with continuous traits, diagnostic classification models inform mastery of a finite set of latent attributes, e.g., vocabulary or syntax in the reading context, and allow to provide fine grained feedback to learners and instructors. The generalized deterministic, noisy “and” gate (G-DINA) model was fit to item responses of 1000 male and female individuals to a high-stakes reading comprehension test. Use of the G-DINA model allowed for minimal assumption on the relationship of latent attribute profiles and item-specific response probabilities, i.e., the G-DINA model can represent compensatory or non-compensatory relationships of latent attributes and response probabilities. Item parameters were compared across the two samples, and only a small number of item parameters were statistically different between the two groups, corroborating the result of a formal measurement invariance test based on the multigroup G-DINA model. Neither correlations between latent attributes were significantly different across the two groups, nor mastery probabilities for any of the attributes. Model selection at item level showed that from among the 18 items that required multiple attributes, 16 items picked different rules (DCMs) across the groups. While this seems to suggest that the relationship among the attributes of reading comprehension differs across the two groups, a closer inspection of the rules picked by the items showed that almost in all cases the relationships were very similar. If a compensatory DCM was suggested by the G-DINA framework for an item in the female group, a model belonging to the same family resulted for the male group. An important requirement of construct validity is measurement invariance. Instability of model parameters across different samples of test takers, for example, may call comparability of test results into question and hence its fairness. Structural equation models (SEMs) and item response theory (IRT) have been used routinely to investigate invariance e.g., . Typical3 = 8 possible combinations of them, Diagnostic classification models (DCM) are a class of latent trait models that link item level information on the involved attributes to probabilites of correct responses . To use While the DINA model is non-compensatory  in the DINO model mastery of one of the attributes suffices. The G-DINA model is more flexible and contains DINA, DINO, and other models as special cases. In general, the G-DINA model produces partially compensatory response probabilities like the one displayed. Note that the DINA and DINO models feature guessing parameters (a bit above 22% here) and slipping parameters (chance of slipping is around 10% here), while the parametrization in the G-DINA model is more complex. The Additive CDM ACDM; , linear Next to item parameters, DCMs also provide estimates of mastery probabilities for the population at hand, i.e., for each attribute the percentage of individuals having mastered the attribute can be reported. Similarly, covariances of the latent attributes can be studied, which are useful to understand relationships of attributes.The invariance of the construct of foreign language ability has previously been tested with multi-group confirmatory factor analysis and differential item functioning (DIF) analysis in item response theory (IRT). Invariance analysis is necessary to ascertain that the test scores have the same meaning across groups and the test is fair for everybody. Invariance analysis in language testing research has been conducted across proficiency levels , native Similar to models with continuous latent traits, invariance of item parameters in DCMs is essential: Non-invariance of parameters of DCMs leads to biased classification of individuals across different groups . If this is the case, one cannot claim that the test measures the same construct across the groups. Note that similar to the continuous case, invariance is with respect to item parameters: The latent attribute distribution is allowed to vary across groups, so for example females could have higher or lower mastery probabilities than males.There are few methodological studies e.g.,  investigHowever, helpful these methodological studies might be in shedding light on the invariance property of the DCMs, studies with real data are required to check the tenability of the methodological findings in real applications. To this end, the present study aimed at exploring item property invariance of the G-DINA model across male and female groups in a reading comprehension (RC) test. A minor aim of the present study was to compare performance of two gender groups at attribute level.DCMs provide the means for comparing performances of differing groups of test takers along multiple dimensions besides comparing performance at total score level or via means of continuous latent traits. Total score comparisons provide external estimates of the differences between subgroups performances whereas DCMs offer internal insights into the sources of intergroup differences. Few studies have used DCMs to explore inter-group differences. The main objective of the present study was to explore invariance with the help of the G-DINA model. Since multigroup G-DINA is a recent development in DCM, we illustrated a step-by-step application of the model. Furthermore, DIF and invariance of other model parameters examined across the two gender groups, have implications for the construct validity of the test under study.To follow the objectives of the present study, the following research questions were posed:(1)How stable are item parameters across the gender groups?(2)Is there any statistically significant difference between skill mastery probabilities for the test takers of the two gender groups?(3)Do relationships  among the attributes of RC vary across the two gender groups?Data analyzed in the present study were responses of two equal groups of individuals  who were randomly selected from among the test takers who took the Iranian university entrance examination to enter English programs at master’s level in 2009. For the purpose of the present study, item responses of the selected test takers to the 20 multiple choice RC items were analyzed. It should be noted that guidance of an ethical review board was not sought, as we did not collect the data and analyzed only completely anonymized data provided by the Measurement Organization upon the first author’s formal request. Data of the present study can be made available upon request from the first author.A key component of every DCM is a Q-matrix which represents attribute-by-item relationships. To define the attributes involved in a test, different sources can be used, including theories of content domain, test specifications, content analysis of the test items, and think-aloud protocol analysis . In thisTo determine the attributes that the candidates should have mastered in order to answer the reading comprehension test items, the authors consulted the literature on language ability models in which the reading comprehension attributes and subskills are discussed. The models reviewed included the model proposed by Another model which was consulted in this study is proposed by Other useful sources for identifying the attributes are the previous research conducted in the field of reading comprehension. Therefore, we referred to numerous studies in which reading comprehension attributes were investigated .Reviewing the existing theories and the RC literature, the first author brainstormed with five expert judges and drew a list of 10 attributes and presented it to the judges. The judges were Iranian university instructors who held PhDs in Applied Linguistics with at least 10 years of teaching and researching RC. Prior to coding the test items, they were trained in a 30-min session on how to code the items for attributes. Then, the 20 RC items and the list of 10 attributes were given to each coder. The judges were asked to read the passages and answer the items and code them independently for each attribute. They were asked to rate how sure they were each attribute was necessary for each item on a scale of one to five. We included in the initial Q-matrix attributes which were rated at least four by at least three of the raters.In the next step, we subjected the Q-matrix to statistical analysis through the procedure proposed by Data were analyzed using the CDM package  and the For the purpose of the present study, the sample was split into two halves based on the gender of the test takers. To answer the first research question, item parameters , tetrachoric correlations between the attributes along with their corresponding jackknife standard errors (SEs) were estimated. Thanks to the estimation of SEs, we were able to test the differences between the groups in terms of statistical significance.Moreover, the attribute profiles of the two groups were compared. For the purpose of Question 2, the average mastery probabilities for each attribute across the two ability groups were estimated and the differences were tested for statistical significance.Finally, to answer the last research question, model selection at item level was carried out for the two samples, separately. The models thus selected were compared to investigate stability of the nature of the relationships among the attributes across the two groups. Model selection at item level is a recent development which except for few studies e.g.,  has not To perform multigroup analysis, the following steps were taken:1.The G-DINA model was fit to the data;2.Multigroup G-DINA was applied assuming invariance of the model parameters;3.Multigroup G-DINA was applied assuming non-invariance of the model parameters;4.The models were compared using a likelihood ratio test;5.Item parameters, model fit at item level, attribute mastery proportions, tetrachoric correlations between the attributes, and class probabilities across the two gender groups were compared.First, a single-group G-DINA model was run. As with any other modeling practice, before interpreting DCM results, model fit has to be investigated. Only after sufficient fit of any given DCM to the data  has been evidenced, one can proceed to compare the model with other DCMs . There are an array of absolute fit indices generated by the CDM package. However, the issue of fit in DCMs is in its infancy and there are no cut-offs or significance tests for most of these indices. In the present study, MX2  which isIn the next step, a multigroup G-DINA which assumes invariance of item parameters across the groups was to be tested. However, prior to that a differential item functioning (DIF) analysis should be run . In othep < 0.05). It should be noted that the last column in p-values obtained from an asymptotic normal assumption were corrected, using the false discovery rate (FDR) correction of The results of DIF analysis  show thaIn the next step, two multigroup G-DINA models were compared: one with the assumption of invariance (Mod1) and the other with the assumption of non-invariance of item parameters across the two gender groups (Mod2). Note that mean (and covariance) of attribute profiles were group specific in both models. A significant difference in the log-likelihood values indicates that the nested DCM (here Mod2 which is the model with more parameters) fits significantly better. As t-value” column is obtained from dividing the values in the “value” column by those in the “jackknife_se”. Without a correction for multiple testing absolute t-values of 1.96 and beyond show a significant difference for the respective parameter between the groups, when an asymptotic normal distribution of the t-values is appropriate. Notably, most of the non-invariant parameters were main effects which involved Discourse and Pragmatic. However, the 126 p-values obtained from an asymptotic normal assumption were corrected, using the false discovery rate (FDR) correction of p-values, and as one can see four of the adjusted p-values are not significant.Complementary to the DIF analyses and the global likelihood ratio test item parameter invariance was investigated in detail. Item parameters for the two groups were estimated simultaneously as a multigroup model with the help of the CDM package. Jackknife standard errors were obtained for item parameter differences across the two groups. The jackknife method removes single observations (=response vector) from the dataset one by one, fits the model and averages the calculated estimates. Each Jackknife sample is the original data with a single observation omitted. To estimate the standard errors, 1000 Jackknife samples were used in the present study. In the interest of space, only the parameter differences which were statistically different across male and female groups have been reproduced in Patterns of tetrachoric correlations between the attributes are shown in To further investigate the group differences, the average probabilities of attribute mastery across the two groups were compared. As In order to compare performance of the two groups, we further compared the attribute profiles of the test takers across the male and female samples. For the male group, there were two attribute profiles whose membership was about 10% and higher as follows: [00011] and [10011] populated by about 28 and 12%, respectively. A comparison of the attribute profiles across the two groups shows that the attribute profile [00011] was among the highly prevalent attribute profiles in both groups. Another commonality between the groups was that the two flat attribute profiles of [00000] and [11111] were not prevalent in either male or female groups. A notable pattern in We also examined the relationships among the attributes at item level across the groups. To this end, model selection was carried out at item level see , separatp-values. The better fit of the model with invariant item parameter assumption was corroborated by the fact that a relatively small number of item parameters were significantly different between the two groups. One possible reason for the significant differences mentioned might be the sample size used in the present study. Previous studies .HR came up with the idea, wrote the sections “Introduction” and “Discussion,” and did the analyses. PB wrote the review and helped to write the section “Introduction.” PD wrote the review, helped to write the section “Introduction,” and helped with the analyses and the R codes.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.