In many regions of the world, early diagnosis of non-small cell lung cancer (NSCLC) is a major challenge due to the large population and lack of medical resources, which is difficult toeffectively address via limited physician manpower alone. Therefore, we developed a convolutional neural network (CNN)-based assisted diagnosis and decision-making intelligent medical system with sensors. This system analyzes NSCLC patients’ medical records using sensors to assist staging a diagnosis and provides recommended treatment plans to physicians. To address the problem of unbalanced case samples across pathological stages, we used transfer learning and dynamic sampling techniques to reconstruct and iteratively train the model to improve the accuracy of the prediction system. In this paper, all data for training and testing the system were obtained from the medical records of 2,789,675 patients with NSCLC, which were recorded in three hospitals in China over a five-year period. When the number of case samples reached 8000, the system achieved an accuracy rate of 0.84, which is already close to that of the doctors (accuracy: 0.86). The experimental results proved that the system can quickly and accurately analyze patient data and provide decision information support for physicians. Lung cancer is the second most common type of cancer in the world 11.4%) after female breast cancer (11.7%) and remains the leading cause of cancer-related deaths 1.4% afte. Since pIn many developing countries in Asia, the health and lives of many patients are difficult to protect effectively due to underdeveloped medical technology, large populations, and the failure of patients to be diagnosed and receive effective treatment in a timely manner . For exaHeavy physician burden. The number of patients is large while the number of doctors is small. As a result, doctors are overwhelmed by the repetition of inefficient and cumbersome diagnostic tasks, making it difficult to attend to all patients and delaying the diagnosis or treatment of some patients.Uneven geographical distribution of medical resources. While advanced medical resources are scarce and concentrated in large cities, most patients come from underdeveloped rural areas without better medical conditions or resources, which may delay their treatment, causing their condition to deteriorate.Overtreatment due to doctor misdiagnosis. The lack of advanced medical testing technology and excellent qualified doctors can easily lead to misdiagnosis. Not only does this result in the burden of additional physical treatment as well as high and unnecessary medical costs for the patient, but it also worsens the doctor–patient relationship.In many developing countries, in order to save patients’ lives without advanced medical resources, both doctors and patients have to face several dilemmas. These are as follows:As the number of patients grows, the workload per physician continues to increase. When presented with a large number of case samples, physicians face the challenge of needing a lot of time and energy to handle the complicated diagnostic process and analyze the huge amount of case data as there is no fast and effective decision-making mechanism. To this end, artificial intelligence (AI)-based assisted medical systems can be built to address such problems. In recent years, with the development of 5G, mobile Internet, and AI-related technologies, many intelligent medical systems based on IoT, smart devices ,7, big dCurrently, most of the hot research work in the field of smart medicine is to extract features from the medical images  of patients through machine learning or deep learning techniques, then attempting to train the staging aid diagnosis model of lung cancer to provide clinical decision support for doctors. However, very few hospitals have large and expensive advanced medical equipment such as PET-CT, and the examination process is complicated and cumbersome. The electronic medical record in sensors for patients with NSCLC records a large amount of relevant diagnostic and decision pathology information, such as patient symptoms, signs, and relevant examination information. This data can be obtained through a variety of implantable, wearable, and invasive sensors on the patient’s body that provide continuous remote monitoring of life parameters. It is necessary to reasonably extract features from these medical record data and construct models to achieve more accurate disease diagnoses and to improve the efficiency of doctors’ diagnoses.Semantic features of relevant pathological information can be automatically extracted from medical text data using natural language processing and deep learning algorithms  to trainHowever, many research works are missing important diagnostic and decision-making information when extracting pathological features from medical record data. In addition, the semantic feature extraction of text data is not accurate enough. More importantly, the model performance is affected by the unbalanced proportion of cases in the training dataset across the pathological stages of NSCLC, which results in less accurate diagnoses.To address the above problems, this paper establishes an intelligent medical system for the assisted diagnosis and decision-making of NSCLC. The system obtains the patient’s medical record using sensors in real-time, converts it into corresponding text description data, and uses CNN deep learning technology to automatically extract semantic features and assist in the diagnosis of the patient’s cancer stage. To avoid missing important diagnostic and decision-making information, we pre-selected highly relevant tumor markers that accurately reflect the benignity and malignancy of lung cancer as the main training parameters—using multi-scale convolutional kernels to extract rich semantic features. Samples of patients with NSCLC cases were in the majority of those at more advanced stages. As a result, when training the early-stage prediction model, we used transfer learning techniques to migrate the parameters of the prediction models of most classes of large sample types to the prediction models of small sample types; therefore, achieving the transfer of shared knowledge between the samples of different stages of NSCLC, and avoiding the underfitting of the model due to an insufficient sample size. During the iterative training of the model, dynamic sampling techniques are used to construct positive and negative sample sets to form a balanced training data set, which in turn improved the prediction accuracy and robustness of the model.The main contributions of this paper are:The development of a new CNN-based assisted diagnosis and decision-making intelligent medical system with sensors, which can diagnose the staging of NSCLC patients and provide recommend treatment strategies to physicians by extracting semantic features from the text of highly relevant diagnostic and decision parameters. The system can be used to help physicians assess the effectiveness of patient treatment and adjust the next stage of the treatment plan in a timely manner according to the patient’s recovery.The method of migrating the parameters of large-sample training models to small-sample training models using transfer learning techniques, which realizes the knowledge sharing and solves the impact on model performance caused by the problem of insufficient training samples.The dynamic sampling technique training algorithm is proposed to construct a balanced training set of positive and negative samples for iterative training to improve the accuracy and robustness of the auxiliary diagnosis model.The experimental data were all obtained from real-world NSCLC patient case samples recorded in three hospitals in China. The results show that our proposed new intelligent medical system can approach the diagnostic accuracy of NSCLC staging to the level of real doctors with good performance.The rest of the article is prepared as follows. In Over the years, medical AI has become a hot topic in the research field, and many research publications based on advanced AI technologies have emerged discussing staging and subtype classification identification, early detection, survival analysis, and the assisted diagnosis of lung cancer.The determination of the histopathological stage of lung tumors is of great importance to physicians in both the diagnosis and treatment of patients with NSCLC. In order to report a classification prediction of the pathological stage on CT images of patients with NSCLC, Yu et al.  used a rFor the accurate classification identification of lung cancer and benign and malignant diseases, Yang et al.  showed hRegarding the screening and early risk prediction of whether a patient has lung cancer, Cheng et al.  proposedDue to the low 5-year survival rate of lung cancer patients, there is an urgent need for accurate survival analysis so that doctors can better diagnose and manage treatment for their patients. For this reason, survival analysis systems constructed with the Cox proportional hazards model (CPHM) as the backbone model have emerged ,37,38. IPET image preprocessing  has exceThe above research work has shown that machine learning and deep learning have powerful predictive power for the classification and identification of diseases with high accuracy. However, each of the already proposed medical systems has room for expanded functionality. In comparison to the studies that have been presented, we developed an intelligent medical system with sensors, which is based on CNN to assist diagnosis and decision-making. This system can statistically analyze the medical record of NSCLC with sensors, not only to predict whether a patient’s disease is lung cancer or not but also to diagnose the stage of lung cancer patients and recommend treatment strategies to doctors.Natural language processing techniques have found new applications in NSCLC assisted diagnosis and decision-making systems . IntelliIn order to provide effective diagnostic and decision-making information quickly, we pre-selected important parameters that are highly relevant indicators for lung cancer diagnosis, rather than directly processing massive amounts of raw data, to avoid losing important information. We also adopted multi-scale convolutional kernels to extract rich semantic features. These were used to transfer learning techniques and realize the transfer of shared knowledge from large samples to small samples, employing dynamic sampling methods to construct a balanced sample training set for the iterative training of the model.During the diagnosis and decision-making process of NSCLC patients via intelligent medical systems, the general process consists of five parts , carcinoembryonic antigen (CEA), and carcinoembryonic antigen 125 (CA-125), play an important indicator role in determining the pathological stage of tumors and assessing the treatment effect in patients with NSCLC. To make the system prediction more accurate, other low-degree relevant indicator parameters recorded in the medical record in sensors, such as NSE, PSA, gender, age, symptoms, signs, years of smoking, and family history, are preprocessed and converted into training word vectors.Data pre-training: In the second part, the medical record in sensors for NSCLC patients is pre-processed in a structured way. The textual word vectors are pre-trained using the Skip-gram model of word2vector to transform the selected diagnostic and decision parameters are into a low-dimensional data matrix that can be recognized by the system model.Model reconstruction and training: In the third part, the system model is reconstructed and trained by dividing the dataset using One-Vs-Rest to address the sample class imbalance. This involves the use of transfer learning methods to deal with the problem of slow convergence of models due to the insufficient number of small samples. The processing of the sample training data takes a dynamic sampling approach to improve the robustness and generalization performance of the model.Disease prediction: In the fourth part, we must first determine if the patient is an NSCLC patient. If so, we can proceed to the staging diagnosis. The trained system model will use CNN to extract and map semantic features to the word vector matrix of patient’s medical records, and use the four pathological stages  of NSCLC, initial, early, intermediate, and terminal, as medical record sample category labels for the prediction.Diagnostic decision-making: In the fifth part, the doctor will diagnose the disease based on the staging prediction made by the intelligent medical system on the text data of the new patient’s medical record,. The system will also recommend the corresponding treatment strategy to the doctor, as well as give the corresponding information back to the patient. In the case of stage I or II, the tumor of the NSCLC patient may be benign and highly operable, and surgery is the primary recommendation of the system. In the case of stage III or IV, the tumor is malignant, and the cancerous cells may have metastasized. In this case, the system recommends a comprehensive treatment strategy with radiochemotherapy as the main treatment, and surgery as adjuvant treatment.After a phase of treatment for patients with NSCLC, the system can redo the stage-assisted diagnosis of three tumor marker text data, CYFRA21-1, CEA, and CA-125, which are highly relevant diagnostic and decision parameters recorded in the patient’s medical record in sensors. By comparing the staging decision values calculated by the system before and after treatment, doctors can determine whether a patient’s tumor has progressed from malignant to benign. If the staging decision value decreases significantly, it indicates that the treatment strategy recommended by the system is effective, and the next stage of the treatment plan can be adjusted. Otherwise, the patient’s condition should be further observed before deciding on a treatment strategy.Identifying the pathologic stage of NSCLC patients is an important prerequisite for doctors to develop effective treatment strategies, as the treatment strategies for different NSCLC pathologic stages are different. Firstly, we transformed the training set of four pathological stages of NSCLC into a binary classification training set of four pathological stages using the One-VS-Rest approach to train the prediction models for different NSCLC pathological stages. Then, the deep learning NSCLC staging prediction model was trained on the type with a high sample size, and the model parameters generated by the training are saved. Subsequently, when training the small-sample staging prediction model, the pathological stage with the highest number of co-occurrence with the small-sample pathological stage was selected. The knowledge of the model parameters of this majority class pathology stage was then transferred to the minority class pathology stage model training, which meant that the parameters of the majority class pathology stage prediction model were used as the initialization values of the minority class pathology stage prediction model. The dynamic sampling technique was also used to obtain a balanced data set to train the staging prediction model and improve the performance of the overall NSCLC staging prediction model.In the CNN-based binary classification staging prediction model, the text in the medical record was split into word sequences by word processing. Then, word vectors of the words were pre-trained on medical-related diagnostic and decision parameter text data using Word2Vector’s Skip-gram model to represent discrete word symbols as semantic vectors in a low-dimensional continuous space. Next, after expressing each word of the case as its word vector, a two-dimensional word vector matrix expressing the case was obtained. Finally, the convolutional and pooling operations of CNN were used to extract features from the word vector matrix of cases and perform the binary classification of the four pathological stages. The structure diagram of the NSCLC staging prediction model based on CNN is given in The NSCLC medical record in sensors records natural language descriptions of the patients’ symptoms, signs, and histopathology-related examinations, such as serum, tumor markers CYFRA21-1, CEA, CA-125, etc., which contain important information for physicians to diagnose and make decisions about NSCLC patients. The text data of this important information is transformed into a data matrix by the Skip-gram model pre-trained word vector, which retains a large number of rich pathological semantic features. Then, the CNN deep learning model extracts key pathological semantic features to predict the pathological stage type of NSCLC patients.The Skip-gram model is a model that takes the current word Equation (1) shows the maximized log-likelihood function. where The CNN-based prediction model for NSCLC staging proposed in this paper is shown in Suppose an After generating a two-dimensional matrix representation of the text, the result is input to the convolutional layer, and the convolutional kernel is used to extract semantic features from the training data. Given (1) , can be calculated as in Equation (3).tanh function, etc., The convolution operation is shown in To extract a richer data representation of diagnostic and decision parameters in the case text, the model uses convolutional kernels with multiple windows to obtain more semantic information. After the convolution layer, feature maps with dimensions of varying sentence length are generated. These are usually of large dimensions and difficult to train a suitable classification model directly. These feature maps are used to input the pooling layer for dimension reduction, and at the same time, capture the most important information. In this paper, the model uses the maximum pooling method, and, as a result, the maximum pooling outputs the maximum value in the feature map. After the pooling layer, a fixed-length feature vector is generated , and the feature vector is input into the fully-connected classification layer for sample classification.When there is an imbalance in the training data across pathological stage categories, the CNN tends to classify the samples into the label categories with larger sample sizes in the training data. In the binary classification training data for each pathological stage, the positive sample set is the case sample of that pathological stage, and the negative sample set is the case sample of all other pathological stages. The number of negative samples is much larger than the number of positive samples, resulting in a low recall rate when predicting the stage of NSCLC.In addition, the sample of cases at each pathological stage were uneven, with a larger number of cases at stage III or IV and a smaller number of cases at stage I or II. Moreover, the degree of imbalance in the amount of training data varies widely across pathological stage categories. The imbalance is particularly severe in the small-sample pathological period case data, which are much less likely to be selected into the training set than the large-sample pathological stage data. Therefore, the model’s underfitting learning of small sample case data leads to the low recall of a few classes of pathological stage prediction models, which affects the performance of multi-pathological stage label prediction for NSCLC and cannot meet clinical use.First, the efficient staging prediction model was trained on a large sample of pathological stage cases with a high co-occurrence frequency with the present pathological stage and a sufficient data set. The large sample NSCLC staging prediction model was used as the initialization value for the small sample model. Then, the NSCLC staging prediction model was retrained on the small sample data set. The set of training samples is selected by dynamic sampling during training, the positive and negative samples are sampled separately. Finally, the sampled datasets are merged and used as the training data set for the next round, as shown in The training steps of the proposed algorithm (Algorithm 1) for predicting the pathological stage of NSCLC with small samples, combining transfer learning and dynamic sampling, are shown below:Algorithm 1: The dynamic sampling technique training algorithmInput: Output: The prediction model  Step 1: For any label Step 2:Step 3: The parameters of the pathological stage prediction model Step 4: The positive and negative sample sets are sampled separately based on the sampling probability Step 5:Step 6: If result of calculating the probability that the predicted sample of model  The sampling probabilities of positive samples are regulized, where Step 7: Determine whether the specified number of iterations is reached and return the final classifier if it is satisfied; otherwise, continue with steps 4 to 7 using the new sampling probabilities. So far, after continuous iterative model reconstruction training, the prediction of the model All experimental case data in this paper were collected from three hospitals in China from 2011 to 2015. All patients were aged 45–60 years old, and the data statistics over a five-year period are shown in During the diagnosis of NSCLC, physicians focus mainly on important, relevant reference indicators in order to be able to make efficient decisions. Therefore, the three highly relevant diagnostic and decision parameters with the highest weights in our proposed assisted intelligent medical system are selected: CYFRA21-1, CEA, and CA-125. The values of these three tumor marker parameters are important for disease detection, pathological stage determination, evaluation of treatment efficacy, and the prognosis of patients with NSCLC. In order to extract more comprehensive semantic features of the medical record text in the sensor, and thus improve the accuracy of the model, other low-degree relevant diagnostic parameter indicators are considered, such as other relevant examination indicators, patient symptoms, etc. Among them, the normal ranges of five low-degree relevant diagnostic and decision parameters are shown in After model training, the system model If the decision value calculated by the system is less than 18, the patient is in good health and NSCLC is ruled out, but further tests are needed and confirmed. If the decision value is greater than 18 and less than 57, the patient’s NSCLC tumor is now at Stage I. Again, for decision values within 58–119, the patient is in stage II of NSCLC. Correspondingly, if the decision value is in the Stage III range, the patient is more severely ill at that point. If the decision value is greater than 180, the patient’s NSCLC has progressed to Stage IV, indicating that the patient’s disease has deteriorated to a very serious level.As shown in In all three groups, the mean values of the patients’ CA-125 examined were over the normal range . In particular, the mean CA-125 values of the patients in the second group were all over 480 (KU/L), which means that the mean values were more than ten times greater than the highest threshold of their normal range, as shown in It is difficult to diagnose exactly what stage the NSCLC patient is at from a single index parameter. Therefore, we performed a joint computational analysis of three highly relevant diagnostic and decision-making parameters: CYFRA21-1, CEA, and CA-125. In each of the three groups, one representative patient was randomly selected, and text data from the five examination records of the three selected representative patients with highly relevant diagnostic and decision parameters were used to calculate decision values for staging diagnosis by the NSCLC assisted diagnosis and decision intelligent medical system. The results of the system diagnosis are shown in The identification of the pathological stage of NSCLC patients is very important for physicians to develop effective treatment plans. The treatment strategy for patients with NSCLC is different according to the stage they are in. Accurate diagnosis determines the effectiveness of treatment strategies and thus avoids misdiagnosis or overtreatment. The efficacy of the treatment is mainly based on whether the values of various diagnostic and decision parameters  are significantly decreased after the implementation of the treatment strategy. After each phase of treatment, sensors detect and transmit data values for each diagnostic and decision-making parameter of the patient. The physician keeps track of the patient’s recovery based on the system’s recalculation and analysis. From the three highly relevant diagnostic and decision parameters , the values of the corresponding parameters were high at the first examination. The doctor re-diagnosed and adjusted the treatment strategy after each cycle of treatment. After seven treatment cycles, CYFRA21-1 decreased from 4.16 μg/mL to 3.89 μg/mL, CEA decreased from 285.41 ug/L to 21.17 μg/L, and CA-125 decreased from 711.01 KU/L to 178.2 KU/L. This indicates that the patient’s treatment plan was effective after a series of cycles of therapy.To evaluate the performance of the CNN-based deep learning algorithm for the NSCLC staging prediction model in this paper, we additionally conducted comparative experiments on staging prediction using Naïve Bayes (NB), Classification And Regression Tree (CART), Support Vector Machines (SVM), and Feedforward Neural Network (FNN). The specific parameter configuration of the CNN model is shown in TP is true positive, FN is false negative, TN is true negative, and FP is false positive.In addition, we also evaluated the performance of the CNN model on the diagnosis of each stage of NSCLC by a set of evaluation metrics, as shown in Doctors make dynamic decisions based on the system’s calculations and analysis, adjusting treatment plans and controlling the development of patients’ conditions in real-time, thus achieving precise treatment. Upon initial diagnosis, the decision value calculated by the system was as high as 233.52 greater than the maximum critical value of 180 in the normal range, indicating that the patient’s NSCLC was in stage IV, which was more serious, and the system recommended chemotherapy strategy. After the physician administered two cycles of chemotherapy to the patient, the system recalculated and analyzed that the decision value decreased and was less than 180, and the patient was in stage III, at which point the recommended treatment strategy was adjusted to radiation therapy because chemotherapy is harmful to the patient’s body and should not be performed for a long time. After two more radiotherapy cycles, the parameter values decreased to between 58 and 119. This indicates that the condition of the patient was in remission and in stage II of NSCLC, which was excised by surgery on removable lesion tissue. The decision value dropped to 67.75 at the eighth diagnosis, at which point the main recommendation was drugs for late treatment. With the help of the assisted diagnosis and decision-making intelligent medical system, doctors can qualitatively analyze the condition of NSCLC patients throughout the whole process. Based on the system’s recommendation of specific and effective treatment strategies after each disease monitoring, the doctor can make accurate decisions and track the patient’s treatment in real-time, thus reducing the patient’s pain during the treatment process and effectively saving the patient’s life.However, trained assisted diagnostic and decision-making intelligent medical systems are only an auxiliary doctor role and can never fully replace clinicians in the diagnosis and decision-making of patients with NSCLC. We expect the auxiliary system to obtain accurate information from the various data recorded in the medical record in sensors to help the physician make the yes or no decision, and improve the efficiency of diagnosis. This relieves doctors from the heavy burden of complicated and trivial diagnostic work in the early period, and the processing of a huge amount of patient case data saves the time and energy of doctors, improves the accuracy of patient diagnosis, and thus avoids misdiagnosis or overtreatment.In this paper, we used the Skip-gram model to pre-train word vector matrices on the medical record text data of NSCLC patients, to extract semantic features using the CNN deep learning algorithm, and to use the techniques of transfer learning and dynamic sampling, fusing it into the training process of the system model to build an intelligent medical system for the assisted diagnosis and decision-making of NSCLC. The system accurately stages NSCLC patient case samples to aid in the diagnosis, recommend treatment strategies, and provide physicians with information to support diagnosis and decision-making. After training and testing with data from 2,789,675 patients at three hospitals and comparing experimental analysis with results diagnosed by physicians, the system achieved 84% accuracy, close to that of physician experts, when the case sample data size had reached 8000. The experimental results show that the intelligent medical system we built to assist diagnosis and decision-making can provide doctors with fast and accurate decision-making suggestions, effectively simplify the diagnosis process, saving time, and reducing the load of tedious medical work for doctors.However, we also recognize that the clinical characteristics of a single clinical data source are limited. In addition, the specificity of our proposed model is relatively low due to the lack of obvious disease characteristics at the first stage of NSCLC, and its performance in this aspect will be further improved in the future. Furthermore, the stability and reliability of the system model requires further research and clinical validation, and other types of NSCLC medical sensors detection information will be adopted by the system for a more comprehensive auxiliary diagnosis of lung cancer. We also hope that the proposed model framework for the assisted diagnosis and decision-making of intelligent medical systems can be applied to other cancers.


This paper proposes a privacy-preserving energy management of a shared energy storage system (SESS) for multiple smart buildings using federated reinforcement learning (FRL). To preserve the privacy of energy scheduling of buildings connected to the SESS, we present a distributed deep reinforcement learning (DRL) framework using the FRL method, which consists of a global server (GS) and local building energy management systems (LBEMSs). In the framework, the LBEMS DRL agents share only a randomly selected part of their trained neural network for energy consumption models with the GS without consumer’s energy consumption data. Using the shared models, the GS executes two processes: (i) construction and broadcast of a global model of energy consumption to the LBEMS agents for retraining their local models and (ii) training of the SESS DRL agent’s energy charging and discharging from and to the utility and buildings. Simulation studies are conducted using one SESS and three smart buildings with solar photovoltaic systems. The results demonstrate that the proposed approach can schedule the charging and discharging of the SESS and an optimal energy consumption of heating, ventilation, and air conditioning systems in smart buildings under heterogeneous building environments while preserving the privacy of buildings’ energy consumption. A shared energy storage system (SESS) is a promising technology to efficiently manage the energy consumption in residential and commercial sectors. Compared to individual ESSs, installment of the SESSs reduces the installation costs for individual consumers and minimizes the operation costs of ESS . RecentlConventional algorithms for the energy management of buildings connected to the SESS have been developed using various model-based optimization methods. However, the model-based approach is formulated with approximated constraints for the operation characteristics of appliances so that it may yield an inaccurate and diverging solution. To address the limitation, this study presents a machine learning (ML) method for developing the energy management framework for a SESS with multiple smart commercial buildings.Building energy management systems (BEMSs) are the core solutions for BEMS operators to monitor the real-time building energy usage through smart meters and accordingly, manage the optimal energy consumption of building appliances  systems and lights), thereby reducing the electricity cost while ensuring the comfort and satisfaction of the building’s occupants. Approximately 45% of the total electricity consumption in the building comes from the HVAC system; therefore, the primary goal of the BEMS is to calculate the optimal energy consumption schedule of the HVAC system and consequently, reduce the electricity cost of the building.There have been various optimization methods ) for the energy management of smart residential and commercial buildings while considering their uncertain and dynamic operation characteristics. For the robust optimization method, the range of the values of the uncertain parameters is determined by their continuous set. By contrast, for the stochastic programming method, the uncertain parameters are characterized by a number of discrete probabilistic scenarios. MPC is an optimization control approach that predicts the dynamic system operation characteristics over a predefined horizon time period for minimizing the cost function of the system via optimal control decisions. A robust optimization-based BEMS for a smart building with a wind turbine was developed in  where thRecently, considerable effort has been devoted to developing the optimization-based BEMS algorithms considering the operation of HVAC system. HVAC control methods for minimizing the electricity cost of commercial and university buildings were presented in ,9. In 1, an HVACNote that the aforementioned optimization-based BEMS methods with the SESS highly rely on abstract optimization models and have a limited capability in larger system models with uncertain input data, thereby leading to a significantly high computational complexity and in some cases, even yielding a diverging solution. To handle these limitations from the model-based BEMS with the SESS, deep reinforcement learning (DRL) has recently attracted attention for being a model-free BEMS approach. A DRL algorithm using deep Q-network and deep policy gradient was adopted to classify the patterns of building energy usage and optimize the energy consumption schedule of the building in . In 18]18], a joHowever, in the aforementioned centralized DRL-based frameworks, the consumer’s private data aggregated at a global server (GS) can be exposed to attackers who may exploit these data for finding the energy consumption behavior of consumers. There has been many studies to present energy management methods while considering the privacy of consumers in the residential and commercial buildings ,27,28,29However, to the best of the authors’ knowledge, no study has been performed to construct a privacy-preserving DRL-based SESS model with an FRL-based BEMS model to manage the energy scheduling of the SESS and energy consumption of the HVACs in buildings in a distributed manner. In comparison with our previous study  where noWe present a distributed FRL architecture in which the energy consumption of smart buildings and the energy charging and discharging of the SESS are optimally scheduled within the heterogeneous environments of the buildings while preserving the privacy of energy usage information of individual buildings.We develop a robust privacy-preserving FRL-based BEMS algorithm against building privacy leakage in the hierarchically distributed architecture. During the FRL process, the HVAC DRL agent improves the energy consumption model of the LBEMS through an iterative interaction with the GS and preserves the privacy of energy usage data using a selective parameter sharing method. Subsequently, the SESS DRL agent trains the optimal energy charging and discharging model of the SESS by using the LBEMS agent’s neural network model without sharing the relevant energy consumption data to preserve the privacy of the buildings’ energy consumption.The main contributions of this paper are provided as follows:The rest of the paper is organized as follows. t into state t. T equals to determine a policy s in order to maximize the discounted cumulative rewards that an agent receives from the state-action transition procedure during a finite time horizon T. In a Q-learning method, the policy An MDP is defined as a 5-tuple  method  for updaN local agents. Each agent n constructs its own optimal neural network model n transmits its model As mentioned earlier, FRL is one of the prospective ML methods to train the agents for different local devices in a distributed manner without allowing all agents to share their data for preserving their privacy. The learning process for FRL consists of the following two steps: (i) training for local models and (ii) assembling and updating local models with a newly generated global model. Assume that N LBEMSs, which is shown in n) are periodically transmitted to the GS. Subsequently, the GS aggregates and updates the global model  FRL for HVAC energy consumption scheduling: each HVAC agent in the LBEMS trains its own model to schedule the energy consumption of HVAC using the actor–critic method with its data. The randomly selected part of trained local models  SESS charging/discharging: the optimal HVAC energy consumption models calculated from Step (1) along with the fixed loads in the building are fed back into the GS where the SESS agent trains the model for charging and discharging energy from and to the utility and the LBEMSs using the actor–critic method. The trained discharging schedules are transmitted to the LBEMSs, and these schedules are added to the HVAC energy consumption schedules that are calculated by the HVAC agents in Step (1).We assume a situation in which the LBEMS manages the economic operation of HVACs, a major appliance of smart buildings, with a PV system. Multiple smart buildings are connected to a single SESS to reduce the total electricity cost of purchasing conventional energy from the grid. The charging and discharging operations of the SESS are controlled by the GS. In this study, we present an FRL-based BEMS that consists of a single GS and n chooses and delivers a part of its local neural network model . The primary feature of the selective parameter sharing method is that each LBEMS n and the SESS agent in the GS, respectively, are defined as follows: n at time t. The state t. t. n’s indoor temperature at time n at time t and it provides the population density in the building n where n, respectively. n at time t. n at time t .n is expressed asn. The discomfort cost is formulated as the deviation in the building n’s preferred temperature The reward functions for both HVAC and SESS agents are written as the sum of the negative energy cost and negative penalty cost, which are related to the building’s thermal comfort preference and the SESS operation. To begin with, the reward function for the HVAC agent Here, Prior to the learning procedure, the energy consumptions and discomfort parameters of both HVAC and SESS agent are initialized (line 1).Probability of actions, weights of the actor network and the critic network, Q-value for the HVAC and the SESS agent are initialized (line 2).The global neural network model During every local training episode per communication round, each building’s HVAC agent iterates the following procedures to compute its optimal energy consumption schedule from (a)Sample action (b)Execute action (c)Compute the loss functions of actor network and critic network to minimize the losses and update the model of the HVAC agent using the ADAM optimizer  n using the model The SESS agent infers the energy consumption of the LBEMS (b)Sample an action (c)Compute action (d)Estimate the loss functions of the actor network and the critic network by minimizing them, and update the model of the SESS agent using the ADAM optimizer .Algorithm 1: FRL-based energy management of a SESS with multiple smart buildings.We describe Algorithm 1 for privacy-preserving energy management of the SESS with FRL-based distributed BEMS. The HVAC agents in the LBEMSs and the SESS agent in the GS employ the DRL algorithm based on actor–critic method that is illustrated in Under the TOU pricing, the predicted outdoor temperature and the PV generation output are plotted in The graphs in We compared the performance of our proposed FRL method to that of two existing methods that employ the distributed multi-agent model without FRL and the mixed-integer linear programming (MILP) optimization model, respectively. The simulation result showed that the electricity cost of HVAC systems using the proposed approach deviates only 1.1∼1.4% from the electricity cost of HVAC systems using the distributed multi-agent model without FRL. In addition, we verified from the simulation study that compared to the MILP method, our proposed method enables all three HVAC agents to reduce 24∼32% of energy consumption and 18.6∼20.6% of electricity cost. We evaluated the total computation time for the proposed algorithm, which is defined as the sum of the maximum training time among the three HVAC agents and the training time for the SESS agent. The proposed method requires a total of 512 s  to find their optimal policy. The proposed algorithm is carried out for one day with a 1-h scheduling resolution; it is hence computationally efficient. In addition, we quantified the total computation time for the distributed multi-agent method without FRL. This method requires a total of 430 s . Compared to the distributed multi-agent method without FRL, the proposed approach requires only additional 82 s.In this study, we considered that each LBEMS calculates the energy consumption schedule of only HVAC appliance that consumes the largest energy consumption in the smart building. However, it is possible that various types of building appliances such as refrigerators, elevators and electric vehicles can be controlled by the LBEMS to reduce the electricity cost of the building ,41. In tFor simplicity, we approximated a real-world HVAC system model by focusing on analyzing the impact of the outdoor temperature, the building occupancy, and the HVAC energy consumption on the indoor temperature of the building. In addition, the DRL-based LBEMS was implemented in a single-zone building model rather than a multizone building that has multiple temperature zones. However, we emphasize that our study is the first step toward constructing the privacy-preserving FRL framework to schedule the energy consumption of smart buildings connected to the SESS. An important extension of our study is to construct the FRL-based energy management framework for multizone buildings with a complete model of the HVAC including the operation and air temperature of supply fan and external thermal disturbance ,42, and This paper presented a distributed and privacy-preserving FRL algorithm that conducts the energy management of a smart community that comprised of one SESS and multiple smart buildings equipped with the HVACs. The presented framework has the following two key components: (i) an FRL module that enables the HVAC agent and the GS to collaboratively learn the optimal consumption model of the HVAC while preserving the privacy of buildings’ energy consumption data using a selective parameter sharing method, and (ii) an SESS management module that enables the SESS agent to train the optimal charging and discharging model of the SESS without having the knowledge of the optimal building energy consumption schedule. Simulation results are presented for a smart community comprised of one SESS and three smart buildings with PV systems. The results demonstrated the effectiveness of the proposed approach in terms of training convergence, energy consumption schedule of the HVAC, charging/discharging schedule of the SESS, flexibility under different number of the HVAC agents, decreased energy consumption as compared to the conventional optimization approach, and computation time.


The human eye gaze plays a vital role in monitoring people’s attention, and various efforts have been made to improve in-vehicle driver gaze tracking systems. Most of them build the specific gaze estimation model by pre-annotated data training in an offline way. These systems usually tend to have poor generalization performance during the online gaze prediction, which is caused by the estimation bias between the training domain and the deployment domain, making the predicted gaze points shift from their correct location. To solve this problem, a novel driver’s eye gaze tracking method with non-linear gaze point refinement is proposed in a monitoring system using two cameras, which eliminates the estimation bias and implicitly fine-tunes the gaze points. Supported by the two-stage gaze point clustering algorithm, the non-linear gaze point refinement method can gradually extract the representative gaze points of the forward and mirror gaze zone and establish the non-linear gaze point re-mapping relationship. In addition, the Unscented Kalman filter is utilized to track the driver’s continuous status features. Experimental results show that the non-linear gaze point refinement method outperforms several previous gaze calibration and gaze mapping methods, and improves the gaze estimation accuracy even on the cross-subject evaluation. The system can be used for predicting the driver’s attention. Monitoring a driver’s attention is one of the most vital components of advanced driver assistance systems (ADASs) which monitor the driver’s status and prevents traffic accidents from happening when the driver is distracted ,2,3,4. AEye gaze is often closely related to driving behaviors and is treated as a vital cue for detecting the driver’s visual attention. Many remote gaze-tracking-based techniques, which are based on head and eye pose features extracted from driver videos, have popularly been used in driving scenarios due to their high user friendliness. Gaze mapping is the essential step to build the mapping relationship between the driver’s eye gaze features and the corresponding gaze location for gaze prediction inside or outside the vehicle. Manual gaze mapping methods necessarily require driver compliance ,6 and huA dual-cameras-based driver eye gaze tracking system using non-linear gaze point refinement is presented for deploying a pre-trained supervised gaze model in the unconstrained environment. This method makes an initial attempt to reduce the estimation bias in separate model training. It increases the flexibility of system setup and does not require any human intervention.An effective gaze point non-linear global refinement with two-stage clustering is presented to extract the typical gaze points by maximizing fixation possibilities. This method aligns the initial unknown gaze points to specific calibration points by topology preservation. It is person-independent and can be directly utilized as post-processing for many pre-trained gaze models.Experimental results of real driving scenarios demonstrate that the proposed method reduces the gaze estimation error of the pre-trained model and even has a better performance on cross-subject evaluations. It can be used as a simple-but-effective baseline method for driver gaze calibration or gaze mapping.This paper presents a domain knowledge-based solution to mapping and tracking the driver’s predicted eye gaze points to their real location in the naturalistic dual-cameras-based monitoring system. The driver’s natural mirror-checking actions are used for finding representative gaze points in the context, which are later generated by progressive clustering in an automatic manner. The representative gaze points can be treated as known gaze points for gaze point refinement. The non-linear gaze point refinement treats these representative gaze points as known gaze points, builds simple and straightforward re-mapping for gaze estimation, eliminates the estimation bias related to the session, and makes the pre-trained gaze model more generalized. The main contributions are as follows:The rest of this paper is organized as follows. In The human head and eye dynamics are fundamental to revealing the drivers’ gaze points that represent their current visual attention. Therefore, it has been widely used to detect the visual distraction and understand driver behaviors by exploiting the driver’s head and eye orientation . In earlIn vision-based driver behavior monitoring systems, it is acceptable for the driver’s head orientation regarding as coarse gaze direction . ThroughWang et al.  estimateWith the vigorous development of deep learning technology, various attempts have started being made to achieve eye gaze estimation in the driving environment. All these deep learning-based methods consist of a facial landmark detection or feature extraction module connected to the model input. The network is used to fuse the high-dimensional rich features. Yu et al.  fused apFor the dual-cameras-based system, the gaze points are predicted on the dynamic scene, not just the interior components of the vehicle. The face camera is often fixed at the windshield or center console to capture the driver’s video in the field of view. The scene camera is installed behind the driver to mimic the driver’s view and shows the gaze points in a naturalistic driving environment. Most current driver gaze estimation research studies have evaluated the predicted accuracy of gaze location regression or gaze zone classification inside or outside the vehicle. These studies are based on the domain knowledge in the training dataset and build the gaze model beforehand. It is vulnerable and inapplicable to the uncertain and complex on-road driving environment. Therefore, the driver’s gaze tracking system needs a more flexible setup for the subjects about looking by head movements and eye movements.Gaze calibration is a significant step in the gaze tracking system, which is employed to build the relationship between the human eye and the gaze location. The gaze calibration of driving scenes is usually realized by using typical characteristics of the scene, such as common components inside the vehicle, or the driving behavior characteristics of the driver. Some methods bring the calibration features of the indoor scene as additional prior information.Fu et al.  introducIt is worth noting that the gaze classification accuracy varies significantly between drivers and that the person-specific driver gaze estimation system obviously makes sense. To the best of our knowledge, few studies have aimed to implement the gaze mapping system with person-specific gaze point refinement for a naturalistic driving environment.The proposed method contains three modules: driver status tracking, pre-trained gaze model, and non-linear gaze point refinement. For the dual cameras system, the driver status tracking module infers the status features from the face camera, and the pre-trained gaze model module maps the features to the gaze points on the scene camera.In the driver status tracking module, all the relevant driver status is defined and initialized. The status contains both the signals of the head movement and the eyes movement. Since the observed measurements have random disturbances to a certain extent, the Kalman filter is adopted to track and update the measurements. In the pre-trained gaze model module, the corresponding gaze mapping from the driver status features to the gaze points is built during the offline training. The model is then used to give the initial gaze points. At this point, the initial driver gaze tracking system has been built.In the non-linear gaze point refinement module, the initial gaze points is corrected to the real locations. Firstly, the candidate gaze points are automatically extracted by two-stage gaze point clustering. The forward gaze zone center is measured by density peak clustering. The candidates of the three mirror gaze zones are selected from the maximum distance points on the time sample fragments. All possible gaze zone centers are found by Gaussian mixture cluster, and the parameters are updated by the expectation-maximization algorithm in each gaze zone. Then, based on the known mapping relationships of the candidate gaze points, a simple non-linear method is adopted to further adjust the gaze points on the image plane.The whole framework is running automatically without any manual interpretation. Within it, the non-linear gaze point refinement module periodically refines and updates the gaze mapping parameters, making the framework itself an automatic system. This paper uses a single camera to achieve the signal acquisition of the driver. Since head position, head rotation, and eye rotation all contribute to driver gaze estimation, this paper considers all these signals in the driver status model. This is consistent with the driver’s eye gaze glance in real driving scenarios, as drivers usually move their head and eyes to focus on the target. To perform the prediction step, the Unscented Kalman filter is employed.In this paper, the driver state The head position and head rotation are denoted by In this paper, the corresponding measurement vectors are given as This paper uses the following head model and eye model as the process models for the information processing. In practice, the driver’s head and eyes correspond to the camera coordinate system, respectively. Hence, the driver’s state can be expressed using the head status and eye status.Inspired by Ref. , a constThe eye movement can generally be categorized into three types: fixation; saccades; and smooth pursuit. The eye model describes eye motion that either fixates on the target, quickly changes between fixation points, or smoothly follows the moving target. The correlation between the head movement and the eye movement is considered in the eye model. The previous method used the gaze signal as the combination of the head pose and gaze pose; here, in this paper, the eye rotation provided by the eye model is just the gaze pose. The model describes the eye rotation as the weighted sum of the current eye rotation and the eye rotation generated by the head model:This part introduces the measurement model as the function of the driver state. As the process model, the measurement model consists of the head model and eye model.Head Model: The head position and head rotation is measured in this paper, and the state vector in the measurement model is defined asEye Model: The eye model uses the three-dimensional gaze direction as the state variable. The measurement model is stated ask-th obtained status features in the dataset. In this paper, the traditional driver gaze estimation algorithm is used to train the gaze model when getting enough status samples in the dataset. Generally, the linear or non-linear model regresses the driver status features to specific gaze points on scene image. The pre-trained gaze model should not be over-fitted and has a balanced performance on the labelled training dataset and testing dataset. Mathematically, the gaze model can be written asIt is worth noting that the robustness of the pre-trained gaze model obviously has a significant impact on the next step. If the prediction results of the pre-trained model are particularly poor, it is necessary to add much driver status features to accommodate the data diversity and train a new gaze model. Our goal is to refine the output of the model and make it easy to apply. At this point, it is also practical to build an online model as Ref. . The refIn this section, the main processes of non-linear gaze point refinement are described.Mirror-checking behaviors can be regarded as typical eye glance allocation that benefits from both head and eye movement. The forward-view gaze zone is the major region of the driver’s visual attention, which means that the glancing of the mirror gaze zone goes back and forth from the forward-view gaze zone.x is negative; otherwise, its value equals 1. k-th gaze point and The driver maintains their visual attention on the road by moving their head and eye during the on-road driving, and uses the mirror gaze zones to assist their observation of the surrounding environment. Thus, most of the gaze points should be fixated on the forward-view gaze zone. This paper sets the center of the forward-view gaze zone as the global density center of the initial gaze points which is consistent with the actual situation. Considering the peak density measurement, the center has the characteristics of the largest value in the local neighbor density and the smallest value in the point distance. Hence, the local neighbor density and point distance can be computed byThe mirror gaze zones also have a higher local neighbor density, thence previous works directly clustered the forward-view gaze zone and mirror gaze zones in the same procedure. This matches the relatively dense distribution of the gaze points during long-term driving. However, the complexity and uncertainty make the exceptions inevitable. It is important to increase the effectiveness of the clustering results and improve the speed of the algorithm. Therefore, this paper filters out the candidate gaze points of the mirror gaze zone on the segmented data out of the forward-view gaze zone, and further use the Gaussian mixture clustering method to eliminate the noise.The original gaze points data are converted to binary status data with directionality Mirror-checking behavior is a continuous action with processive time samples. The candidate gaze point for the single action can be the representative gaze point among the time samples. In this paper, the time samples are segmented into an importance fragment M is the components number as well as the number of the mirror gaze zone. In this scenario, the maximum likelihood is defined and estimated as follows:In this paper, the mirror gaze zones are assumed as the region with high local density and modeled by a Gaussian mixture model as single two-dimensional distribution for each mirror gaze zones. The gaze points of each gaze zone are i.i.d. with an underlying density of After obtaining the clustered gaze points, the main task is to assign the unsupervised gaze points with correct gaze zone labels and anchor positions. This paper assigns the clusters by typical topology preservation. As shown in This paper uses the non-linear global refinement to fine-tune the output of the pre-trained model. The corrected positions of the initial gaze points have undergone an unknown transformation relative to the initial values. Therefore, the refinement method should take care of the accuracy of all gaze zones and achieve the balance of gaze points’ adjustment. The refinement method is computed asIn this section, we experiment with several pre-trained models to validate the generalization ability of the proposed system. Firstly, the driving data collection under naturalistic conditions is introduced. Then, the pre-trained model and baseline method for comparisons is given. Subsequently, evaluations of the proposed system are performed. Finally, an ablation study is given on within-subject evaluation and cross-subject evaluation, and error analysis is carried out on gaze estimation accuracy.To evaluate the proposed system, a dataset for the field study is collected in the naturalistic driving environment. The face camera was fixed to the bottom of the windshield and the scene camera was mounted behind the driver. The images captured from the face camera are 720P high-quality images. The frame rate of the camera video is 30FPS.The gaze region in front of the driver is partitioned into 36 gaze zones, as shown in To facilitate the data collection, yellow markers were pasted into the center of every gaze zone, which can easily be noticed in the vehicle cabin. The drivers who participated were asked to look at the marker one after another during the calibration procedure. In the training and testing data for calibration, the dataset retains five driver subjects’ data. For each subject, the dataset has 720 annotated gaze zone data to train the model, and another 720 data to test the model. This means 20 annotated data remain for each gaze zone.In this dataset, we adopted OpenFace  for convIn addition to the dataset, we evaluated the proposed method with the common pre-trained model, Gaussian process for regression (GPR) ,35, partHere, the experiments are conducted on within-subject evaluation, cross-subject evaluation, and normal evaluation. Within-subject evaluation means that testing and training are performed on the same driver subject’s data and the total number of experiments is This section compares the proposed method with the baseline methods on the pre-trained models. The dataset consists of the driver status for each gaze zone, with labels of pixel-level gaze points and annotations of the gaze zone index. The locations of the gaze zones are labeled on the image coordinate axis. For the particular driver, the proposed method will gradually revise the gaze points and build the refinement of the gaze model.The gaze estimation error is calculated in the perspective of the direction angle view and image pixel view. The ground-truth values are provided by the labeled anchors of gaze zone centers. The size of the scene image is This section studies the effect of various gaze zone calibration data. In addition, the error analysis on the horizontal and vertical directions of the image plane is given for further validation. As shown in This paper implements a gaze tracking and mapping system for monitoring driver eye gaze using a face camera and a scene camera. The proposed system applies a non-linear gaze point refinement mechanism to automatically facilitate the gaze prediction of the pre-trained model. Extensive experiments show the potential usage of several gaze models, and give the effectiveness of the proposed method. Future works should further investigate the impact of gaze estimation variance in long-term experiments, and apply it in the deep learning-based gaze estimation method.


The capability of magnetic induction to transmit signals in attenuating environments has recently gained significant research interest. The wave aspect—magnetoinductive (MI) waves—has been proposed for numerous applications in RF-challenging environments, such as underground/underwater wireless networks, body area networks, and in-vivo medical diagnosis and treatment applications, to name but a few, where conventional electromagnetic waves have a number of limitations, most notably losses. To date, the effects of eddy currents inside the dissipative medium have not been characterised analytically. Here we propose a comprehensive circuit model of coupled resonators in a homogeneous dissipative medium, that takes into account all the electromagnetic effects of eddy currents, and, thereby, derive a general dispersion equation for the MI waves. We also report laboratory experiments to confirm our findings. Our work will serve as a fundamental model for design and analysis of every system employing MI waves or more generally, magnetically-coupled circuits in attenuating media. Inspired by the invention of metamaterials7, magnetoinductive (MI) waves were first observed within a chain of magnetic resonant circuits9, and owe their existence to the near field coupling between those elements. MI waves exhibit several exciting advantages that may be exploited for potential applications, such as the simplicity of design, the relatively simple mathematical description of their operation, the full-dimensional penetration , and the wide range of operating frequency, from the MHz to the THz region14. In the last decade, MI waves have gained significant attention as an emerging and favourable communications approach for non-ideal transmission environments20. They were first proposed in underground wireless networks17 and then adapted in underwater communications over the conventional acoustic, optical, and electromagnetic techniques19. Apart from the field of underground/underwater communications, the study of coupled resonant circuits in a dissipative medium has also attracted great interest in medical diagnosis and treatment applications22 because human organs and tissues are moderately conductive at radio frequencies25.Faraday’s law of magnetic induction is the fundamental principles of countless technological applications—from electrical generators, transformers, inductors19. Particularly, the attenuation of the near fields due to the dissipative medium has been primarily accounted by a skin-depth-based-plane-wave model27, where the mutual coupling is assumed to undergo the same decay rate as plane waves propagating in the lossy medium. Recent work28 has explored the validity of that model via simulations and experiments, showing that it is too simplistic and crude. By adopting the mathematical approach in28, the authors in29 developed a model to calculate the mutual inductance between two coils, completely immersed in an infinite homogeneous conductive medium, e.g., buried in the ground or submerged in the water. They showed that the mutual coupling is generally complex and frequency dependent due to the additional contributions from eddy currents. In addition, the eddy currents induced in a non-ideal medium are expected to alter the self impedance of the radiators as the absorption of electromagnetic waves for near-field systems generally creates a feedback effect on the load of the transmitter. Recently, advanced studies31 in wireless power transfer for underwater vehicles have investigated the loss and the detuning effects due to eddy currents in seawater. Although these works proposed an analytical method to quantitatively evaluate the eddy current impacts, some important parameters are obtained through simulation and experimental means31.In contrast to free space, in an attenuating environment a time varying magnetic field induces eddy currents throughout the medium, which inevitably attenuates the inter-element coupling strength due to Lenz’s law. In most current literature, considering only the path loss suffices to describe the effect of eddy currentsIn this paper, we present a general equivalent circuit model for two coupled circuits in infinite homogeneous dissipative media as a combination of circuit theory and field theory. Very different from the previously reported models, our work analytically incorporates all electromagnetic coupling and feedback effects. We further derive and experimentally demonstrate the dispersion relationship of MI waves in a non-ideal medium with variable conductivity.n in the absence of conducting surrounding, n respectively, and We start by considering a general configuration of two coupled coils with the same orientation placed in lossless dielectric cavities , and medium conductivity, 33). The two indentical coils are placed coaxially at a distance of 30 mm from each other. Unlike normal inductive terms which are always considered constant at low frequency, the complex coefficients are indeed frequency-dependent, as shown in Fig. 29.Firstly, the theoretical predictions for the complex Kirchhoff coefficients are compared to those obtained with CST EMS over a range of frequency, entclass1pt{minimaNext, we examine the validity of the analytical model by comparing the theoretical values of the complex coefficients to those obtained with CST EMS for different cavity sizes Fig. e. When t34. Due to high Q-factor of the resonators in free space, the internal resistance of the resonators is negligible. By solving the natural response of the system, the splitting frequency can be approximated as , the dispersion constants can be extracted from the transfer coefficients at elements about the middle of the waveguide. Figure given by , consideWithin the pass band, we find the experimental data and theoretical values for the phase constant in excellent agreement, although towards the edges of the passband there is some deviation. Meanwhile, the agreement for the attenuation diagram is satisfactorily good. We believe the discrepancy in the passband is due to the multiple fields on the measuring probe, and most importantly the strong effect of the eddy currents on the mutual coupling between the measuring probes and the split rings.36. Physically, this complex mutual coupling results in power dissipation of MI waves. In the case of retardation effect in free space, it is radiation loss, while it is eddy current loss in our case. As can be seen in Fig. 37. When the magnitude of the coupling constant between unit cells of two lines is larger than that within a unit cell of a line, forward MI waves occur for a certain range of frequency because the in-phase electromotive forces (emfs) can cancel out the out-of-phase ones. In our case, the existence of backward MI waves is entirely due to the phase of the complex mutual coupling (corresponding to the net phase of the emfs relative to their driving currents) rather than the disparity in modulus of the mutual couplings. The phase difference between currents in two neighbours is now not equal to At the lower end of the passband, MI waves in an attenuating environment exhibit a non-zero phase constant due to the extra phase introduced by the eddy currents. It was reported in the literature that the mutual inductance in free space becomes complex in the presence of the retardation effect when the size of the structure is no longer electromagnetically smallGenerally, excited eddy currents in the transmission medium alter the dispersion of MI waves. This feature may open up an opportunity to tailor the dispersion of MI waves by adjusting the phase of the coupling constants, which can be done by inserting non-ideal media with different permittivity, permeability and conductivity between elements of the waveguide. We expect power losses due to the medium is the trade-off in this scenario. In addition, by validating the general dispersion equation, we also obliquely corroborate the equivalent circuit model for two coupled coils in non-ideal media with the existence of the lossy Kirchhoff coefficients.In a broader context, we have formulated an equivalent circuit model for two coupled resonators inside a homogeneous attenuating medium with the emergence of the complex Kirchhoff coefficients, arising from the impacts of eddy currents. By employing this circuit model, we have derived a general dispersion equation for MI waves in those environments. We experimentally demonstrated this equation for a 1-D axial waveguide made of capacitively-loaded split rings in an aqueous solution with four different conductivities. Our simulation results show that with increasing medium conductivity, MI waves can change from forward to backward waves at the higher branch of the stop band because of the extra phase shift the eddy currents introduce. Even though the turning point, where the MI waves switch their type, cannot be observed in the experiments due to severe signal-to-noise ratios in the branch of evanescent waves, our study may give a new degree of freedom to tailor the dispersion of MI waves based on the phase rather than the strength of the magnetic coupling, leading to unusual MI wave propagation behaviours. It can be engineered by inserting non-ideal media between waveguide elements. The additional losses introduced by the media may be the trade-off in this scenario.39, given that their complex Kirchhoff coefficients are known. We expect that the MI waves in planar waveguides may change their type at the edge of the passband as well, but from backward to forward waves, however, it is notable that the mathematical method to estimate the complex terms in this paper is indeed not applicable to the planar cases. When replacing the dielectric cavities by infinitely wide layers, the boundary value problem therefore becomes: an infinite broad layer of dielectric containing the resonators embedded in an infinite volume of a conductive dissipative environment. It would cause a significant inaccuracy in the prediction of complex coefficients.Although we only determine the dispersion constants of an axial waveguide, the general dispersion is equally valid for planar waveguide configurations, which are immediately relevant to body area networksFinally, we believe that our model can offer insights to the network analysis, design and optimization for any system employing magnetic induction or MI waves in arbitrary media, such as underground/underwater sensor networks, wearable/implanted sensors for biological systems, and wireless power transfer in harsh environments.Fully vectorial numerical simulations, from which all lossy Kirchhoff coefficients in  are extr41.Since there is no natural solid material which has variable electrical conductivity suitable for measurements, the media in the experiments are made of aqueous solutions of sodium chloride  with variable concentration. The conductivity is assessed with a conductivity-meter calibrated with standard calibration solutions . Only the rings that exhibit a resonant frequency of In order to minimize misalignment errors, we designed two symmetric rings attached to the sides of insulating cavities. The insulators were then interconnected via two polytetrafluoroethylene (PTFE) rods through these rings. We carefully aligned our MI waveguides within a n is proportional only to the current flowing in that ring, i.e., n-th element of the array can be rewritten in the form:d is the distance between 2 elements. After a few algebraic operations, the value of n and its six neighbour resonators.The retrieving procedure to extract the propagation constants is as follows. We assumed that the transfer coefficient recorded by the measuring probes at the split ring Supplementary Information


An bH control system is used to adjust bH to maintain an optimum distance between the boom and the crop canopy, and an bH detection sensor is a key component of the bH control system. This study presents a new, low-cost light detection and ranging (LiDAR) sensor for bH detection developed based on the principle of single-point ranging. To examine the detection performance of the LiDAR sensor, a step height detection experiment, a field ground detection experiment, and a wheat stubble (WS) height detection experiment as well as a comparison with an ultrasonic sensor were performed. The results showed that the LiDAR sensor could be used to detect bH. When used to detect the WS height (WSH), the LiDAR sensor primarily detected the WS roots and the inside of the WS canopy. WSH and movement speed of the LiDAR sensor (LiDARV) has a greater impact on the detection performance of the LiDAR sensor for the WS canopy than that for the WS roots. The detection error of the LiDAR sensor for the WS roots is less than 5.00%, and the detection error of the LiDAR sensor for the WS canopy is greater than 8.00%. The detection value from the LiDAR sensor to the WS root multiplied by 1.05 can be used as a reference basis for adjusting bH, and after the WS canopy height is added to the basis, the value can be used as an index for adjusting bH in WS field spraying. The results of this study will promote research on the boom height detection method and autonomous bH control system.Sprayer boom height ( In recent years, as the level of agricultural mechanization in China has increased, boom sprayers have become the primary implements for spraying operations in fields due to their relatively good nozzle atomization performance, relatively large operating widths, and relatively high operating efficiency ,2,3. SinbH) can be detected in real-time with sensors, bH, and the acceleration sensors are used to measure boom vibration. Based on the sensors in bH, which makes it possible to adjust the boom of a sprayer to follow the ground contour, thus maintaining the distance between the boom and ground or distance between the boom and the crop canopy within a stable range as the sprayer passes through an area with a complex and varied terrain. Each John Deere sprayer uses three ultrasonic sensors and one tilt sensor to achieve automatic detection of the position and attitude of the boom. Each Case sprayer uses ultrasonic sensors and adjusting wheels to control bH. Each Amazone sprayer achieves autonomous control and intelligent adjustment of the height and posture of the boom through tilt and ultrasonic sensors.The boom moves primarily by swinging and rotating the boom sprayers. These are the main movement forms that affect the deposition and distribution of droplets in fields. Researchers in China and abroad have conducted a series of studies on the effects of boom movement on the deposition and distribution of droplets and found that a significant difference between the two ends of spray boom can lead to a high coefficient of variation for the distribution of droplets. The number of droplets deposited by a sprayer with a boom swing of 0.3 m can reach three times that deposited by a normal spraying process ,12. LangbH control systems have been developed. Cui et al. [bH control system based on a digital signal processor (DSP) and proposed a signal processing method that integrates limit filtering, data smoothing, and an optimum weight-based data fusion algorithm. This system can effectively control changes in bH and meet field operating requirements. Magalhães et al. [bH using a fractional proportional-integral-derivative (PID) control technique that was more suitable for bH control than conventional PID control techniques. Tahmasebi et al. [bH control systems perform relatively well on flat terrain and plains with large areas of farmland but are unsuitable for high-speed, complex operating settings. Complex farmland grounds require that autonomous bH control systems have relatively high response speeds; however, ultrasonic sensors are prone to disturbance from external factors during the bH detection process, and there are bottlenecks in the research on signal processing algorithms for ultrasonic sensors for high-speed or complex operating settings. As boom sprayers are becoming increasingly large and efficient, it is urgently necessary to make breakthroughs in signal processing algorithms for bH detection sensors or to develop new sensors suitable for bH detection.Ultrasonic sensor-based autonomous i et al. ,15 devels et al.  controlli et al. ,25 contrbH detection based on the principle of single-point ranging. A step height detection experiment, a field ground detection experiment, and a wheat stubble (WS) height detection experiment as well as a comparison with an ultrasonic sensor were performed to evaluate the detection performance of the LiDAR sensor. Meanwhile, a bH detection method based on the LiDAR sensor was proposed to adjust the boom height for spraying on WS fields. The results of this study support research on boom height detection method and autonomous bH control system.The objective of this research was to develop a new, low-cost light detection and ranging (LiDAR) sensor for bH detection and the detection performance experiment for the LiDAR sensor. This paper is structured in five sections, as follows: bH detection was developed based on a low-cost, small-sized, low-power single-point laser rangefinder, TFmini Plus, manufactured by Benewake Co., Ltd. , which measures distances based on the time-of-flight (ToF) principle. During the measurement process, the sensor periodically outwardly transmits a modulated near-infrared wave that is reflected upon contact with an object. The ToF is calculated by determining the difference in phase between the incident and reflected waves. On this basis, the distance from the tested target is determined. This single-point ranging sensor is composed primarily of a front shell, a transmitting lens, a receiver lens, a circuit board, screws, lenses, an ultrasonic welded structure, a back shell, a tail cable, a tail cable fastener, location columns, a mounting hole, a barbed structure, a protective shield, mounting grooves, and a partition structure . This module can convert the signal output of the LiDAR sensor to a 4–20 mA signal output. The single-point LiDAR sensor outputs signals at the transistor-transistor logic (TTL) level and thus has a limited signal transmission distance. The LiDAR sensor needs to be checked prior to use. bH detection by the LiDAR sensor was constructed, as shown in bH values detected by the LiDAR sensor at different locations in the horizontal direction, an S85-MH-5-Y03-OOI laser rangefinder with a measuring range of 10 m and a current signal output of 4–20 mA ) was fixed at one end of the sliding platform. The laser-reflecting plate and the LiDAR sensor were placed at the same horizontal location. The laser rangefinder recorded the horizontal location of the LiDAR sensor by measuring the location of the laser-reflecting plate. An SG-AD-Modbus-8I data acquisition card (DAC)  Co., Ltd.) was used to simultaneously collect measurement data from the laser rangefinder and the LiDAR sensor. The DAC communicated with the upper computer via a RS485 serial port. A general acquisition system for analogue signals designed by Zhai et al. [The detection performance of the LiDAR sensor was mainly affected by the moving speed and the uneven wheat stubble canopy. To investigate the effect of moving speed and uneven wheat stubble canopy on the detection performance of the LiDAR sensor, an experimental platform for i et al.  that canLiDARV) on the detection performance, a step detection experiment was designed. In this experiment, 70 paper boxes (0.30 m × 0.20 m × 0.10 m) were stacked to form a flight of eight steps, as shown in bH detection was manually pushed to change the distance detected by the LiDAR sensor. A tape measure was used to measure the distance of the LiDAR sensor from the ground and each step, and the corresponding measurements were recorded. During the experiment, the center of the step-like ground for detection was placed on a straight line parallel to and at a horizontal distance of 1.0 m from the center line of the track of the sliding platform. The sliding block was used to drive the LiDAR sensor to move along a straight line at a uniform speed. The LiDAR sensor was located directly above the surface of the steps for detection at the site throughout its movement with the sliding block. In addition, the position limit switches were used to control the LiDAR sensor in such a way that the LiDAR sensor took measurements at distance intervals of 5.0 m. According to the experimental design in Zhai et al. [To examine the effects of the movement speed of the LiDAR sensor  of 20 cm at the National Research Station for Precision Agriculture, Beijing, China, as shown in bH during each test were recorded.The experiment was performed in a rotary-tilled field (400 m × 50 m) and a WS field (560 m × 240 m) with a WS height . and the detection error of the LiDAR sensor for the WS roots is greater than 8.00% (the minimum detection error is 8.50%).In summary, bH values detected by the LiDAR and ultrasonic sensors in the rotary-tilled and WS fields, respectively. As demonstrated in bH values detected by the two sensors show similar trends. Currently, ultrasonic sensors are the main type of sensor used to detect bH, and detection data from ultrasonic sensors can reflect changes in bH well, which further demonstrates that LiDAR sensors can satisfactorily reflect changes in bH and can be employed to detect changes in bH. In addition, the bH values detected by the two sensors in the rotary-tilled field are basically the same, whereas there is a difference between the bH values detected by the two sensors in the WS field. Moreover, in the WS field, the bH values detected by the LiDAR sensor were lower than those detected by the ultrasonic sensor. For a boom sprayer, bH changes less in the middle section of the boom than at the two ends of the boom. The changes in bH monitored by the two sensors for the middle section of the boom can reflect the distances detected within the canopy. bH values detected by the two sensors in the middle section of the boom. As demonstrated in LiDARV affects the detection accuracy of the LiDAR sensor and that there is a delay in the detection signal of the LiDAR sensor when there is a significant step-like change in the distance detected by the LiDAR sensor. However, in the field ground detection experiment, a relatively small detection delay is found in the LiDAR sensor for the slope at which the distance detected by the LiDAR sensor changes continuously. This result suggests that significant changes in the crop canopy height during spraying operations can affect the detection accuracy of the LiDAR sensor. When developing an bH detection system, we must develop a signal compensation algorithm to address the signal-delay problem found in the LiDAR sensor. Moreover, there is a decrease in the number of sampling points for detection signals when the LiDAR sensor operates at high speeds. Low amounts of sampled data also affect the detection accuracy of the LiDAR sensor. Thus, it is necessary to study high-speed sampling algorithms to increase the number of effective sampling points for the LiDAR sensor. Currently, the ultrasonic sensors are the primary sensors used to detect bH. There is also a delay in the detection signal of the ultrasonic sensor as the detection speed increases [bH control systems for high-speed or complex operating settings.The comprehensive analysis of the results obtained from the step and field ground detection experiments shows that ncreases . MeanwhiLiDARV and WSH has a greater impact on the detection performance of the LiDAR sensor for the WS canopy than that for the WS roots. The detection height values between the boom and WS roots are less affected by the irregularity of the WS canopy. The detection error of the LiDAR sensor for the WS roots is less than 5.00% (the maximum detection error is 4.66%). Thus, in a spraying operation, it is possible to first classify the detection data obtained by the LiDAR sensor for the WS roots and canopy in a field using the K-means clustering algorithm and to subsequently increase the value detected for the WS roots by 5% as a reference basis for adjusting bH and add the height of the WS canopy to this basis as an index for adjusting bH. This method can reduce the effects of an uneven WS canopy on the detection accuracy of the LiDAR sensor, which is more suitable for bH detection in WS field spraying.During the wheat harvest, the height of the harvester header can be changed according to terrain changes, and such height changes of the harvester header can lead to the irregularity of the WS canopy. The detection position of the ultrasonic sensors is the surface of the WS canopy . The irrbH detection sensors or to develop new sensors for bH detection. Future studies would focus on developing automatic bH control systems based on the LiDAR sensor designed in this study and conducting field experiments at high speeds and universal experiment of different crop canopies at different stages.In recent years, as boom sprayers are becoming increasingly large and efficient, it is urgently necessary to make breakthroughs in signal processing algorithms for bH detection was developed based on the principle of single-point ranging. The results obtained from a step height detection experiment, a field ground detection experiment, and a WS height detection experiment as well as a comparison of the LiDAR sensor developed in this study with an ultrasonic sensor showed that the LiDAR sensor could satisfactorily reflect changes in bH and be used to detect bH. The LiDAR sensor will promote research on the boom height detection method and autonomous bH control systems for high-speed or complex operating settings. The distances detected by the LiDAR and ultrasonic sensors differed within the WS canopy, and the LiDAR sensor primarily detected the WS roots and the inside of the WS canopy. The K-means clustering algorithm can be used to classify the heights detected for the WS roots and canopy in a field. In spraying operations, the detection value from the LiDAR sensor to the WS root multiplied by 1.05 can be used as a reference basis for adjusting bH, and after the WS canopy height is added to the basis, the value can be used as an index for adjusting bH. This method can reduce the effects of an uneven WS canopy on the detection accuracy of the LiDAR sensor, which is more suitable for bH detection in WS field spraying.In this study, a new, low-cost LiDAR sensor for