Understanding attention dynamics on social media during pandemics could help governments minimize the effects. We focus on how COVID-19 has influenced the attention dynamics on the biggest Chinese microblogging website Sina Weibo during the first four months of the pandemic. We study the real-time Hot Search List (HSL), which provides the ranking of the most popular 50 hashtags based on the amount of Sina Weibo searches. We show how the specific events, measures and developments during the epidemic affected the emergence of different kinds of hashtags and the ranking on the HSL. A significant increase of COVID-19 related hashtags started to occur on HSL around January 20, 2020, when the transmission of the disease between humans was announced. Then very rapidly a situation was reached where COVID-related hashtags occupied 30–70% of the HSL, however, with changing content. We give an analysis of how the hashtag topics changed during the investigated time span and conclude that there are three periods separated by February 12 and March 12. In period 1, we see strong topical correlations and clustering of hashtags; in period 2, the correlations are weakened, without clustering pattern; in period 3, we see a potential of clustering while not as strong as in period 1. We further explore the dynamics of HSL by measuring the ranking dynamics and the lifetimes of hashtags on the list. This way we can obtain information about the decay of attention, which is important for decisions about the temporal placement of governmental measures to achieve permanent awareness. Furthermore, our observations indicate abnormally higher rank diversity in the top 15 ranks on HSL due to the COVID-19 related hashtags, revealing the possibility of algorithmic intervention from the platform provider.The online version contains supplementary material available at 10.1140/epjds/s13688-021-00263-0. In our times of information deluge the dynamics of public attention is of eminent importance from many aspects, including education, politics, marketing and governance. In the new media the flow of information has dramatically accelerated, leading often to rapidly changing public attention. At the same time these media provide unprecedented possibilities to study attention dynamics , 2 as thPublic attention becomes a focal issue in times of crises like pandemics. As early as 2010, four years after it was launched, Twitter was shown to be an adequate, real-time content, sentiment, and public attention trend-tracking tool  and was The first COVID-19 epicenter was in China where the service of Twitter is blocked, whereas its local substitute Sina Weibo, is very popular . TherefoSocial media search indices have shown correlations with the epidemic curve –27 and hThe Sina Weibo Hot Search List (HSL) depicts the popularity ranking of the top 50 hashtags in real-time, according to an algorithm in which the number of searches on the hashtags is dominant. The ranking dynamics on Weibo enables us to investigate the dynamics of popular attention in such details which is not available on other platforms. We will focus on the temporal evolution of this ranking in order to study the attention dynamics.Recently, ranking dynamics has been studied widely from sports –32 to scCollective attention towards online news decays with time due to the fade of novelty and attractiveness in the competition with the other news . Recent In this study, we focus on the HSL of Sina Weibo during the period of COVID-19, which provides rich data about public attention and its dynamics in China. Based on that data, we have been able to track the evolution of public attention in different periods during the pandemic, follow how the attention of the population shifted from one group of topics to another and study the changing correlation patterns of different COVID-related topic categories with the evolving COVID-19 situation in the world. During our studies we have discovered signatures of the possible algorithmic control on this social media platform. Understanding the dynamics of public attention on social media promotes instant and effective communications among governments, health experts and the public, helping the government to monitor public opinion, maintain the stability of the society as well as develop and deliver more effective measures to minimize the effects of the pandemic.The paper is organized as follows: in Sect. Sina Weibo is the biggest Chinese microblogging website, with MAU (monthly active users) reaching 550 millon and DAU (daily active users) 241 million in March 2020 . Instead”   and their interactions with patients in hospitals. The Science category incorporates scientific understandings of the virus properties, vaccine development, and ways for public protection given by authoritative doctors. The Supports category takes into account hashtags on worldwide donations and emotional supports. For ambiguous cases which contain information of more than one category, our classifications were based on the focus of the main subject. Due to the syntactic-semantic complexity of Chinese language, the classifications were made by two independent annotators. Final consensus was reached in case of disagreement. The Mainland China sub-categories are summarized in Table T is the time index set, the Pearson’s correlation is calculated using the percentage change series To further understand how the Mainland China related COVID-hashtags are correlated with each other and with the daily number of infections/deaths/recoveries in the three separated time periods, we measured the Pearson’s correlations between the seven series of daily number of new hashtags in each of the sub-categories defined above, together with the three series of daily number of infections/deaths/recoveries. The correlation of these ten time series are calculated using the percentage change between the current and the prior element instead of the actual value in order to reduce the effect of the trend which can cause spurious correlations. For time series category One natural measure of social media attention towards a topic category is the quantity of the related hashtags. The growing pattern of the cumulative number of hashtags on the HSL with time reflects the dynamics of the public attention. We separately measured the growth of the cumulative number of all hashtags and all COVID-related hashtags that ever appeared on the HSL in our observation period. To understand how much COVID-information occupies the HSL at each timestamp, we constructed the historical ratio trajectory of the COVID-related hashtags on the HSL since the first COVID-hashtag #The lifetime duration of a hashtag on the HSL indicates the ability of obtaining persistent attention from the public. We quantified the duration (continuous existence on the HSL) of a hashtag with τ: j. α whose first appearance was on day i. We compared the duration of the hashtags across various categories and different time scopes. We compared the duration of the hashtags before the outbreak on January 19, all COVID-related hashtags, and non-COVID hashtags after the outbreak. To ensure complete life cycles of the hashtags, we took all hashtags whose first arrivals on the HSL are between December 19, 2019 and January 18, 2020 as the sample for hashtags before the pandemic, which includes 6161 in total. Similarly, we took all COVID-hashtags whose first arrivals are no later than April 14, with a total number of 8808. For the non-COVID hashtags after the outbreak, we took a random sample of all non-COVID hashtags with the same size as the COVID sample. Hashtags that reappeared after disappearing from the HSL were excluded from our calculation. To understand the overall attention variation towards COVID-hashtags with time, we investigated the daily value of their cumulative average duration. We denote k at some point during a given length of time. Rank diversity is known to give characteristic profiles for different types of systems; e.g., in open systems (where only the top part of the competing items is ranked) behaves differently from closed systems  [The changes in the ranking patterns of the hashtags at different time periods reflect the general public attention dynamics. Rank diversity ent}d(k)  is defin ranked) , 33. In j. α whose first appearance was on day i. The public attention towards a hashtag can also be indicated by its highest rank during the lifetime on the HSL. The highest rank of a hashtag reveals its highest ability and achievement when competing for attention with the other hashtags. We studied the highest rank distribution of the classified COVID-hashtags and compared the results with the hashtags before the outbreak as well as the non-COVID hashtags after the outbreak  Similarly, the average duration is another measure of attention. However, in the latter case it should be mentioned that short duration can be caused by decaying attention to the general topic (in this case the hashtag is likely to be replaced by another from a different topic) or because of the heavy stream of new hashtags of the same topic.How do the average highest rank and average duration accumulate with time? As Fig. The fast decay of In this work, we have studied the public attention dynamics on the real-time Hot Search List (HSL) of the biggest Chinese microblogging website Sina Weibo under the influence of the COVID-19 pandemic. On the one hand, such study contributes to the understanding of the dynamics of public attention on social media and how it reflects the dynamics of the public thoughts and behaviors. On the other hand, identifying the online attention dynamics patterns and their relationship to events and measures during pandemic may contribute to its efficient management.In order to follow the dynamics of public attention we have introduced sub-categories of COVID-19-related hashtags. Our results show diversification of the public attention after the outbreak on January 19, 2020 as indicated by changing frequencies of such hashtags in the different sub-categories. Moreover, the pattern of correlations with the real-world events and measures vary in three identified periods during the investigated time span. We conclude that at the beginning the dominant driving force of the public attention was the infection and death situation in Mainland China, with mainly domestic cases, while the international situation and the imported cases influenced the attention later. Our observations point towards the complexity of the attention patterns indicating that several components should be taken into account if such data are used to the prediction of the epidemic curve , 27.Furthermore, we have shown that the cumulative average duration follows exponential decay immediately after the attention peak in the pandemic, but a slower decay for a longer time. The exponential decay suggests that the speed of governmental response is crucial in the early pandemic phase. This exponential decay sets a scale for the governments within which it should take quick actions and publish crucial measures and regulations to control the pandemic, healthcare experts should deliver scientific knowledge to inform the public how to protect themselves efficiently.The attention towards COVID-hashtags decayed as the circumstances in China got better. Nevertheless, the attention was influenced by the world pandemic situation which kept changing, hence the decay of public attention on the Chinese social media Weibo has become less clear cut. In any case, targeted and timely stimulus should be given to keep the attention and awareness of the public throughout the pandemic to prevent future waves of COVID-19.In this paper we have made the first step to relate the ranking dynamics of hot topics on social media with the public attention dynamics. We have provided a novel approach to study and quantify the attention dynamics taking advantage of the real-time Hot Search List (HSL) on Weibo. The rank diversity in the top 15 ranks containing COVID-hashtags are higher than normal. This could result from the spontaneous preference from the public towards COVID-related information. More likely, it is due to an algorithmic intervention towards COVID-hashtags from the platform provider. Sina Weibo may intentionally promote the COVID-related important information to make sure people will get aware of them. In this sense, such an algorithmic intervention can be useful for the public. The empirical rank diversity could be a combined influence of both the Weibo algorithm and the spontaneous public preference. This observation shows the possibility that rank diversity could be an adequate tool to investigate further the important aspect of algorithmic intervention in social media data.Besides exploring the attention dynamics on the Chinese social media Sina Weibo, we also studied the cumulative growth of all topics and all the COVID-topics on Twitter trending list in the United States. As is shown in the supplementary material Fig. SI1(A), the cumulative number of all the Twitter trending topics in the United States was almost perfectly linear from January 1, 2020 to April 16, 2020. The time period that the cumulative number of all COVID-topics on Twitter trending list increases is in accordance with the rising period of the number of hashtags in the Other Countries category in Fig. Supplementary information (PDF 2.3 MB)Below is the link to the electronic supplementary material.


Recent advances in the development of new methods of cancer immunotherapy require the production of complex cancer animal models that reliably reflect the complexity of the tumor and its microenvironment. Mice are good animals to create tumor models because they are low cost, have a short reproductive cycle, exhibit high tumor growth rates, and can be easily genetically modified. However, the obvious problem of these models is the high failure rate observed in human clinical trials after promising results obtained in mouse models. In order to increase the reliability of the results obtained in mice, the tumor model should reflect the heterogeneity of the tumor, contain components of the tumor microenvironment, in particular immune cells, to which the action of immunotherapeutic drugs are directed. This review discusses the current immunocompetent and immunocompromised mouse models of human tumors that are used to evaluate the effectiveness of immunotherapeutic agents, in particular chimeric antigen receptor (CAR) T-cells and immune checkpoint inhibitors. The tumor microenvironment (TME) consists of a large number of different normal cells that play an important role in the development and progression of the tumor . StromalTo evaluate the effectiveness of immunotherapy, animal models are required in which a human tumor and its microenvironment are genetically, physiologically and anatomically modeled so that the formation and development of the tumor in humans are reliably reflected. Mice are low in cost relative to other animal models, have a short reproductive cycle, exhibit high tumor growth rate, and can be easily genetically modified . These aTransplantation of in vitro cultured tumor cells into immunocompetent mice is the oldest and most commonly utilized approach to investigate antitumor therapy, including immunotherapy A 15]. S. S15]. SUndoubtedly, the formation of transplanted tumors is different from tumors originating de novo. Moreover, the tumor microenvironment will be mostly determined by the local innate immune response triggered by injection-induced inflammation and the presence of a large number of tumor cells ,24. UsuaBesides the fact that TME may prove to be unnaturally homogeneous, the syngeneic model of the tumor also lacks genomic heterogeneity, which makes each tumor unique . The absTo study the effectiveness of CAR T-cell therapy in syngeneic models, mouse cell-derived CAR-T-cells are used . Mouse mSyngeneic tumor models are also used to investigate the antitumor activity of ICIs, including anti-cytotoxic T lymphocyte-associated antigen-4 (CTLA-4)  and antiKRAS and MYC in breast cancer ,6867,68]Rag1 and Rag2, as well as the interleukin (IL2)rγ genes made it possible to create of NOD SCID gamma (NSG) mice with impaired functions of IL2, IL4, IL7, IL9, IL15, and IL21 receptors and the absence of NK cells that allows successful transplantation of the primary tumors and human immune components [The models described above allow the establishment of mouse tumors with a microenvironment consisting of mouse cells, which makes it possible to elaborately study the mechanisms of the tumor formation and their interactions with TME cells. However, to evaluate the effectiveness of immunotherapeutic approaches, models with human tumors that interact with cells of the human immune system are required, since the components of the mouse immune system do not always correspond to those in humans . For thimponents . Howevermponents . For examponents . Adoptivmponents .Transplantation of primary patient tumors allows the creation of patient-derived xenograft models (PDX) that accurately reflect the complexity mediated by the natural development of the tumor, including genomic heterogeneity, tumor architecture and microenvironment factors, especially if the tumors are transplanted orthotopically ,94. PDX + HSCs develop into a complex human immune system without the risk of a graft-versus-host reaction (GVHR), since thymus education of human T-cells occurs in the context of mouse MHC molecules [+ T-cells in the tumors [+ HSCs in the same model can give different results. For example, it was shown that the therapy with pembrolizumab (anti-PD-1 antibody) both suppressed and did not affect the growth of bladder cancer and non-small-cell lung carcinoma (NSCLC) in NSG mice with the immune system reconstructed with HSCs from different donors. However, the authors did not find any connection between the effectiveness of the therapy with the level of mismatch of human leukocyte antigen (HLA) class I and II or with the number PD-1+ leukocytes. In addition, the evaluation of therapy effectiveness can be complicated by the fact that due to the partial mismatch of HLA between the tumor and immune cells, human T-cells can inhibit tumor growth regardless of therapy [+ cells can be used to evaluate the toxicity of CAR T-cells against HSCs. For example, the use of CAR T-cells targeted at the CD44v6 adhesion receptor prevented the engraftment of acute myeloid leukemia and multiple human myeloma in the CD34+ humanized mice. The only side effect was transient monocytopenia, which cleared up after a decrease in the number of CAR T-cells [However, in order to effectively evaluate other types of immunotherapy, in particular ICIs, it is necessary to reconstruct the immune system, ideally from the same patient from whom the tumor was obtained. HSCs can be used to reconstruct the immune system D, both aolecules . In immuolecules . To dateolecules ,102. For therapy . NSG mic T-cells .+, CD8+ and regulatory T-cells (Tregs)), B cells and myeloid cells [The main difficulty PDX models may face in the context of immunotherapy adjustments for patients is the time it takes to produce the in vivo tumor model. For a reasonable time, it is necessary to successfully transplant a tumor into several animals, as well as reconstruct the patient’s hematopoietic system . In ordeid cells . Anotherid cells ,110. Theid cells ,111. In id cells .+ tumors due to the high Treg tumor infiltration associated with the presence of the CD28 domain [In highly immunocompromised mice, individual immune cell populations can also be restored. Functional human NK cells, Tregs and γδT-cells can be restored in immunodeficient animals and used to evaluate the antitumor effects of these cells or their effects on immunotherapy ,113,114.Comparative characteristics of various mouse tumor models are represented in Mice are good models for evaluating the effectiveness of immunotherapeutic agents because these laboratory animals are easy to maintain, relatively cheap, have a short lifespan, and short reproductive cycle. Syngeneic tumor models make it possible to create an in vivo model of a tumor by transplanting mouse tumor cell lines into immunocompetent animals in a short period of time. Such models are used for investigating the interaction of the tumor and the immune system, however, they do not reflect all the difficulties of both the tumor itself and its microenvironment. The rapid kinetics of tumor growth in syngeneic models often provides an inadequate time interval for evaluating the effectiveness of immunotherapy. In addition, to study the effectiveness of CAR T-cell therapy in syngeneic models, CAR T-cells have to be derived from murine cells, and the encouraging results of such studies often fail to repeat in clinical trials. Therefore, they cannot provide adequate models for the study of immunotherapeutic drugs for the treatment of human cancer.Tumor GEMMs provide more opportunities for testing new immunotherapeutic agents due to de novo tumor formation and better TME modeling. The sequential development of the tumor allows the formation of more complex TME and provides a more complex interaction with the immune system, since the tumor passes through all stages of immunosurveillance. GEMMs are very important for investigating the undesirable side effects associated with CAR T-cell therapy, as this tumor model can be modified to express human TAAs, which are expressed not only in tumors, but also at lower levels in healthy tissues. However, since the creation of GEEMs is a very time-consuming and expensive process, and again the evaluation of immunotherapy in mouse tumors with TME represented by mouse cells yields mixed results, the use of these models to evaluate CAR T-cell and ICI immunotherapy is limited.CI mouse models are usually not used to evaluate the effectiveness of CAR T-cell therapy or ICIs due to the fact that in such animals it is difficult to control and evaluate the immune response due to the large number of unknown neoantigens.+ HSCs or PBMCs, which makes it possible to obtain the most accurate PDX models in which the tumor and immune components are obtained from the same donor. The development of this direction looks most promising, since such models can allow immunotherapy adjustments for a particular patient. These models may more accurately reflect a patient’s immune responses compared to current models with partial HLA matching. However, human bone marrow-derived stem cells have a limited ability to develop into immune cells in humanized mice, and PBMCs have limited engraftment possibilities. Therefore, mouse models that support the engraftment and development of human immune cells in humanized mice due to the secretion of cytokines and growth factors that do not cross-react between the mouse and human have been created. Improving these models will allow the formation of the fully functional human immune system in xenograft models and provide a significant tool for assessing the effectiveness of cancer immunotherapy.Transplantation of human tumor cell lines or primary tumors into immunocompromised mice allows the creation of human xenograft models, which are the most used method for the study of CAR T-cell therapy and ICIs. The reconstruction of the human immune system in xenograft models allows investigating aspects of human biology, such as the effect of various cytokines or other populations of human immune cells on the effectiveness of immunotherapy. The human immune system in animals can be reconstructed using CD34Thus, today we have at our disposal mouse models, none of which can reflect all aspects of the complex genetics and biology of human cancer. Understanding the advantages and disadvantages of each model will ensure the most efficient use of the systems available today in order to facilitate the development of immunotherapeutic drugs and enhance continued evaluation of their effectiveness. At the same time, the development of new mouse models reflecting the interaction of the tumor and TME in as much detail as possible will increase the efficiency of translation of encouraging results of immunotherapy from bench to clinic.


To improve the standard point positioning (SPP) accuracy of integrated BDS (BeiDou Navigation Satellite System)/GPS  at the receiver end, a novel approach based on Long Short-Term Memory (LSTM) error correction recurrent neural network is proposed and implemented to reduce the error caused by multiple sources. On the basis of the weighted least square (WLS) method and Kalman filter, the proposed LSTM-based algorithms, named WLS–LSTM and Kalman–LSTM error correction methods, are used to predict the positioning error of the next epoch, and the prediction result is used to correct the next epoch error. Based on the measured data, the results of the weighted least square method, the Kalman filter method and the LSTM error correction method were compared and analyzed. The dynamic test was also conducted, and the experimental results in dynamic scenarios were analyzed. From the experimental results, the three-dimensional point positioning error of Kalman–LSTM error correction method is 1.038 m, while the error of weighted least square method, Kalman filter and WLS–LSTM error correction method are 3.498, 3.406 and 1.782 m, respectively. The positioning error is 3.7399 m and the corrected positioning error is 0.7493 m in a dynamic scene. The results show that the LSTM-based error correction method can improve the standard point positioning accuracy of integrated BDS/GPS significantly. Over the past few years, satellite navigation technology has achieved rapid development, but there are still some problems to be solved in some aspects. For example, in terms of reliability and integrity, a single navigation system is difficult to meet the requirements of positioning accuracy in some areas with complex terrain due to insufficient visible satellites . In addiMany methods can be used to achieve the positioning of integrated BDS/GPS, such as real-time kinematic (RTK), precision point positioning (PPP) and SPP. RTK is a technique that uses the carrier phase in the satellite signal as the observation and uses the difference equation to solve the ambiguity to achieve the dynamic positioning between the mobile station and the reference station . At presThere are many error sources in the positioning process ,14,15. IDifferent methods have been proposed by many researchers to reduce positioning errors. Ke Han et al. proposed a wavelet packet algorithm based on two-dimensional moving weighted average processing (WP-TD) for extracting multipath . HailianRecently, deep learning has become one of the most active technologies in many research areas. Deep learning usually refers to stacking multiple layers of neural network to perform machine learning tasks , which cThe SPP calculation model of Integrated BDS/GPS is implemented based on the BDS/GPS original ephemeris file and observation file data, which are collected through Sinan M300 GNSS receiver.The LSTM-based error correction method is proposed and implemented combined with the traditional filtering method to reduce the multiple sources errors, in which the LSTM recurrent neural network is used to predict the positioning error of the next epoch, so as to reduce the positioning error at the receiving end.Experiments in static and dynamic scenarios were conducted on the data collected by Sinan M300 GNSS receiver and the result of proposed approach was compared with the traditional positioning methods, which proved that the proposed approach can improve the standard point positioning performance of integrated BDS/GPS significantly.In this paper, a novel standard point positioning approach to integrate BDS/GPS , which uses the learning method to predict the multi-source error as a whole, is proposed to reduce the multiple sources errors. The contributions of this paper can be summarized as follows:The rest of the paper is organized as follows. In Both the BDS and GPS time systems use an atomic time base with consistent seconds, both of which are counted in weeks and seconds. The time reference of the BDS is Bei Dou navigation satellite system time (BDT), whose starting epoch is the coordinated universal time (UTC) of 00:00:00 on 1 January 2006. The time reference of the GPS is GPS time (GPST), whose starting epoch is the UTC of 00:00:00 on 6 January 1980. Their different time bases determine the time of each other, but both can be linked to UTC. The conversion relationship between BDT and GPST can be defined as :(1)BDTweBDS adopts China geodetic coordinate system 2000 (CGCS2000), while GPS adopts world geodetic coordinate system 1984 (WGS-84). The coordinate frames of both systems are the International Terrestrial Reference Frame (ITRF) defined by the International Earth Rotation Service (IERS) organization. In terms of coordinate system accuracy, the difference between the CGCS2000 and ITRF is about 5 cm, while the accuracy of the WGS-84 has reached 2 cm, which is basically consistent with the accuracy of ITRF. The study has proved that the coordinate and gravity changes caused by the flatness difference of the two coordinate systems are negligible compared with the current high-precision measurement level. The WGS84 coordinate system and the CGCS2000 coordinate system are compatible at the level of accuracy for standard point positioning.P is the pseudorange measurement from satellite to receiver; c denotes the speed of light; dt means the receiver clock error; dt is the star clock error; T, I and e represent the tropospheric delay, ionospheric delay and other delay error terms, respectively; and G stands for GPS satellites.For GPS satellites, the observation equation for pseudorange is:C stands for BDS satellites and the meaning of the parameters is the same with the GPS satellites.For BDS satellites, the observation equation for pseudorange is:m and the number of BDS satellite signals is n. The mathematical model of the combined positioning is shown in Equation  . Wf, bf The output vector of forget gate layer can be defined as:C is used to denote the vector of new information to be injected into the memory cell, which is the output of the tanh layer. The weights and offsets of the tanh layer are represented by W and b. The tanh activation function can be defined as:The input gate layer controls the information injected into the memory cells by coupling with the forget gate layer to determine the update of the cell state. C can be defined as:The C can be defined as:The vector of the memory cell state at the current moment C after the memory cell is updated. Therefore, the input of the output gate layer is composed of three components, which are the state C of the current memory cell, the output of the memory cell h at the last moment and the input of the memory cell X at the current moment. W and b are used to represent the weight and offset of sigmoid neural network layer of the output gate, respectively. Then, the representation of the output vector of output gate layer O can be described as:The output gate layer has a peephole, and the vector it inputs is the state h can be defined as:The current memory cell output vector The training algorithm of LSTM including the following three steps:Step 1: Forward calculation of the output value of each neuron in the LSTM .Step 2: Reverse calculation of the error term value ne layer .Step 3: Calculate the gradient of each weight according to the corresponding error term .The weighted least square method minimizes the sum of squares of all measurement errors and makes the positioning easy and quick. However, the estimation accuracy of the least square method is not high, and. when the observation shows a large deviation, it also causes a large deviation of the estimation result. The Kalman filter algorithm can overcome the shortcomings of the position at the adjacent time and make the filtered positioning result smoother and more accurate, thereby improving the positioning effect. To effectively reduce common errors and receiver errors, and suppress multi-path effects, the LSTM-based error correction method is proposed.The overall flow of the LSTM-based error correction method is shown in The LSTM-based error correction framework including WLS–LSTM and Kalman–LSTM is shown as The integrated BDS/GPS Point Positioning was performed on a Windows PC with two 3.5 GHz Intel Xeon processors. Compass Receiver Utility software version 1.7.3 was used to operate and obtain the relevant raw data, the SPP calculation model of Integrated BDS/GPS was conducted on MATLAB and the error prediction and correction process was implemented in Python. The experimental data in this study were obtained from 11:31 to 11:59 by the Sinan M300 GNSS receiver shown in To further verify the positioning effect of the proposed method in a dynamic environment, a dynamic experiment was conducted on the campus of a university in Wuhan, China. The moving speed of the vehicle with the experimental equipment ranged from 20 to 40 km/h, and the equipment sampling interval was 1 s. In the experiment, the T30 GNSS receiver data of Sinan Navigation, whose horizontal accuracy is In satellite positioning, the position dilution of precision (PDOP), which is an important indicator for measuring the ability of satellite positioning, is usually used to represent the quality of the satellite’s geometric distribution. Generally, the more visible satellites there are, the better the satellite constellation structure, the better the satellite geometric distribution, the smaller the PDOP and the higher the positioning accuracy. The PDOP and the number of visible satellites during the experimental period are shown in The WLS and Kalman filter were performed on collected data, and the results are shown in The dynamic positioning results are shown in To reduce the common errors, receiver errors and multi-path effects more effectively on the basis of WLS and Kalman filter, the LSTM-based error prediction was performed on the integrated positioning. The prediction result can be used to correct the errors caused by multi sources.The prediction results of LSTM-based error prediction performed on WLS is presented in The error prediction based on LSTM was also performed on dynamic experimental data. The prediction result is presented in The prediction results were used to achieve the error correction. The LSTM error correction performed on the weighted least square method is called WLS–LSTM error correction, while the LSTM error correction performed on Kalman filter is called Kalman–LSTM error correction. The result of WLS–LSTM error correction and Kalman–LSTM error correction are presented in The positioning results of the four different methods are presented in The corrected dynamic positioning results of proposed method are shown in To further evaluate the performance of each method, the cumulative distribution function (CDF) and probability density function (PDF) of positioning errors were used as evaluation indicators. The CDF is used to describe the probability that the random variable X falls within a certain range and PDF is a function that describes the probability of the output value of the random error, near a certain point of determination ,38. The The CDF and PDF results in a dynamic scene are presented in This paper provides a novel approach to the standard point positioning of integrated BDS/GPS. First, the different error sources of standard point positioning of integrated BDS/GPS are elaborated. Among them, some error sources are difficult to estimate during the positioning process. In some current positioning methods, they are also difficult to obtain very effectively reduced or suppressed. Therefore, an LSTM-based error prediction and correction method is proposed in this paper. On the basis of the traditional positioning method, the data with multiple error sources are learned and predicted, and the prediction results are used for error correction to effectively reduce and suppress the error containing multiple error sources.The four methods, namely weighted least square method, Kalman filter, WLS–LSTM and Kalman–LSTM, were tested and compared based on the measured data. It turns out that the LSTM recurrent neural network-based error correction method has greatly improved the accuracy of standard point positioning. The proposed Kalman–LSTM error correction method achieves the best performance, whose point positioning error can almost reach the sub-meter level under the experimental environment. Furthermore, through the error correction method, the positioning error caused by various factors such as the receiver clock deviation can be effectively suppressed. The greater the error is, the more the Kalman–LSTM can contribute to the positioning improvement of the methods without LSTM error correction. In addition, the experimental results in dynamic scenes also show that the proposed LSTM-based error correction method can effectively improve the positioning accuracy.As for future work, parameter tuning of the LSTM network structure can be developed, and the models in different environments and scenarios can be trained to adapt the method to the navigation and positioning requirements in different environments and scenarios.


Lamb waves are used to locate any damage in the stator insulation structure of large generators. However, it is difficult to extract the features of Lamb wave signals in a strong background noise environment, thus significantly reducing the accuracy with which the damage is located. This paper proposes a method based on variational mode decomposition (VMD) and wavelet transform to enhance and extract the location features of stator insulation damage signals of large motors. First, considering that the characteristics of VMD are sensitive to noise, the Lamb wave detection signal is decomposed, denoised, and reconstructed; the reconstructed signal is then wavelet-transformed to extract the time of flight (TOF) of the damage-scattered wave as the damage location feature; finally, the damage location is determined using the TOF features. The proposed method is experimentally tested and verified under various noise environments. The results show that the VMD and wavelet transform methods can significantly improve the signal-to-noise ratio of Lamb wave detection signals and the accuracy with which the damage is located under strong background noise. This study extends the applicability of Lamb wave-based non-destructive detection of stator insulation damage in complex environments. Stator winding is a core component of large generators. The insulation condition of the stator largely determines the life and operating reliability of such generators. The stator insulation structure gradually deteriorates because of the combined effect of electricity, heat, mechanical vibration, and environmental factors during its long-term operation, leading to various types of internal or surface damage and eventually causing insulation failure and fault shutdown ,2,3,4. HStructural health monitoring (SHM)  has beenIn Lamb wave-based stator insulation damage detection, to reduce the impact of environmental noise on the Lamb wave signal, it is necessary to perform noise reduction processing on the received Lamb wave signal. Because of the nonstationary characteristics of Lamb waves, conventional noise reduction methods (such as linear filtering and nonlinear filtering) have some limitations in processing nonstationary signals. For nonstationary signals, wavelet threshold denoising has been applied ; howeverTo locate the damage in a stator insulation structure, it is important to accurately extract the time of flight (TOF) features of the Lamb wave ,16. The Based on the theory of non-destructive testing using Lamb waves, this paper proposes a method for enhancing and extracting the TOF features from stator insulation damage signals of large motors based on VMD and wavelet transform. The method can accurately identify the damage location. The rest of this paper is organized as follows. In the health monitoring of insulation structures of large motor stators based on Lamb waves, accurately locating the stator insulation damage can provide important reference information for insulation state diagnosis and life assessment . HoweverIn D, the Lamb wave will scatter at the damage position based on its propagation characteristics. Under this condition, the received wave signal As shown in According to Equations (1) and (2), the damage-scattered signal The damage location From Equations (4) and (5), we can infer that the stator insulation damage can be located as long as the TOF feature is accurately extracted when the distance from the excitation point to the receiving point and the speed of the scattered wave are known.The actual running state of the stator windings is complex and may be subject to severe noise interference. Noise reduction methods can improve the SNR of the received signal, and the TOF location features can thus be more accurately extracted. Due to the nonstationary characteristics of Lamb wave signals, conventional noise reduction methods have some limitations in processing Lamb wave signals. The VMD method can effectively deal with nonlinear and nonstationary signals and is therefore suitable for analyzing Lamb wave damage signals. This method can be used to adaptively analyze the damage signal and decompose it into different sub-components, so that the useful signals can be effectively separated from the background noise.The main idea of the VMD method for blind source separation is to divide the original signal into different sub-components called band-limited intrinsic mode functions (BLIMFs), which can be expressed as:The BLIMF in Equation (6) is typically required to be a narrow-band signal. In this study, the Lamb wave signal is a narrow-band signal with a fixed center frequency. The actual received Lamb wave damage signal can be expressed as follows:To extract the real damage signal K is the number of BLIMF sub-components.Equation (8) is a VMD decomposition expression. For one K BLIMF sub-components. K represents the number of different sub-components that the received Lamb wave damage signal is decomposed into. Generally, the larger the K is, the richer the damage signal types that the sub-components can represent. However, if K is too large, the decomposition of useful signal representing the real damage signal of Lamb wave will be over decomposed, which is not conducive to blind source separation. Generally, the more types of noise, the higher the value of K.According to Equation (8), the received Lamb wave damage signal The following are the specific steps involved in the adaptive decomposition process through the VMD method:K is set in advance, and by solving Equation (9), each BLIMF gets a limited bandwidth around its center frequency, and the total bandwidth is the lowest, so as to achieve the best separation effect.i. The number ured. In , α was sii. The alternating direction method of multipliers (ADMM) algorithm can automatically find the saddle point of Equation (9); thus, the VMD method can adaptively decompose the received Lamb wave damage signal ils, see .Through the above algorithm, the real damage signal As mentioned previously, the extraction of the TOF location features is key to locating the damage in the stator insulation structure. Because this structure is made of a composite material, and Lamb waves have multi-mode and dispersion characteristics, Lamb waves are prone to wave packet deformation and aliasing during propagation, which seriously affect the extraction accuracy of TOF location features.Wavelet transform has multi-resolution characteristics. It can focus on the local features of the signal adaptively, which can help accurately extract the TOF features and improve the accuracy with which the damage is located. According to Equation (3), the damage-scattered signal The wavelet basis function in Equation (10) serves as an observation window in wavelet transform. It helps analyze the damage details such as the trend, discontinuity, and peak value of the lamb signal to be measured. Hence, the selection of the wavelet basis function is important. The complex Morlet wavelet has advantages such as the lack of endpoint effect and small amount of calculation . In addiThe energy spectrum is plotted after performing continuous wavelet transform on the Lamb wave signal using Equation (10), so that the damage characteristics of the Lamb signal can be analyzed more intuitively. The energy distribution equation for the Lamb wave damage-scattered signal in the time-scale domain is expressed as:With Equation (11), the energy spectra of the Lamb wave excitation signal To verify the feasibility and effectiveness of the feature enhancement and extraction method based on VMD and wavelet transform, an experimental system was set up, as shown in In the experimental system, the sensor network composed of PZT . H(t), i.e., the Lamb wave signal received in the non-damaged state. D(t) received in the damaged state. In this study, the effects of stator insulation damage location under different noise environments were tested. Second, the VMD method is used to further decompose the Lamb wave signal. As an example, the BLIMF sub-components are obtained after decomposing the Lamb wave damage signal with mixed noise see d, as shoThe correlation coefficient between different sub-components and the original signal calculated according to the cross-correlation function are listed in The wavelet transform is applied to further extract the damage location features. According to Equation (5), the identified location To effectively evaluate the noise reduction effect of the VMD method based on the wavelet transform feature extraction method, the effects of stator insulation damage location before and after noise reduction are compared under different noise environments. The statistics of the damage location results under different noise environments are listed in As listed in As mentioned previously, the extraction of the TOF features directly influences the accuracy with which the damage is located. In addition to noise, which affects the accuracy of TOF extraction, the damage feature extraction methods influence the accuracy of TOF extraction. To effectively evaluate the effect of TOF feature extraction and damage location based on wavelet transform, the damage location effect of the proposed method is compared with that of the Hilbert-based  TOF featAs listed in Surface cracks are another type of typical damage in stator insulation structures. If they can be detected, early diagnosis and prediction can be carried out, and timely response measures can be taken, which has high practical value. As with specimen A, a comparison test was performed on the surface crack damage location in specimen B under different noise environments. The results of the damage location test are listed in From The actual working environment of large motor stator windings is relatively complicated, and it is susceptible to severe noise interference. Moreover, Lamb waves have multi-mode and dispersion characteristics when propagating in the stator insulation composite material, making it difficult to locate the damage. Improving the SNR of the received signal and accurately extracting the TOF location features are key to performing accurate detection. This paper proposes a method for enhancing and extracting the damage localization features based on VMD and wavelet transform. Verification experiments are performed with two types of typical damages: puncture damage and surface crack damage.The experimental results show that the VMD method can effectively improve the SNR of the received signal and overcome the problems due to the relatively weak useful signal and difficulty in extracting it under strong background noise, thereby extended the application range of the detection method employed for stator insulation damage. The wavelet transform method can effectively overcome the influence of the Lamb wave dispersion effect, accurately extract the TOF location features, and further improve the location accuracy of stator insulation damage. The proposed method based on VMD and wavelet transform can detect the damage that occurs during the stator insulation aging process and locate the damage location. It can be used as an effective condition monitoring tool for large generators in actual operating environment to supplement existing methods. The proposed approach significantly extends the applicability of non-destructive detection using Lamb waves. In future work, we will research how to combine the proposed method with non-invasive detection methods, which is more conducive to practical industrial applications.


Arthrospira platensis) and brown (Rhodosorus marinus) microalgae species determines that the selected strain must be considered in the development of this system.Optical Camera Communication (OCC) systems have a potential application in microalgae production plants. In this work, a proof-of-concept prototype consisting of an artificial lighting photobioreactor is proposed. This reactor optimises the culture’s photosynthetic efficiency while transmitting on-off keying signals to a rolling-shutter camera. Upon reception, both signal decoding and biomass concentration sensing are performed simultaneously using image processing techniques. Moreover, the communication channel’s theoretical modelling, the data rate system’s performance, and the plant distribution requirements and restrictions for a production-scale facility are detailed. A case study is conducted to classify three different node arrangements in a real facility, considering node visibility, channel capacity, and space exploitation. Finally, several experiments comprising radiance evaluation and Signal-to-Noise Ratio (SNR) computation are performed at different angles of view in both indoor and outdoor environments. It is observed that the Lambertian-like emission patterns are affected by increasing concentrations, reducing the effective emission angles. Furthermore, significant differences in the SNR, up to 20 dB, perceived along the illuminated surface (centre versus border), gradually reduce as light is affected by greater dispersion. The experimental analysis in terms of scattering and selective wavelength attenuation for green ( Microalgae culture has gained significant momentum during the last decade, triggered by the necessity of developing new and sustainable resources. They have become a promising alternative source for biofuels and biogas production, human and animal nutrition, cosmetics and bioactive supply for nutraceutical and pharmaceutical applications.Microalgae biomass production is currently carried out using both open ponds  and closed photobioreactors . The lasGenerally, microalgae production plants are designed to take advantage of the Sun as the primary light source due to cost optimisation. However, the Sun’s irradiance depends on several factors, such as weather conditions, latitude and day time. Furthermore, these open-air plants need vast extensions to be profitable since effective biomass production depends on the directly-exposed surface. Nonetheless, artificial lighting can provide advantages in photosynthetic efficiency (custom spectrum and intensity profiles) and the tight control they offer concerning microalgal biochemistry and growth, increasing industrial processes’ reliability. Using this type of lighting, biomass production depends no longer on the plant’s exposed surface, but on its equivalent volume. Although optimising light quality could considerably reduce energy consumption, another approach concerns how light is delivered to the culture: flashing (pulsed) light instead of continuous illumination.In this context, pulsed illumination could serve as a communication link, enabling the re-utilisation of light sources as effective visible light communication (VLC) transmitters . The lowNevertheless, there are still a few parameters that are either too complex to be measured in situ or imply invasive methods. These parameters are microalgae biomass concentration and its growth phase. In ,5, diffeOn the other hand, the use of cameras as communication receivers for VLC links is a research topic that is receiving significant attention. Furthermore, there is a current standard specification by the working group IEEE 802.15.7 , which hThis work proposes the use of low-cost cameras for both remotely sensing microalgae culture parameters and for establishing a direct communication link with flat-panel photobioreactors. Compared to other radio or wired technologies, the use of OCC has the following advantages. First of all, this technology reuses the light that comes from the photobioreactors and makes better use of this excess of energy that would otherwise be wasted. Second, it replaces a generic communications coordinator with an intelligent camera that performs additional routines apart from establishing a communication link. On the one hand, it carries out continuous surveillance of the reactors and the personnel who access the room and can act as an early warning system. On the other hand, it simplifies the estimation of some culture parameters that are difficult to measure or involve intrusive procedures, such as the concentration of biomass or the strain’s growth state. For example, for evaluating the biomass concentration, high-cost equipment must be used to automate the extraction of representative samples. In the worst case, this procedure must be carried out periodically by the staff. Furthermore, concerning communications security, an extra layer of protection is added thanks to the light confinement within the room. Finally, cameras’ inherent spatial multiplexing capabilities significantly simplify link protocols, making communications more robust and less prone to errors.This work presents part of the ATICCuA Project results, a multidisciplinary research project carried out by the Spanish Bank of Algae (BEA by its Spanish acronym ) and the Photonics and Communications Division of the Institute for Technology Development and Innovation in Communications (IDeTIC). This project addresses the development of prototypes based on the visible light communication (VLC) and, more specifically, those used in underwater wireless optical communications (UWOC) for application in microalgae culture systems. Precisely, its main objective consists of the design of an LED-based dual-use system, which provides configurable lighting for the culture and production of microalgae and cyanobacteria and optical wireless communication capabilities for optical underwater channel characterisation.This work introduces and discusses the microalgae cultivation restrictions that affect the communications’ performance and the overall system’s design. In addition, a comprehensive analysis based on geometrical constraints is carried out, providing results such as optimal camera positioning concerning a custom-defined metric that relates the aggregate data rate and effective space exploitation. From this analysis, a preliminary study of the plant distribution of a case study is carried out. Furthermore, several experiments that evaluate the system’s communications performance are conducted in an indoor and outdoor scenario.The remainder of the paper is structured as follows. The proposed architecture follows a many-to-one unidirectional network topology where photobioreactor nodes transmit sensor-related data to a receiver camera node .The photobioreactor nodes comprise a uniform-radiance LED flat panel attached to a glass container that holds the microalgae culture. The receiver node consists of a rolling-shutter (RS) camera connected to the communication endpoint that exposes a control interface and a standard wired communication bus. The transmitter nodes use two different signals to establish a link: a well-known beacon signal that uniquely identifies the node and data packets composed of Manchester-encoded pulses. The receiver uses the beacon signal to perform three separate routines: establishing the communication link, estimating the channel response to complete the decoder training, and finally, sensing the culture by estimating the biomass concentration and growth phase of the microalgae species. Therefore, this system performs two functions. It establishes a monitoring link and directly senses essential culture parameters.In the following sections, both nodes are presented, highlighting the influence of the microalgae cultivation needs and requirements in their design.The transmitter is comprised two parts: the LED flat panel and the LED driver that generates the corresponding OCC driving signal to transmit sensor data provided by probes deployed inside the container . This section is focused on light generation and transmission.As was previously mentioned, the competitiveness of any artificial light-driven microalgal production requires improvements in photon harvesting and the conversion efficiency of light sources . Hence, On the other hand, recent experimental studies have shown that combining short and intense light flashes with extended dark periods instead of continuous illumination might increase the culture’s growth efficiency, as discussed in ,18. The Therefore, the transmitting source’s design must consider all these restrictions: light quality  and light delivery (frequency and duty cycles) concerning the selected strain and the expected results.N is an integer that relates the exposure time to the sampling time  cameras, the whole image sensor is exposed simultaneously. Thus, the achievable data rate is upper-bounded by their frame rate, restraining the light source’s switching frequency considerably. On the other hand, RS cameras scan the scene sequentially row by row of pixels , allowing capturing different light states  within the lamp’s source projection in the frame . In thisEquation , which fEquation ).(1)H|H. In previous works, it was shown that the reduced received power due to the extinction along the path or due to low exposure times can be overcome using the analogue amplifier of the CMOS camera by increasing the gain it provides before the analogue-to-digital converter (ADC), ultimately improving the SNR ,21.The proposed architecture presents some specific characteristics from the communication system point of view. First, the light propagates through different media from the emitter to the camera, which modifies the received optical signal. On the other hand, the camera position affects the communications’ performance since it determines the received power and the emitter image size. Furthermore, the distribution and size of the photobioreactors also impact the complete OCC system performance. In this section, all these aspects are addressed.The channel can be divided into different layers. This preliminary analysis shows two critical interfaces: the inner glass/water and the outer glass/air interfaces that affect communications’ performance and constrain harvesting optimisation.Light rays emitted in a specific direction by the light source ; in othAfter characterising the radiance of the surface, the total irradiance over a pixel is obtained by integrating the incoming radiance from any direction in the normal hemisphere that encloses the pixel area Equation .As an example, In the stated conditions, the radiance would be affected by different phenomena: the culture’s absorption capacity, the shading effect between cells, and light scattering. When the amount of biomass is negligible compared to the volume of water in the container, these phenomena might be neglected, and radiance can be estimated using geometrical optics. Now, considering that the channel consists of symmetrically repeating parallel layers that start and end in the same air medium, light’s incident angle x,y, of the output point ReffEquation ) and Rp Equation ) are theEquation ). This eEquation  over theEquation .(9)dIλ , which rEquation ).(12)σλThis RTE equation given by Equation  reveals Regarding the communications’ performance, although the presence of these microorganisms attenuates the signal before it reaches the exposed container’s surface, it may be beneficial in some cases because it distributes the optical power more evenly, uniforming the radiation pattern over the entire container’s surface. In conclusion, microalgae particles’ scattering phenomena can be exploited for communications in scenarios where the camera is not facing perfectly perpendicular to the transmitter surface.The maximum achievable data rate for a photobioreactor depends on its projection’s vertical size over the image; in other words, the total number of vertical samples (rows of pixels) recovered from the signal . ConsequEquations  and 14)14) relatFollowing Nyquist’s criterion (Equation .(15)hs=It must be remarked that communication takes place in a windowed manner. Leaving aside the reflections with the objects present in the scene ,28, the ous work . With thFinally, it is worth mentioning that the proposed system’s achievable capacity considerably exceeds the requirements for accurate monitoring of the culture parameters. To obtain a preliminary idea of these requirements, the following considerations are taken into account. The selected parameters to be measured are acidity, temperature, light conditions, carbon, nutrients, inhibitors presence and OThe distribution of the photobioreactor nodes across the room plays an essential role in communications. To establish a link, the receiver should visualise each node, and consequently, it is necessary to reserve some space free of obstructive interference between the transmitters and the camera. This reveals the importance of analysing different possible solutions for placing the nodes until finding the one that best suits the project’s initial requirements, either in terms of better link quality, higher capacity or more efficient space exploitation.In a previous work , a metriN, a communications performance metric, Q, and the space utilisation ratio (SUR). The first two variables account for the overall achievable data rate of the setup. The last term relates the total equivalent volume of the containers bound to biomass harvesting, It consists of three independent variables: the number of simultaneous photobioreactors monitored by a single camera, Q metric is derived from the well-known Shannon–Hartley’s equation [C is the capacity in bits per second, The proposed equation  for estiEquation ). HoweveAdding this factor to Equation , the effQ is then defined as the relationship between the channel’s capacity for a particular arrangement and container, C, and the ideal capacity, The proposed metric Equation .(23)Q=CThe vast number of parameters involved in this metric’s computation makes this metric unreasonable for a practical analysis of different plant distributions, even more so if there are plenty of configurations to be compared. Hence, for this metric to be a useful tool for this purpose, it is necessary to assume a series of coarse approximations. These approximations would greatly simplify the practical comparison of two cases without incurring harsh penalties. First, it is important to remark that the camera’s hardware and configuration would not change. Consequently, the bandwidth, which depends on the sampling frequency and the image sensor’s exposure time, remains constant (Equation  could beEquation .(24)Q=lNow, using the first order Taylor approximation of the logarithmic function (at x = 0), Equation , resultsIn this way, the measurement of the capacity of a placement can be approximated to the comparison of the SNR with the best case and the size of its projection in the image. However, assumptions regarding the relationship between the SNR under study and the best-case SNR can simplify the analysis. These assumptions are detailed below. First, the container radiance varies smoothly through the entire surface, or at least in the area were the signal is recovered. Not the whole surface of the container is utilised for signal detection, but a fraction of it. The radiance at any given direction is expressed with respect to the maximum radiance Equation ).(27)L where the signal will be recovered is constant in width and height, then the total received power would be the aggregation of all contributions from the ROI’s pixels. Therefore, the optical power received within the ROI is approximated by Equation .(29)SroRegarding the noise power, no external interfering optical sources were considered for simplicity. The primary noise sources are thermal noise  is obtaAs mentioned, this metric is intended to be easy to compute and practical when guiding the engineer to distribute the plant most optimally. The approximations stated above greatly simplify the comparison of two cases without incurring non-affordable penalties. This Q parameter allows indirectly approximating the maximum transmission rate achievable by each photobioreactor node (Equation ).In this work, the minimum value of Q  in a 300 × 200 × 300 cm empty room. These containers must be placed in individual racks capable of holding up to 4 containers, two at the top and two at the bottom.In terms of communication performance, those cases must satisfy the following derived constraints. The top photobioreactors will fill the upper half of the image and the lower ones the lower half. Consequently, the camera should always point to the shelf’s vertical centre, and it will be aligned with respect to the centre . In this way, the transfer rate is ensured to be equal for each reactor, with the number of vertical pixels equal to half of the image’s vertical resolution (Equation ). The miTo properly address this analysis, it is necessary to select a camera as an example. The camera selected is the PiCamera v2, detailed in Taking into account all these starting requirements, three different cases are proposed. Those cases are shown in In the remainder of this section, the procedure for calculating the metrics for each case is detailed. In all the cases, the restrictions mentioned above were used to compute the camera’s location by optimising the SUR quantity.For clarification, In this case, to attain the initial restrictions  an an34) anEquation ).The other restriction is that the camera’s vertical FoV, Equation ). OtherwTo resolve the stated equations, Using Equations  and 38)38), the The translation of the camera to the side has the advantage that its distance is considerably reduced. However, as can be seen in Equation .(39)FoVIn this case, utilizing the maximum posed in ; thus, tWe highlight that, in this particular case, the camera does not visualise the whole surface of the nearest containers, but merely a region of their base  and wouquations  and 41)AoVH is tThe location of the camera Equation ).Two experiments were conducted at the BEA facilities to evaluate signal reception and deterioration due to the increase in biomass concentration. The first experiment evaluated the container surface’s radiance in a controlled indoor environment, whilst in the second experiment, the system’s performance in an outdoor scenario was assessed. The shared materials, resources and equipment are summarised in The following subsections describe each scenario separately to facilitate understanding the motivation, methods and data analysis for each experiment.BEA 0007B Arthrospira platensis (A. platensis).In this experiment, the radiance emitted by specific regions from the container’s surface is evaluated. The aim is to determine how the radiance profile changes as the biomass concentration increases for different camera viewing angles. At the same time, the SNR of the channel is measured, and the bit error rate (BER) is estimated. The selected species for this experiment was −1, starting from 0 mg l−1 and reaching 100 mg l−1.Regarding the evaluation methods, the experiment was based on acquiring nine images at different viewpoints with respect to the azimuthal angle After capturing the image samples, they were processed offline to obtain the radiance and SNR, from two different regions, the centre and the boundaries of the container’s surface, to analyse the skewness phenomenon in the outermost regions of the surface.The radiance was measured indirectly by averaging the pixel values in a tiny window no higher than 50 × 50 pixels. For the area projected in this window and considering the distance and the size of the container, it can be assumed that the radiance pattern varies very smoothly within this region. The original horizontal and vertical window’s length was established at the start of the experiment when In addition, the SNR was computed within the same window by measuring the mean, Equation , taking Equation  using th(erfc(·) .(42)SNRThe key parameters of this experiment are summarised in BEA 1286B Rhodosorus marinus (R. marinus)  to evaluate the communication restrictions that imply the use of microorganisms with different absorption curves. The beacon signal is comprised five sequential pulses  followed by a dark guard. The key parameters of this experiment are summarised in This experiment aimed to evaluate how the presence of external light sources interferes with the signal received by the camera, considering both the light that is reflected on the container’s transmission surface and the light that enters through the sides of the container. In this experiment, the camera node faced just one photobioreactor, which repeatedly transmitted a beacon packet through different biomass concentrations (from low to high). The selected microalgae species was F.The first configuration, Case I, has a relatively low SUR. Approximately only four percent of the available space is utilised for harvesting. Therefore, this proposal should be discarded, even though the conditions for establishing a communication link are practically ideal. This SUR could be increased if a greater In Case III, however, just the opposite happens. The SUR is almost four times higher than in Case I. However, the optical received power from the most distant containers is so low that it reduces the effective data rate and increases the complexity at reception. This reveals the importance of the camera viewing view. Those acute view angles will significantly affect the total received optical power. They will make it difficult not only to establish the link, but also to estimate the microalgae culture’s biomass precisely.N double their original values, maintaining the communication performance.In conclusion, Case II becomes the optimal choice for this application. It is the solution that achieves a balance in all parameters and has greater replicability. This replicability allows easily creating new plant distributions, such as the one shown in However, despite this being the chosen solution, there is still a discussion to be made. The fact that the camera scans only a fraction of the first containers and performs the full scan of those that are further away (due to the image’s viewing perspective) gives rise to new challenges. For example, the signal-to-interference-plus-noise ratio (SINR) is different for each container, especially under the presence of tiny air bubbles in motion within the reactor, which are utilised generally to aerate the organisms and induce a continuous movement. The projection of these small air bubbles increases in size for the nearest containers, generating more noticeable interference phenomena. Furthermore, in terms of biomass sensing, the scanning of just a fraction of the culture can provide reasonable estimations only if the biomass is correctly distributed within the recipient.On the other hand, analysing the SUR values, it is observed that they are relatively small. The maximum aggregated container’s volume is approximately 13% of the entire required space. However, it is important to notice that, in all vertical cultivation plants, it is necessary to reserve some room for the technicians to execute periodic control routines, handle the extraction of samples with ease and react quickly to warning alarms. Conventionally, it is recommended to reserve up to one meter of separation between racks. This separation coincides with the optimal camera distance in Case II.A. platensis’s SPF and its absorption and scattering cross-section coefficients. A. platensis is a planktonic filamentous cyanobacterium. Its cylindrical morphology gives the cell a highly directive SPF [In the container’s central region, the radiation pattern follows a Lambertian with n = 1 for red and green channels when no microorganisms are suspended in the water. In the blue channel, it deviates minimally for wider angles. It can be seen that as the biomass increases, the emission becomes more directive. Actually, in the case of the red and green channel, at the point of maximum concentration, it follows a perfect Lambertian pattern with n = 2. The reason behind this has a connection with tive SPF   at a certain angle and to understand the reason behind selecting these two regions for the analysis, A. platensis and the internal shape of the container.The location of this interface on the surface depends on the container’s geometric construction and moves with the point of view. In thicker containers, this interface gets closer to the centre, and consequently, the border region increases significantly in size. On the other hand, as the container is viewed from a tighter angle, this interface moves towards the edges c. In thiRegarding the SNR, A. platensis scattering slowly contributes to distributing the optical signal more evenly across the surface. Besides, the difference in power also gradually decreases, at least in the red and green channels. Therefore, slow attenuation combined with increasing scattering is beneficial, mainly because it reduces the gap between the two regions, increasing the effective signal reception area. Finally, regarding the sensing of the biomass concentration and culture’s growth state, it is concluded that it slightly depends on the viewing angle. The relative differences in attenuation per angle for each channel as a function of concentration are non-linear. As the concentration increases, the radiant intensity diagram varies slightly in shape, relatively different for each channel. Therefore, to estimate the biomass correctly using only the pixel values inside a given area, it is necessary to consider the viewpoint. However, this radiation’s behaviour provides valuable information that can be favourably exploited for more accurate crop parameters’ estimation. The larger the variations, the better the estimation will be. In this sense, Case II, which proved to be the ideal candidate for a real deployment, has the added advantage that it analyses the strain from different angles. If all containers are interconnected through pipes and the strain is transferred from container to container periodically, its sensing will be more accurate.The most evident result extracted from these graphs is that the SNR in the blue channel decreases much faster than in the other channels. Furthermore, the SNR starts to decay at 40 degrees at the edges, approximately 20 degrees earlier than in the red and green channels. This was discussed previously in terms of the light intensity emitted by the surface. Nevertheless, another result can be extracted. The differences between the central and outer SNRs are reduced progressively as the biomass concentration increases. The results are presented in It can be observed that this species produces significant attenuation in the blue and green portions of the spectrum (attaining the Bayer filter’s spectrum response of the camera). Despite that the green, blue and red pulses cannot be distinguished in Finally, analysing the reference signal d, it is In this work, OCC was proposed as a suitable communication technology for monitoring microalgae production plants based on artificial lighting. The theoretical channel model was introduced alongside the analysis of the parameters that significantly impact the achievable data rate, such as the nodes’ geometrical configuration and the camera sampling frequency and exposure time.This research highlighted the importance of optimising plant distribution in terms of link quality, channel capacity and efficient space exploitation. A multidimensional metric was designed for this purpose and tested in the conducted case study, where three different node arrangements were classified. This study’s main result was that the configuration in which the camera observes the reactor shelf from the side is the preferred solution. On the one hand, it provides comparable link qualities between all containers without incurring detrimental losses of the signal strength due to the camera’s viewing angle. On the other hand, it better exploits the scarce space available than a camera watching the containers frontally.A. platensis and R. marinus . In the indoor experiment, the container surface’s radiation pattern was measured for each image channel at different concentrations. This analysis reveals significant differences in the SNR between channels due to the algae absorption spectrum. Moreover, notable differences can be observed between the borders and the central part of the surface. These differences tend to reduce as the concentration of microalgae increases due to the phenomenon of scattering. Despite that the viewpoint must be taken into account when designing the system, it additionally provides valuable information for in situ sensing the algae’s biomass and its growth state. In the outdoor experiment, the blue and green channels were highly attenuated, which compelled discarding these wavelengths for data transmission. Furthermore, in this scenario, the sunlight decreased the available dynamic range considerably for signal transmission, especially for low biomass concentration. Finally, examining the obtained results for the two microalgae species with different absorption profiles determined that the selected strain and its temporal evolution must be considered in the development of the optical link.In addition, the experimental evaluation of the proposed flat-panel photobioreactor prototype was carried out in indoor and outdoor environments for two different microalgae species: Future research should further develop and confirm these initial findings by extending the SNR’s evaluation to the actual achievable BER for this deployment in both indoor and outdoor scenarios. It should examine more deeply how the presence of aeration bubbles might affect the signal reception. It could also focus on comparing and developing novel framing strategies for sensor data, reducing packet overhead, easing transmitter discovery and data synchronisation and equalisation. In terms of culture sensing, artificial intelligence for biomass estimation may constitute the object of future studies.In conclusion, it was demonstrated that microalgae production plants are a potential use case for OCC. This technology provides simultaneous node monitoring capabilities using a cost-effective deployment, paving the way to develop smart farming strategies within the microalgae cultivation field.


Consumer-grade RGBD sensors that provide both colour and depth information have many potential applications, such as robotics control, localization, and mapping, due to their low cost and simple operation. However, the depth measurement provided by consumer-grade RGBD sensors is still inadequate for many high-precision applications, such as rich 3D reconstruction, accurate object recognition and precise localization, due to the fact that the systematic errors of RGB sensors increase exponentially with the ranging distance. Most existing calibration models for depth measurement must be carried out with different distances. In this paper, we reveal the mechanism of how an infrared (IR) camera and IR projector contribute to the overall non-centrosymmetric distortion of a structured light pattern-based RGBD sensor. Then, a new two-step calibration method for RGBD sensors based on the disparity measurement is proposed, which is range-independent and has full frame coverage. Three independent calibration models are used for the calibration for the three main components of the RGBD sensor errors: the infrared camera distortion, the infrared projection distortion, and the infrared cone-caused bias. Experiments show the proposed calibration method can provide precise calibration results in full-range and full-frame coverage of depth measurement. The offset in the edge area of long-range depth (8 m) is reduced from 86 cm to 30 cm, and the relative error is reduced from 11% to 3% of the range distance. Overall, at far range the proposed calibration method can improve the depth accuracy by 70% in the central region of depth frame and 65% in the edge region. The consumer-grade RGBD sensor has the potential to be used in many fields, such as 3D reconstruction ,2, camerThe SLP RGBD sensor calculates ranging information based on the disparity For the object m, n are the normalized parameters that remain unknown: For commercial RGBD sensors, such as Structure Sensor and Kinect V1, the actual disparity Combine (1) and (2): Early works on SLP RGBD calibration ,15 descrA visual camera or a RGB camera often have a centrosymmetric or near centrosymmetric distortion pattern. When the distortion model is adopted in RGBD sensor calibration, the assumption that the distortion of SLP RGBD is also centrosymmetric and can be modeled by a classic distortion model is accepted by many researchers ,20,21,22This paper firstly makes a comprehensive discussion of the sources and forms a mechanism of depth distortion of the SLP RGBD sensors. By compositing the calibration model that is strictly based on actual physical model of camera distortion and projector distortion, we propose a new method which can effectively and accurately calibrate the full range depth data. The proposed method calibrates the whole range depth with just one model and one set of parameters. Compared to the existing model that requires to separately fit calibration parameters for different distance range or a look up table of parameters, the proposed method largely improved the efficiency of the calibration procedure. By applying the proposed calibration model, we can accurately improve the ranging accuracy by 70% in the central area and 65% in the edge area of the depth frame. Two distortions and one infrared cone related bias will be used to model the overall systematic error of SLP sensors. The two distortions are a pincushion distortion caused by the projector lens and a barrel distortion caused by infrared camera lens, shown in Then a two-step calibration procedure is proposed to overcome the problem that the commuter-grade RGBD sensor often remains a black box to the users. SLP RGBD sensors such as Kinect V1  and StruThe calibration method is evaluated by two experiments in an indoor environment. One is an experiment designed to test the performance of the calibration model. A set of test data with ground truth error is acquired in three different scenes: (1) flat surface, (2) non-flat surface, and (3) flat surface with distance variation. The ground truth error is obtained by plane fitting and compared with the modeled error generated from the proposed method. Comparison results demonstrate that the proposed method can precisely model the systematic error caused by non-centrosymmetric distortion of the RGBD sensor. The other one is an evaluation of proposed method in actual point cloud collected in the indoor environment. By comparing the calibrated and uncalibrated point cloud in the whole working range of the tested SLP sensor, the improvement of the calibration model can be demonstrated clearly. To calibrate the SLP based RGBD sensor, it is important to understand how the error is introduced in the RGBD sensor and how that error affects the depth measurements. The radical systematic error appeared in the depth frame is mainly a combined result of the distortions caused by camera lens and projector lens. The infrared cone also causes a certain radical error to the depth frame. In this section, the mechanism of the non-centrosymmetric distortion of RGBD sensor will be revealed. The distortion appearing in the disparity measurement of SLP RGBD sensor can be traced back to the two optical lenses in its hardware, the projector lens and camera lens. The projector lens causes a pincushion distortion as shown in The data accessed by users are the normalized disparity unit, which is a normalized pixel difference between the pre-stored reference pattern and the captured pattern. The most basic observation of SLP RGBD sensor is the disparity The pincushion distortion occurred when the pre-stored pattern was casting through the projector lens into the object scene. As shown in Considering both camera and projector distortions, the measured disparity in pixel can be described as:According to Zhang , the optCamera distortion is normally barrel distortion with a negative term for Both the camera-caused barrel distortion and projector-caused pincushion distortion are uncorrelated to the distance. The camera-caused barrel distortion in the disparity is a fixed centrosymmetric distortion with a centre at the optical centre of the IR image plane for the whole distance. For the pincushion distortion of the projector lens, it can be proved that the casted projector distortion’s effect in the image plane is uncorrelated with distance, and same as the barrel distortion caused by the camera lens, the pincushion distortion on the image plane should be a fixed distortion for all distances. It is because the ratio between the original pincushion distortion in projector plane and the captured pincushion distortion in image plane is a constant value for all the distances. Where,As shown in Combine the two parts in Equation (6), the following relationship can be obtained: Since Both camera barrel distortion and projector pincushion distortion are centrosymmetric distortion in the normalized disparity unit. The camera barrel distortion is constant at the optical centre of the IR image plane for all the distance range. For the pincushion distortion appeared in the image plane, although it is still a centrosymmetric distortion, it has varying distortion centre Although both the barrel distortion and pincushion are centrosymmetric distortion in the disparity unit, the combined overall distortion is a non-centrosymmetric distortion due to the pincushion distortion’s varying centre. If the centrosymmetric barrel distortion and pincushion distortion have the same distortion centre in the image plane, as shown in  pattern d, its ovNormalize the disparity in Equation (4) using Equation (2):Based on the Brown’s model , δcep anThe distortion model can be expressed as: In this way, Equation (10) can be further expanded as:Combine (9), (10) & (12), the proposed calibration model in normalized disparity can be expressed as: Therefore, for the SLP-based RGBD sensor, the distortion error in disparity consists of two parts, one is a barrel distortion caused by camera lens with fixed location at optical centre. The other is pincushion distortion caused by the projector lens with a varying location over the camera image plane. In practice, since the actual distance In the SLP sensor, both camera distortion and projector distortion occur. In the image plane, the camera distortion is fixed as its optical centre, and the projector lens distortion has a varying optical centre in the image plane according to the distance from projector to the object. Therefore, if the distortion model for the camera and projectors is known, the combined distortion can be calculated proximately by the measured distance. The distortion calibration for infrared camera can be conducted in the traditional ways. But for the raw observation, the structured pattern of infrared projector of RGBD sensor is inaccessible, and there is no way to determine the projector distortion model by using the existing method  that reqA two-step calibration procedure is designed to specifically calibrate the SLP-based RGBD sensors. The RGBD sensor used in this paper is the Structure Sensor from Occipital , shown iIn order to calibrate the SLP RGBD sensor, a calibration field is set in an open space. Establish a control plane by placing the control markers with known coordinates measured by total stations.Fit the plane equation of the control plane.Place the Structure Sensor in front of the control wall and take multiple observations (100 observations) at each designed distance , as shown in Calculate the pixel-wise average depth frame for each distance to reduce the effect of random noise in each pixel.Use the perspective-n-point algorithm  to calcuUnify the coordinates system between Structure Sensor and the control wall based on the calculated camera extrinsic parameters in 5.Calculate the ground truth depth for each distance based on the fitted control plane equation.Calculate the ground truth disparity based on the processed ground truth depth.Use the ground truth disparity to calculate the disparity error of the averaged disparity measurement in each distance.The data acquisition and processing procedure is as follows: The disparity error obtained after the data processing is used for model parameters estimation. The plane-fitted method is used to fit the equation of the control plane to provide adequate data for the parameter estimation. With the fitted plane In the reference system of the fitted plane, the projection location Then the ground truth depth Therefore, based on Equation (9), the disparity error The disparity errors obtained at different distances in the data acquisition are visualized in the following figure. Firstly, the IR camera distortion part can be calibrated separately by the standard checkboard calibration procedure . It can The parameters of Structure Sensor’s camera radial distortion and tangential distortion are estimated and listed in After using the standard calibration method to model the camera-related barrel distortion, the pincushion distortion caused by projector can be simply separated by subtracting the observed data by the camera part  a bTwo experiments were designed to test the performance of the proposed calibration method with the SLP RGDB sensor, Structure Sensor. The performance of the calibration model was evaluated by comparing the difference between the modeled error and the actual error. Then a more straightforward showcase of the calibrated point cloud and the uncalibrated point cloud was provided to demonstrate the performance of the proposed calibration model. To test the calibration method’s performance, a test dataset that followed the same setup as mentioned in A more direct method to evaluate the quality of the calibration model is the visualization of data quality improvement of the measured point cloud. The proposed method was tested in three different scenes: (1) flat surface; (2) non-flat (curved) surface; and (3) flat surface with distance variation, shown in The accuracy improvement of the proposed calibration method can be demonstrated from two different regions within the depth frame. As shown in The RGBD sensor has a promising future to replace the high-cost 3D laser scanner and be applied in high-precision applications such as robotics, high-precision localization and mapping. The radial systematic error in the depth frame significantly limits its potential applications. Targeting this problem, this paper presented range-independent disparity-based calibration method for the SLP RGDB sensor. By revealing the real cause and forming mechanism of the non-centrosymmetric depth distortion, the proposed calibration method targets the disparity unit rather than the depth. By applying the calibration model in disparity and calculating the calibrated depth based on the calibrated disparity, the calibration model is independent of distance. No additional distance-based calibration or parameters look up table is required for the proposed method. With only one model with 20 parameters, the proposed calibration method can provide a full range coverage for the SLP RGBD sensor. A new non-centrosymmetric distortion calibration model for the normalized disparity is proposed in this paper based on the discussion on the form of the mechanism of the SLP RGBD sensor distortion. The proposed non-centrosymmetric distortion model can significantly reduce the large offset in the edge area of the depth frame. Since the heavily distorted edge area now can be calibrated to a similar accuracy level to the central area, more usable and valid information can be extracted and used to benefit applications such as SLAM, robotic exploration and obstacle avoidance. A new two-step calibration procedure is also developed in this paper to calibrate the barrel distortion caused by the IR camera lens, pincushion caused by the IR projector lens, and the systematic error caused by the IR cone. In the experimental results, the full frame and full-range coverage of the proposed calibration method is demonstrated. The comparison between the calibrated and uncalibrated point cloud clearly shows that the systematic errors in the measured point cloud have been removed by the calibration model. The significant offset in the edge area of long-range depth is reduced by the proposed model from 86 cm to 30 cm, which means that the relative error is reduced from 11% to 3% of the range distance. Overall, at far range the proposed calibration method can improve the depth accuracy by 70% in the central region of depth frame and 65% in the edge region.Further work will study the long-term stability of calibration parameters for the consumer-grade RGBD sensor. Other potential systematic error sources, such as illumination condition, temperature, humidity, and air refractive index, will be investigated for a more comprehensive calibration model.