We study video inpainting, which aims to recover realistic textures from damaged frames. Recent progress has been made by taking other frames as references so that relevant textures can be transferred to damaged frames. However, existing video inpainting approaches neglect the ability of the model to extract information and reconstruct the content, resulting in the inability to reconstruct the textures that should be transferred accurately. In this paper, we propose a novel and effective spatial-temporal texture transformer network (STTTN) for video inpainting. STTTN consists of six closely related modules optimized for video inpainting tasks: feature similarity measure for more accurate frame pre-repair, an encoder with strong information extraction ability, embedding module for finding a correlation, coarse low-frequency feature transfer, refinement high-frequency feature transfer, and decoder with accurate content reconstruction ability. Such a design encourages joint feature learning across the input and reference frames. To demonstrate the advancedness and effectiveness of the proposed model, we conduct comprehensive ablation learning and qualitative and quantitative experiments on multiple datasets by using standard stationary masks and more realistic moving object masks. The excellent experimental results demonstrate the authenticity and reliability of the STTTN. Video inpainting involves smearing moving or stationary objects in a video frame sequence using masks. The smeared parts are filled back based on the current frame and the content information of other frames of the video, and the repaired video should have the effect that the smeared positions 'disappear'. Typical applications are video restoration  should be extracted from the reference frame? How can the information of the reference frame and the current frame be effectively extracted and used ?We put forward the spatial-temporal texture transformer network to solve these problems in video inpainting. STTTN is divided into the following six parts: (1) Make the feature similarity measure more accurate and frame pre-repair. (2) By introducing Regional Normalization (RN)  introduced into the encoder and decoder creatively stabilizes the effect of video restoration.From the results of various mask experiments on multiple datasets, STTTN achieves excellent results visually and in terms of evaluation parameters.Video Inpainting refers to smearing some fixed areas or moving objects in a video and filling the smeared areas back in a generated way. It is required as far as possible to leave no trace . Image inpainting fills the missing area in a single image. In a narrow sense, image inpainting is a subset of video inpainting. The difference between them is that video inpainting needs to integrate the temporal and spatial information acquired from all video frames and effectively transform and fill it back into the restored image. Image inpainting only needs to make use of the spatial information outside the mask of the current picture without considering the information of other frames because it is only a single picture without any temporal information. From the perspective of repair methods, video inpainting is mainly divided into two methods. One is explicit repair, that is, it acts directly on the image and repairs from the pixel level. The other is implicit inpainting, that is, it acts on image coding and inpainting from the feature level.Explicit inpainting is a network design pattern of 'graph to graph' and is directly constructed. Before the popularity of deep learning, the main image and video inpainting methods were diffusion-based and patch-based methods. The central idea of diffusion-based methods , coarse low-frequency feature transfer (CLFT), refining high-frequency feature transfer (RHFT), and decoder. Finally, we combed the whole video restoration process and explained the model's loss function more clearly through pseudo-code.We use implicit inpainting to repair the video from the feature level. To determine the defects of the previous architecture and achieve a better inpainting effect, we erase the time domain search part of the previous baseline model  to help with network training , K (attention information of memory frame), and V (representative content information of memory frame).The input to the encoder network consists of three frames . Inp, InpΔ, Ref, and RefΔ represent the input frames, input framesΔ, reference frames, and reference framesΔ, respectively. InpΔ and RefΔ consist of an RGB image, hole mask, and no-hole mask. The hole mask on the RGB image is a single-channel greyscale image, and the no-hole mask is an area other than the hole mask area. These inputs are concatenated along the channel axis to form a 5-channel image before being fed into the first layer. Ref are composed of three-channel RGB images. Given an input feature qi and kj. We calculate their similarity by dot multiplying q and kT, where T represents the transpose operation:Different from the previous operation of obtaining q, k, v through a linear transformer and then calculating attention . The previous attention mechanism converted V over different temporal domains with input frames via CLET. More specifically, we first calculate a coarse low-frequency feature transfer map P in which the i-th element Pi is calculated from the relevance Ri, j:However, doing so may transfer a large number of textures that are not useful for the input frame to the target frame, resulting in blurring of the repaired area. To improve the ability to transfer low-frequency texture features for all reference frames. we will correlate the coarse low-frequency features in pi in map P represents the most relevant position index of a frame on all reference frames with the i-th position of the input frame. The specific calculation process obtains the index corresponding to the maximum value through the second item of the return value of the torch.max function. After obtaining the most relevant position index, we extract the low-frequency texture features that should be transferred most, so we only need to take the position of the frame that needs to be transferred in the patch v of the unfolded, then we can get the texture feature map T, where each position of T contains the high-frequency texture features of the most similar position in the Ref, where ti represents the value of the i-th position of T:That is, each value We obtain a rough feature representation T for input frames, which is then used in our refinement of high-frequency feature transfer (RHFT).W is calculated from Ri, j to represent the confidence of the transferred texture features for each position in T. The specific calculation process for obtaining W is to obtain the maximum value of Ri, j through the first item of the return value of the torch.max function, where W records the specific correlation of the most relevant feature block.High-frequency detail information is also essential for video inpainting  and fuse them with high- and low-frequency features. The above operations can be expressed as the following formula:To make full use of the original image information of the input frame, we divide the features of each level into two steps. First, the low-frequency texture features Conv, Concat, and ⊙ represent the convolution (the convolution operation adopted here is consistent with the convolution operation adopted by the DNN above), the concatenation operation, and the dot product, respectively, and Fout is the feature of the spatiotemporal texture output of the input frame combined with reference frames.In the deep network, each corrupted area and uncorrupted area are increasingly difficult to distinguish, and the corresponding mask is difficult to obtain  represents the number of channels, height, and width of the feature map of this layer; and IRef is the reference video for all frames. T is the texture feature transferred from V in The first part is no different from ordinary perceptual loss . The students chose one image from the two repaired images containing STTTN each time they thought the repair was better and better; that is, each person made ten choices, for a total of 100 × 12 = 1, 200 voting choices. In To verify the effectiveness of each part of STTTN, we carried out ablation learning, which is each part of texture transformation, and loss function, where each part of texture transformation contains four sets of ablation learning and the loss function contains two sets of ablation learning.As shown in The five columns in the first row of In this paper, we proposed a novel joint spatial-temporal texture transformer network for video inpainting. Each component cooperates closely, and the repair process progresses layer by layer, making full use of the texture feature information in time and space. The model has outstanding information extraction and content reconstruction capabilities in details and contours, which are essential and suitable for video repair tasks. The excellent results of STTTN's experiments on multiple datasets in multiple scenarios fully demonstrate its superiority over other methods.However, in our exploration process, we found that STTTN has certain defects and room for further improvement. First, the first defect is also the region normalization defect, that is, color casting is prone to occur. The second defect, which is also a defect that the entire implicit inpainting architecture is prone to have, is the inconsistency in the temporal domain . Video tests appear periodically. We hope that our exploration of video inpainting can help other researchers explore further in this field and that researchers can propose more advanced models to improve the defects we have found thus far.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.PM provided the idea of the algorithm and designed the entire architecture and was responsible for the writing of the manuscript and the conduct of the experiments. TX reviewed the manuscript. Both authors read and approved the final manuscript.This research was supported by the Shaanxi Provincial Technical Innovation Guidance Special (Fund) Plan in 2020 (2020CGXNG-012).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


In view of the difficulty of using raw 3D point clouds for component detection in the railway field, this paper designs a point cloud segmentation model based on deep learning together with a point cloud preprocessing mechanism. First, a special preprocessing algorithm is designed to resolve the problems of noise points, acquisition errors, and large data volume in the actual point cloud model of the bolt. The algorithm uses the point cloud adaptive weighted guided filtering for noise smoothing according to the noise characteristics. Then retaining the key points of the point cloud, this algorithm uses the octree to partition the point cloud and carries out iterative farthest point sampling in each partition for obtaining the standard point cloud model. The standard point cloud model is then subjected to hierarchical multi-scale feature extraction to obtain global features, which are combined with local features through a self-attention mechanism, while linear interpolation is used to further expand the perceptual field of local features of the model as a basis for segmentation, and finally the segmentation is completed. Experiments show that the proposed algorithm could deal with the scattered bolt point cloud well, realize the segmentation of train bolt and background, and could achieve high segmentation accuracy, which has important practical significance for train safety detection. Safety is the main research focus in the railway field ,2,3. As The raw point clouds of train key components used in this paper were collected and provided by inspection personnel at the railway site. According to the analysis of these point clouds, it is found that these point clouds generally have problems such as noise points, blurred boundaries, and a large number of points, which will affect the accuracy of the later task of the point cloud, but also bring large calculation costs. Therefore, an efficient preprocessing framework is particularly important. In this paper, a point cloud adaptive weighted guided filtering (AWGF) algorithm and a fast farthest point sampling algorithm based on key points (FFPS-kp) are designed based on the characteristics of noise and point number. After the point cloud preprocessing is completed, a deep learning network is used to segment the point clouds. First, after hierarchical and multi-scale feature extraction, global features are obtained, and these global features are used twice: (1) input them into the subsequent network directly for upsampling using the inverse distance weighted (IDW) interpolation method; (2) combine with a self-attention mechanism to generate weights of optimization characteristics, and the strengthened or suppressed features are used as the input of the skip connection. The other end of the skip connection is the global features sampled by IDW interpolation. After splicing these two features, they are the final basis for point cloud segmentation.Our major contributions can be summarized below:1. A 3D point cloud smoothing and edge-preserving algorithm is introduced which can adjust the filter weight adaptively, which enables us to smooth the scattered point clouds of train key components and get clearer contour lines;2. A fast farthest point sampling algorithm of key points is proposed, which improves the speed of farthest point sampling (FPS) and saves the calculation cost without sacrificing accuracy;3. A 3D point cloud segmentation framework suitable for train key components such as bolts is designed, and a deep learning network for bolt point cloud segmentation was trained. The original point cloud with noise points could be processed directly, and a high segmentation accuracy and mIoU were achieved in the dataset of the point cloud of the train bolt components.The remainder is structured as follows: In Bolt detection based on computer vision technology generally resort to 2D images obtained by cameras. For example, Li et al. trained a support vector machine (SVM) classifier to distinguish bolts from images based on local binary pattern (LBP) descriptors realized positioning by the rotate-and-slide window method and then identified whether the bolts were normal according to whether the bolts had a hexagon shape . In 18]18], a piFirst, the problem of acquisition error and noise can be smoothed by the position of the correction points. Ref.  proposedTo solve the problem of large amounts of point cloud data, there is a series of point cloud descending sampling algorithms. In deep learning networks, FPS  and randAt present, the 3D point cloud segmentation algorithms based on deep learning can be roughly divided into direct segmentation methods and indirect segmentation methods . Among ts. DGCNN  extractes. DGCNN . The cors. DGCNN  applied s. DGCNN  was props. DGCNN  abandonsIn this paper, a 3D point cloud segmentation network is introduced, whose input is some disorderly, scattered, and noisy point clouds. The overall process is shown in First, the raw point cloud is preprocessed, including denoising and simplification. Adaptive weighted guided filtering of the point cloud (AWGF) is used for correcting the noisy points and smoothing the point cloud. Then, the fast farthest point sampling algorithm of key points (FFPS-kp) is used for simplifying and improving the quality of the point cloud. The preprocessed point cloud model is extracted through multi-scale and multi-layer feature extraction, and the global features are extracted for subsequent point cloud segmentation. The feature extraction layer consists of four parts: sampling layer, grouping layer, convolution layer, and max-pooling layer. The receptive field obtained after multiple feature extraction is large and almost close to the global information. Then, the local features of global regions are expanded by step-by-step upsampling through IDW. At the same time, the global features of local regions extracted from each feature are optimized through the self-attention mechanism, and all features used for point cloud segmentation are obtained by skip connection with the expanded local features of global regions. Then, scores are output through the fully connected network to complete the segmentation task.Due to the impact of the scanning device, the external environment, and the target object itself, the point cloud obtained directly by the laser scanner will have abnormal data such as noise points or outliers inevitably. These abnormal data will affect the accuracy and efficiency of the subsequent point cloud task, so it is essential to denoise the raw point cloud data ,40. The First, the topological relationship is constructed by using K-D tree for the input point cloud When the cost function Since the number of the 3D point cloud obtained by scanners is huge, such large point cloud data entering the deep learning network directly will make the training time too long. In another word, if there is a downsampling process, the network efficiency will be improved. The downsampling algorithm named FFPS-kp is shown in The denoised point cloud The three feature vectors of After the curvature of all points in titioned . In eachFinally, the combination of curvature sampling and partition FFPS-kp is regarded as the final sampling result, set as To improve the segmentation accuracy, both local features and global features should be considered when feature extraction is carried out. For example, a fatal weakness of PointNet is that it fails to preserve the local features, resulting in lack of detailed description capabilities . For thiThe global feature learning process is as follows:(1) The preprocessed point cloud (2) FFPS-kp is used for sampling the (3) MLP and max-pooling are used for extracting and concentrating the features of all points in (4) For (5) The third feature extraction is carried out for (6) Finally, the The However, the point cloud features obtained by upsampling fail to make the best of the point cloud local information obtained during feature extraction. Therefore, the global features of local regions extracted by hierarchical multi-scale features are spliced into the corresponding global feature layer after upsampling using skip connection. In the process of skip connection, the self-attention mechanism is adopted for feature optimization of the global features in local regions, and then it is combined with the global features, so that the global features can better express the local information of the point cloud and also improve the feature expression ability and the segmentation accuracy accordingly. The core of the self-attention mechanism is to use other information in the target to enhance the semantic representation of the target information and make better use of the context of the target information .The experimental hardware configuration is NVIDIA GeForce RTX 3060, and the software environment is Windows 10, Python3.6.12, PyTorch 1.7.1, CUDA 11.2. The experimental parameters were set as Batch size 24 and Epoch 200. An Adam optimizer was used when training networks. The initial learning rate is set to 0.001, the learning rate decay index is 0.0001, and the activation function is RELU; the loss function is the cross entropy for the segmentation network.In this experiment, a laser 3D scanner was used to collect the data of point clouds containing bolts as train key components. We selected 1256 point clouds containing bolts and point cloud labels were assigned. Label 0 represented bolts and label 1 represented the background. Each point cloud in the datasets containing the bolt position consists of its X, Y, Z coordinates, X, Y, Z normal values, and labels. In this paper, time is used to evaluate the calculated efficiency of denoising and sampling, overall accuracy (OA). Intersection over union (IoU), and mean intersection over union (mIoU) are used for evaluating the segmentation effect. The calculation formulas are listed below:In order to compare the performance of the denoising and smoothing algorithm AWGF in this paper with the BF algorithm and the GF algorithm, a comparative experiment was conducted on four bolts in the dataset. The corresponding number of bolts is shown in Through the analysis of To sum up, by analyzing the experimental data and comparing the smoothing results in In order to compare the performance of the proposed algorithm FFPS-kp with FPS, four bolt point clouds were selected from the datasets for the downsampling comparison experiment, and the sampling numbers were set as 512, 1024, and 2048, respectively. The number of sampling points and the corresponding sampling time are listed in In order to evaluate the performance of the proposed network architecture on the bolted point cloud segmentation task, we selected several popular point cloud segmentation algorithms, PointNet, Pointnet++, PointConv, DGCNN, and PointMLP to make the comparison. To reduce the calculation consumption in the preprocessing part of the experiment, the actual dense bolt point cloud was downsampled, and 2048 points were reserved for feature extraction. To compare the proposed method with other methods visualized, we selected some bolts randomly and compared the segmentation effect in As shown in In order to explore the influence of the denoising module, the sampling module, and the attention mechanism module on the bolt segmentation results, we conducted a comparison of module ablation, as shown in In short, the ablation experiment verifies the effectiveness of the proposed preprocessing module in point cloud segmentation, including the denoising module and the downsampling module. It also shows that the self-attention mechanism can make better use of local information, strengthen the extraction ability, and improve the segmentation performance.Aiming at the actual train bolt point cloud model, we established a hierarchical and multi-scale feature extraction point cloud segmentation algorithm, which includes filtering smoothing, downsampling, and other preprocessing operations combined with an attention mechanism. First, the AWGF algorithm proposed in this paper can remove the noise points in the scattered point cloud, making the contour of the point cloud smoother and the boundary clearer. To solve the problem of downsampling, we optimized the FPS algorithm and proposed FFPS-kp. Compared with FPS, FFPS-kp greatly shortens the sampling time and improves the sampling efficiency without losing the segmentation accuracy. Finally, the hierarchical multi-scale point cloud segmentation model proposed in this paper performs well in the segmentation experiment of the actual train bolt point clouds and their backgrounds, and can successfully segment the bolt point clouds, which can be used in actual railway environments.However, this algorithm takes too long time to segment the point cloud in large environments. In the future, we would consider designing a more efficient point cloud downsampling algorithm that could pay more attention to key features, so as to reduce the calculation amount for later feature learning, such as the idea of the critical points layer. At the same time, we would try to simplify the feature extraction structure, and further shorten the time without losing the accuracy and mIoU, so that the network could be applied to the rapid segmentation of the original point cloud in large train environments.


This paper presents the realization and testing of a low power sensor node equipped with long range wide area network (LoRaWAN) connectivity and providing 2D Visible Light Positioning (VLP) features. Three modulated LED (light emitting diodes) sources, the same as the ones commonly employed in indoor environments, are used. The localization feature is attained from the received light intensities performing optical channel estimation and lateration directly on the target to be localized, equipped with a low-power microcontroller. Moreover, the node exploits a solar cell, both as optical receiver and energy harvester, provisioning energy from the artificial lights used for positioning, thus realizing an innovative solution for self-sufficient indoor localization. The tests performed in a ~1 m Nowadays, accurate indoor positioning (IP) is one of the most ambitious challenges. While in the outdoor positioning, the global positioning system (GPS) is the dominant navigation technology, the same cannot be said for metropolitan and indoor applications, where its pervasive coverage and its high precision are strongly degraded by the presence of walls, furniture and obstacles. Indeed, one of the main aspects characterizing IP is the great heterogenicity of the indoor environment involved: the geometry, the type of obstacles and the required position accuracy greatly vary according to the application proposed and play an important role on the choice of the localization approach. This has led to the development of indoor positioning systems (IPSs) based on different techniques, extensively treated in the literature with surveys and reviews that compare their strengths and weaknesses ,5,6,7,8.Radio frequency technologies , global system for mobile communication (GSM), Wi-Fi, Bluetooth, Zigbee and long range wide area network (LoRaWAN)), sound-based technologies , optical technologies  and magnetic field navigation are examples of commonly experimented indoor localization techniques relying on the presence of uniquely identified transmitters  as anchors to localize the mobile receiving target . Some of these technologies as Wi-Fi or visible light are advantageous since they exploit existing infrastructures, thus requiring lower installation and maintenance costs with respect to positioning systems using specialized transmitters. Other approaches imply the usage of sensors mounted on the target and providing absolute localization independently from the presence of reference points; nevertheless, frequent recalibration and filtering procedures are required. Some examples are camera-based systems  and inerDepending on the degree of precision desired, different localization strategies can be followed. The simplest approach for estimating the “raw” position is that of proximity detection, which gives information about proximity to a specific anchor instead of giving a precise position in the reference space. This approximate localization strategy finds application only in those contexts where a rough knowledge of the position is required . If a higher level of accuracy is needed, triangulation and fingerprint methodologies can be adopted. The fingerprint method requires an offline training phase for collecting measurements or simulating data in several tests points, thus creating a database or a look up table of the operating space. Starting from the knowledge of the training set, other test points can be inferred, resorting to machine learning techniques or statistical methods. The major weaknesses of this approach are the time-consuming calibration phase, whose duration increases with the desired granularity, and the strong dependence on the specific test environment, requiring new time-consuming trainings every time it changes. The triangulation approach exploits the geometrical properties of the triangles formed by the target and the anchors to infer the target location. It comprises two different methods: angulation—which exploits the estimated angles between the target and multiple reference anchors—and lateration—which uses the estimated distances between the target and multiple reference anchors. The main quantities measured in these localization strategies are Angle of Arrival (AoA) for angulation, and Time of Arrival (ToA), Time Difference of Arrival (TDoA) and Received Signal Strength (RSS) for lateration. The first three techniques provide maximum position accuracy to the detriment of higher system cost and greater complexity. On the other side, the RSS-based method is site-specific and can be affected by errors in the estimation of the path loss model, due to signal reflections, shaded areas and obstacles between the receiver and the transmitters; moreover, some parameters of the model must be appropriately estimated during a calibration procedure to reduce errors in the subsequent localization phase.Some possible application fields of IP are healthcare , industry  and Internet of Things (IoT) . The optimized management of energy resources and the attention towards sustainability are nowadays trend topics that are gathering growing interest. The increasing of IoT devices in paradigms such as Smart Home, Smart Factory and Smart City made more urgent the need to design systems independent from the connection to the electricity grid and to the periodic replacement of disposable batteries, or at least to extend their lifetime. Several sources can be exploited to perform energy harvesting for wireless sensor nodes. Wind , electroFor this reason, among the aforementioned IP technologies, visible light positioning (VLP) has arisen as valid solution for the deployment of low power localizable targets ,2,7, simIn this paper, we propose a low power VLP system equipped with LoRaWAN connectivity and employing a small-sized PV module both for 2D indoor localization and energy provisioning. The designed prototype is conceived in a low power perspective in such a way to be compliant with the energy harvesting aspect. Indeed, resorting to a solar cell-based IPS is inherently a low power approach with respect to radio frequency positioning methods using power-consuming off-the-shelf receivers . Moreover, the electronic components are selected among families specifically thought for applications envisaging low consumption. The localization methodology adopted is lateration, performed by using the RSSs of three LED (light emitting diode) placed in known positions of the measurement area. The RSSs extraction is performed by means of fast Fourier transform (FFT) of the received light signal in order to recover the contribution of each lamp, univocally identified by a unique driving frequency. The coordinates extraction is directly performed on board of the target itself and sent via LoRaWAN, making it available on a server, thus avoiding post-processing operations at the backend side as is the case of RSS-based solutions using radio technologies. Only one measurement point in the center of the measurement area subtended by the three LEDs is needed to calibrate the localization algorithm.Considering that the system is designed keeping in mind the low-cost and low-power requirements, it will not intrinsically reach the sub-cm accuracy of expensive and highly precise devices. However, it must be accounted that in a real context indoors, the achievement of this level of accuracy with localization is quite unrealistic; other factors, such as the presence of obstacles, shadows and reflections inevitably lead to a degradation of the performance or even to the total impossibility of localization.This manuscript is organized as follows: in IPSs based on VLP using LEDs have been widely studied in the literature ,2,7. TheSome applications involving solar cells as photodetector concern visible light communication (VLC) ,24,25 orTherefore, to our knowledge, our contribution is the first one presenting an integrated energy self-sufficient LoRaWAN-based device performing on board 2D indoor positioning using a PV module both for localization and for artificial light energy harvesting. Moreover, the great advantage of this application relies on its low complexity in the demodulating side, which can be entirely realized resorting to off-the-shelf components making the proposed system a valid solution for large scale deployments.The presented approach exploits three LEDs, univocally identified by a unique operating frequency, as anchors to recover the 2D position of the optical receiver from the knowledge of the LEDs coordinates in the measurement space and from the extracted RSSs. In particular, a PV module is used as optical receiver, producing a photocurrent converted into a voltage directly proportional to the incident optical power of each LED . The fact of having assigned a unique frequency to each LED made it possible to avoid, during the reception phase, interference between the light signals coming from the three sources at the same time. This approach also allows to ignore in the demodulation phase the contribution of the background light at a frequency different from those transmitted or which acts as an average value on the detected signal.The geometrical representation of the set-up together with the reference system is reported in RSSs extraction through FFT algorithm;Lambertian optical channel model inversion;Lateration method to predict the coordinates of the target PSome parameters of this model are obtained from the knowledge of the physical characteristics of LEDs and solar cell , dealing with the energy harvesting from the PV module used as light sensor. Low-power components and programming strategies have been selected in order to realize an energy self-sufficient localization device.The mounted optical receiver is an amorphous silicon PV module manufactured by Panasonic and specifically thought for working under artificial light illumination. Indeed, the external quantum efficiency (EQE)’s response band of amorphous silicon is limited to the visible range with a maximum around 540 nm, well-matching the spectral distribution of the LEDs used as light sources. The PV module is made on a glass substrate and is composed of a series of 8 cells with an overall active area During the localization phase, by properly driving the two electronic switches mounted as shown in The output of the TIA is acquired via the ADC of a STM32L476RGT6 ARM Cortex M4 microcontroller mounted on a NUCLEOL476RG development board from ST Microelectronics. The microcontroller is in charge of digital processing the sampled voltages by performing the discrete Fourier transform (DFT) of the signal by means of FFT algorithm in order to extract the RSS of each light source. Once the three magnitudes are computed, the trilateration algorithm is performed outputting the estimation of the 2D coordinates of the optical sensor.These data are then sent via LoRaWAN protocol to a LP68 Dragino gateway, which redirects the packets to a ChirpStack Server connected to a InfluxDB database. The LoRa low power radio module employed is a RFM95x equipped with a λ/8 antenna. The used radio settings are transmitting frequency 868 MHz, bandwidth (BW) 125 kHz, output power 5 dBm, coding rate (CR) 4/5 and spreading factor (SF) 7. Both CR 4/5 and SF 7 are chosen since they guarantee the lowest power consumption and the shortest time on air (working currents with peaks up to 45 mA for ~62 ms in transmission) to detriment of worse error correction at the reception and link margin reduction which, however, are not critical aspects in the proposed scenario where kilometric radio coverage is not required.During the node self-sufficiency test, the MCU is programmed according to a sleep routine, which periodically activates the MCU for the time required to retrieve the 2D position of the object and send the data. Since the position is computed on the average of 20 implementations of the FFT algorithm, a run period of approximately 2.5 s can be considered. One digital I/O port is used both to turn on a MOSFET driving the LoRa module and to supply the TIA, in order to switch them off during the sleep periods of the MCU thus avoiding extra power consumption. In this way, the current absorption of the node in standby mode falls below the μA.The number of points used for the FFT computation, and the ADC sampling frequency are conveniently chosen in accordance with the LEDs signals in order to theoretically avoid spectral leakage and frequency interference.The number of samples required by the FFT algorithm is set to N = 4096 while the ADC sampling frequency is set to In light of these considerations, a possible estimation of the maximum number of LEDs in one single room is 10; in the same room, it is not possible to have two or more LEDs with the same identifying frequency, that instead can be reused in a different room.The ultralow-power energy harvester and battery charger employed to collect energy from the three LEDs is the SPV1050 manufactured by ST Microelectronics, mounted on the STEVAL-ISV020V1 board. It is a commercial integrated circuit (IC) featuring boost charging from sources delivering at least 150 mV in order to charge a storage element, in the case study, a rechargeable 3.7 V 3400 mAh Li-Ion battery employed as power reserve. Undervoltage and overvoltage thresholds can be set through resistive dividers to protect the battery from overcharging and excessive drainage from the load. Indeed, the integrated buck converter provides a regulated voltage supply of 3.3 V used to power the MCU, which in turn drives the system electronics. Moreover, the IC offers an adjustable internal maximum power point tracking (MPPT) functionality, which samples the open circuit voltage of the energy source, In the tested conditions of minimum illuminance , the average voltage and current at the operating point 0.8Solid-state switches (MOSFET and JFET) are used to manage the same PV module both for energy harvesting and for localization estimation, thus preventing the reading of the light signal from being affected by the MPPT process of the BMS, which fixes a constant voltage at the terminals of the solar cell using a variable load and a switching strategy. As soon as the MCU wakes from standby mode, it enables the same I/O port used to pilot the LoRa module to drive the switches, which temporarily detach the cell from the BMS for few seconds and connect it to the TIA input; therefore, during this period, no energy is harvested. Once the position coordinates are sent via LoRaWAN and before the MCU re-enters standby mode, the I/O port is disabled, and the PV module is again used for energy provisioning. Moreover, this strategy assures that in case of deactivation of the load by the BMS  the PV module remains connected to the BMS input to guarantee battery recharging. The implemented configuration is depicted in In this section, the results of the measurement campaigns are reported, performed using the measurement set-up and the methods described in h were measured by means of an independent technique using a measurement tape with an uncertainty of 0.5 cm). In the same figure, the points taken with steps of 30 cm, but using the Hann window, are represented in green, whereas the black circles identify the projections of the LEDs on the floor , by keeping the light sensor laid on the floor, i.e., with the optical axis parallel to the LEDs’ ones, and collecting a triad of LEDs RSSs in each measurement location. This information is then used by the on-board microcontroller to recover the 2D position. In x-axis there is the error in cm.sf = 5 while the illumination in lx at the floor level is analogous to that which would occur in case of a greater distance between LEDs and receiver. Considering Equation (1), the received power The measurement area is the scaled version of a real one with scaling factor Moreover, since in the test bench, x,y) plane reports the absolute difference in mV between the extracted RSSs and the RSSs predicted by the model, i.e., derived by inverting (2) and by substituting to x-axis and y-axis at steps of 10 cm.This aspect is further highlighted in These results confirm that the estimated To assess the capability of the proposed IP system to counteract possible clock inaccuracies, expected in a low-cost implementation, an additional test was performed introducing a very large misalignment between clocks, i.e., injecting an error of 1% in the frequency generation of Note that the dispersion of the results obtained without the Hann window, due to spectral leakage, is related to the random phase among the signals.Finally, a further test was performed by shielding the measurement area in order to evaluate the effect that the external lights have on the localization accuracy . The material used is not anti-reflective since the goal of the shielding was only to hinder the external light sources and not to avoid reflections.The optical receiver is moved with steps of 30 cm along the 14 points constituting the perimeter of the entire measurement area, since these points, for the arranged measurement set-up, are the most affected by the external light presence. The results are reported in The node is tested in the worst illumination condition, that is the center of the measurement grid in point  where the minimum level of illuminance is experienced . The MCU is programmed to work in sleep mode, waking up periodically  to transmit via LoRa technology the estimated position of the target. The goal of the tests is to experimentally validate the energy self-sufficiency of the proposed architecture using the solar cell both as energy harvester and optical receiver for VLP, to this aim the voltage trend of the rechargeable Li-Ion battery, The battery voltage behavior, monitored for approximately 10 h for each measurement campaign, is shown in The increase of the battery level, net of node consumption, is minimal since the power delivered by the cell in the minimum lighting condition is around 305 µW; however, the difference between the voltage level at the end and at the beginning of the tests is positive in both cases. In particular, while with transmissions every 15 min, a battery charge, albeit small, is detectable, in the case of transmissions, every 10 min, a perfect balance between harvested and absorbed energy is achieved and an increasing trend can be detected only in the long period.Thus, these results validate the energy self-sufficiency of the system even in case of minimum illuminance conditions and quite frequent transmissions. Indeed, one transmission every 10 min or at least 15 min can be considered a good transmission rate for those deployments where real-time localization is not required, as car tracking in a parking or goods inventory.This paper presented a solar cell-based low power system featuring 2D VLP and autonomously powered by the indoor artificial lights exploited for the localization task.Three modulated LED sources, univocally identified by a unique operating frequency, were used as anchors to infer the optical receiver position. In particular, the localization was performed by means of a low-complexity algorithm foreseeing FFT extraction of the received light intensities, optical channel estimation and lateration using just one measurement point  for channel parameter extraction. The 2D coordinates were derived directly on board exploiting a low-power microcontroller, then sent to a server using LoRaWAN as data transmission protocol. The system was equipped with an ultralow-power energy harvester and battery charger providing battery overvoltage and undervoltage protection, energy provisioning and a regulated power supply at 3.3 V for the node. A small-sized amorphous silicon solar cell, well-matching the spectral distribution of the used LEDs, was employed alternately as energy harvester and as optical receiver, in order to perform fully energy autonomous indoor localization.Field tests were performed moving the receiver in fixed-length steps along an ad-hoc measurement set-up covering a localization area with dimensions 90 cm × 120 cm, divided into 10 cm × 10 cm squares identifying 130 distinct 2D positions in the plane. In order to make the proposed system less prone to inaccuracy in the generation of the driving signals, a Hann window was added before FFT and new tests were performed considering a frequency mismatch of 11 Hz in the generation of the signal driving Two additional tests were performed to assess the energy self-sufficiency of the node in the  point, which is the one at lower illuminance . The Li-Ion battery voltage was monitored for 10 h with the three LEDs on and considering transmissions every 15 min and 10 min, which are compliant with quasi-real time monitoring tasks. The obtained results prove the fully energy autonomy of the target in the condition of worst illumination , satisfying both the harvesting and the localization tasks, thus gaining a node lifetime virtually infinite. However, since the tests demonstrate that the power delivered by the PV module is sufficient for supplying the node operations, the proposed system may be further engineered to be fully battery-less, embedding, at most, a small supercapacitor to manage the peak current absorptions of the radio transmissions. In this case, the fact of foreseeing an energy provisioning procedure becomes a strength with respect to similar nodes relying on a rechargeable/disposable battery as on-board energy storage.In conclusion, the outcomes of these preliminary measurement campaigns suggest the adaptability of the system to IoT, Industry 4.0 and Smart Cities scenarios, where artificial lights are often available energy sources.To further strengthen these results, field tests may be carried out in future collecting measurements in a wider indoor deployment and facing new challenges aimed at handling the mentioned weakness of the system, such as the parallelism between LEDs and receiver axes, the presence of obstacles and shadows, which are well-known limitations in VLP applications. Although the proposed methodology can manage inclinations in the order of ±10°, PV modules with better relative sensitivity can be selected and a hybrid positioning strategy can be implemented mounting an IMU on board in order to account for the inclination angles. Similarly, novel solutions different from the one proposed can be investigated to detect the presence of obstacles, completely or partially shading the light sources, and consequently to discard the result of the localization when not reliable, considering that in most cases these are transient disturbances or fix obstacles that can be accounted in a preliminary calibration phase.However, despite these limitations, the system in the present form can find application in those scenarios where indoor 2D localization is required and the presence of obstacles is limited, such as localization of cars in a covered parking area, containers in a warehouse or people moving indoor.


This paper presents an innovative multi-resident activity detection sensor network that uses the Bluetooth Low Energy (BLE) signal emitted by tags worn by residents and passive infrared (PIR) motion sensors deployed in the house to locate residents and monitor their activities. This measurement system solves the problem of monitoring older people and measuring their activities in multi-resident scenarios. Metrics are defined to analyze and interpret the collected data to understand daily habits and measure the activity level (AL) of older people. The accuracy of the system in detecting movements and discriminating residents is measured. As the sensor-to-person distance increases, the system decreases its ability to detect small movements, while still being able to detect large ones. The accuracy in discriminating the identity of residents can be improved by up to 96% using the Decision Tree (DT) classifier. The effectiveness of the measurement system is demonstrated in a real multi-resident scenario where two older people are monitored during their daily life. The collected data are processed, obtaining the AL and habits of the older people to assess their behavior. With the development of the Internet of Things (IoT) and the rise of powerful and affordable smart devices, sensing technologies have become the basic pillars of any smart application today to provide independent living solutions for aging people. Monitoring the practices, activities and habits of older people is essential to assess their health and ability to live independently ,2. Autom-Vision-based ,4,5,6,7:-Tag-based ,12,13,14-Signature-based ,18,19,20The most used solutions for smart homes to track residents while carrying out activities are:Many approaches such as data association ,22 and aThe main contribution of this paper is the presentation of an innovative multi-resident activity detection sensor network that combines tag-based (wearable BLE tags) and signature-based (PIR motion sensors) technologies to locate and identify residents and monitor their activities in multi-resident scenarios. The ease of use and deployability of the sensor network, its low cost, non-invasiveness, respect for privacy, robustness and reliability are all advantages that make it suitable for installation in the home to monitor residents compared to systems in the literature. Furthermore, this system can be extended to larger environments and contexts with more residents. The sensor network can also be used in nursing homes where the monitoring of older residents can help caregivers understand their habits and daily activities. The system can track and identify each resident wearing a BLE tag, achieving 96% accuracy in identifying residents using the DT classifier. The effectiveness of the measurement system is demonstrated in a real multi-resident scenario where two older people are monitored simultaneously during their daily life and their activity levels are discriminated. Finally, we show that collected data can be processed by defining specific metrics to understand daily habits and measure the Activity Level (AL) of older people. The methodology proves to be an innovative approach to define a behavioral model: from the number of detected events (PIR activations) the parameters relating to the activity of the older people are obtained, while from the number of events acquired per location, it is possible to understand how much activity they perform to accomplish Activities of Daily Living (ADL). The assessment of this model can help to capture possible changes resulting from cognitive or physical health and to prevent or prognosticate diseases. Early detection of behavioral changes with the assistance of this technology can help improve health interventions and provide older people with greater awareness of an unhealthy and inactive lifestyle.The remaining part of the paper is organized as follows: Distinguishing the residents of the house is the first step in associating the detected activities with each of them. For this purpose, different technologies are used to continuously locate and identify residents in the home. As reported in Parameters such as residents’ comfort and privacy, their role in data collection , the accuracy of the technology, its cost, installation and invasiveness have a significant impact on their experience and are all involved in the decision on the sensing system choice. Since the use of vision-based solutions raises the issue of privacy, and since it is nearly impossible to provide data-binding from signature-based sensors alone, tag-based technologies are the most promising in recognizing residents. In this Section, we present literature works dealing with tag-based technologies that proved to be a promising solution for the purpose of locating and identifying home residents.Tag-based approaches rely on the use of wearable wireless technology to locate and identify people in the home. Each device has a unique ID that can be broadcasted to a receiving unit via wireless communications as BLE, Wi-Fi, RFID or ZigBee. In  a Wi-Fi-As can be seen from the literature, the monitoring of multiple residents in the same indoor environment remains an open challenge, which researchers are trying to answer with different technologies and approaches. In the case of older people-related monitoring systems, it is recommended to use a solution with high accuracy, low cost, low energy consumption and reliability. Based on these requirements, BLE devices are promising technologies. Furthermore, we found that combining tag and signature-based technologies could result in improved activity recognition.To monitor older people living in the home, it is necessary to associate the detected activities with each resident. To achieve this goal, we present a multi-resident activity detection sensor network that makes use of both signature-based and tag-based technologies. The components that make up the system have been chosen based on some requirements: they had to be inexpensive, non-invasive, respectful of privacy, with long battery life, portable and not bulky. We selected the HC-SR501 PIR sensor , price, performance, memory and connectivity. Furthermore, due to the availability of Wi-Fi and Bluetooth communication technologies, the Raspberry Pi Zero W is ideal for making embedded IoT projects. The PIR motion sensor emits a pulse when there is a significant change in the amount of infrared radiation it detects. There are three pins on the PIR sensor: one for the power (Vcc), one for the ground (Gnd) and one for the transmission of the signal of the detected movement (Out). The PIR sensor is wired to the Raspberry Pi Zero W : the VccThe Tile Mate  tags areThe multi-resident activity detection sensor network is designed to ensure simultaneous monitoring of two residents in three areas of the house; therefore, it consists of three PIR motion sensors wired to three Raspberry Pi Zero W devices and two wearable tags. Each Raspberry Pi Zero W is mains-powered and is equipped with a micro-SD card on which the Raspbian OS runs. Considering the cost of individual components, power supplies, electronics and storage, the proposed system costs around EUR 200. An advantage of this system is that it can be easily extended to larger environments and contexts with more residents. In fact, by integrating the system with more Raspberry Pi Zero W devices and PIR motion sensors it can cover more areas of the house , while integrating it with additional tags, we can track more people (each resident is assigned a tag and associated with a unique ID).The sensor network is designed to be installed in three main rooms of the home. Thanks to the integrated Wi-Fi module, the Raspberry Pi Zero W devices are connected to the home internet network, so that they store the data collected by the own PIR motion sensors in an online database. A Google spreadsheet was used as the IoT database to store sensor data and retrieve them for further processing. Through the Google Sheets API, the online spreadsheet access credentials and JSON security key have been created. The OAuth 2.0 authorization protocol was used to allow authentication and access to the Raspberry Pi Zero W devices via the JSON key for sending the collected data and saving them in the Google spreadsheet.Each resident of the home wears a tag that identifies them through a unique identifier (ID), which consists of the MAC address of the tag.When a PIR motion sensor is triggered by a resident’s movement, the Raspberry Pi Zero W connected to it scans the surrounding area searching for devices that emit BLE signals: the system is set to detect only the BLE signal of tags worn by residents.The BLE RSSI of the tags in the detection area are measured.To identify the resident who activated the PIR sensor, the measured RSSIs are converted into distances (in meters). The presence in the room is thus assigned to the residents in the detection area of the PIR sensor.The multi-resident activity detection sensor network  works foMP), a factory-calibrated read-only constant that indicates the expected RSSI at a distance of 1 m to the tag, with the measured RSSI, the distance of the tag  from the PIR sensor is estimated. For the Tile Mate tag, the MP is −58 dBm.Combining the Measured Power (The distance is computed through Equation (1) if 5.The system, meanwhile, stores the collected data in real-time in the online Google spreadsheet for further analysis. Each time the system detects a movement, it stores the identity of the resident , the time of the event, and its location (the room in the house where motion is detected).6.These data are then processed to obtain residents’ AL and habits using the metrics defined in The sensor network, in particular the PIR motion sensors, must be installed in such a way as to monitor the residents at least in the areas of interest within the smart home environment according to a pre-selection of which areas must be covered. In the real multi-resident scenario test, a PIR sensor is installed for each room where residents are expected to spend most of their time, i.e., the kitchen, bathroom and bedroom, in order to monitor their activities.Once the rooms of the house in which to monitor the residents have been identified, the PIR sensors should be installed in such a way as to cover the entire area. Therefore, it is important to know, in addition to the detection range of the PIR sensor, the dimensions of each room to evaluate the number of sensors needed to cover them.PIR sensors should be installed avoiding overlapping detection areas to prevent more than one sensor from detecting the movement of a resident, thus creating false events.It is preferable to install the PIR sensors at a minimum height of 1.5 m from the ground to better detect residents’ movements.It is preferable to avoid installing the PIR sensors near doors or leaves so that they are not obscured by them.To optimize data collection, we have listed several recommendations for PIR sensor deployment in the smart home that come from our experience:Behavioral changes can reflect changes in physical health and the continuous collection of sensor data can help observe these changes, which in some cases may be difficult to detect. Understanding daily habits and monitoring the AL of older people are essential to assess their health and ability to live independently. Therefore, we defined some metrics to analyze and interpret the data collected by the proposed system to extract useful information.The number of detected events (PIR activations) per day may reflect how physically active the older resident is. To assess the AL of older people, three ALs named Low Activity Level (LAL), Medium Activity Level (MAL), and High Activity Level (HAL) have been set. Considering that the minimum AL for healthy older people is reflected in 50 events per hour  and fromThe Daily Activity Index (DAI) reflects the activity performed by older people during the whole day. So, the resident DAI assumes a value of 1 if the performed activity falls within the LAL, 2 if the performed activity falls within the MAL, and 3 if the performed activity falls within the HAL.One of the problems of interest is to monitor the habits of older people. Residents carry out their activities at different times of the day, so it is useful to measure how often they perform activities on an hourly scale. The Hourly Activity Percentage (HAP) is computed through Equation (3) by dividing the number of events detected per hour in the whole house by the total daily events and allows determination of the hours of the day when residents are more active.Furthermore, the collected movements of older people inside the home can be used to retrieve the activities carried out in each room. The multi-resident activity detection sensor network is designed initially to be installed in three rooms of the house: bathroom, bedroom, and kitchen. The number of events captured per location may reflect how many of older people’s activities are assigned to perform a specific ADL such as personal hygiene, eating, etc. The Daily Room Activity (DRA) of each resident is computed by dividing the number of events detected in a room by the total number of events for each day (Equation (4)).Identifying typical habits can help capture possible changes in older people’s behavior by comparing two or more periods of collected data. If a noticeable difference emerges, it can be considered a symptom of an impending decline in physical and cognitive abilities and requires further investigation.The test phases can be divided into two steps. First, the measuring capabilities of the multi-resident activity detection sensor network were evaluated. We determined the accuracy of the system in detecting different types of movements  and its accuracy in discriminating the residents. We then moved on to installing the system inside a home inhabited by two older people to evaluate its functioning in a real multi-resident scenario and the effectiveness of the proposed metrics in determining the AL and habits of the older residents.To measure the accuracy of movement detection, a test was performed by having an individual wearing the tag positioned at a given distance from the PIR sensor. The participant was asked first to make small movements (turning the head or moving the hands) and then to make large movements  while remaining in the same spot. The procedure was repeated 20 times for each of the two different types of motion  for 7 different distances from the PIR sensor, to define the accuracy of the system in detecting different types of movements at different distances. The accuracy is computed through Equation (5):Two individuals wearing the tags performed a supervised test designed to measure the accuracy (Equation (6)) of the system in discriminating against residents. While one participant remains still, the other moves into the detection range of the PIR sensor to activate it. The process is repeated alternating between the participant who moves around the room and the one who remains motionless. For each of the participants, 250 PIR sensor activation events with relative ID detection were recorded for a total of 500 events. A third person tracks the movements of the participants to provide a reference for the comparison between the identity of the participant declared by the measurement system and the actual one. The identified events are then labelled as correct or incorrect detections based on whether the ID declared by the system matched the supervisor-defined ground truth or not.Furthermore, after determining the accuracy of the system, we tested some Machine Learning (ML) algorithms on the collected dataset to assess whether they would increase the effectiveness in assigning the detected activity to the correct individual. Four supervised ML algorithms commonly used for classification problems were tested on the dataset: Decision Trees (DT), Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), and Naive Bayes (NB). Accuracy (Equation (6)), precision (Equation (7)), recall (Equation (8)), and F1-score (Equation (9)) were calculated by splitting the dataset into 70% for training and 30% for testing. Furthermore, DT, SVM, k-NN and NB accuracies were calculated by splitting the dataset using 10-Fold Cross Validation (CV).The proposed multi-resident activity detection sensor network has been tested in a real multi-resident scenario. The participants involved in the test were volunteers, gave written informed consent to use their personal data and were duly informed about the goal of the research. The conducted study fully respects and promotes the values of freedom, autonomy, integrity and dignity of the person, social solidarity and justice, including fairness of access. The study was carried out in compliance with the principles laid down in the Declaration of Helsinki, in accordance with the Guidelines for Good Clinical Practice.The sensor network has been installed inside a home where two older people live: a 72-year-old man and a 64-year-old woman. A PIR sensor and a Raspberry Pi Zero W device were installed in the kitchen, bathroom, and bedroom. The test lasted a week, during which the residents carried out their daily activities always wearing the tag around their neck. The purpose of this test was the evaluation of the multi-resident activity detection sensor network functionality and the effectiveness of the proposed metrics in determining the AL and habits of older residents.This section presents the results of the test phases showing the accuracy of the system in detecting movements and discriminating residents, and the analysis of the data collected in the real multi-resident scenario test to assess ALs and habits of the older people.The presented multi-resident activity detection sensor network achieved an accuracy in discriminating the identity of residents during the supervised test of 76%. The data collected by the system during the test carried out in the real multi-resident scenario with the two older residents living at home were processed following the metrics proposed in From the number of detected events, the ALs and the corresponding DAIs are defined for each resident, shown in The number of events captured per location has been used to compute the DRA for each resident. To assess the robustness of our system, we measured its accuracy in detecting different types of movements and discriminating against residents, and we tested it in a real-life multi-resident scenario with two older people living in their home. We then analyzed the data collected during the monitoring of the two residents through the metrics proposed in During the supervised test, the multi-resident activity detection sensor network achieved an accuracy of 76% in discriminating the involved subjects. The results reported in The main aim of the test carried out in the real multi-resident scenario was the evaluation of the functionality of the multi-resident activity detection sensor network and the effectiveness of the proposed metrics in determining the AL and habits of older residents. The setup was trialed for 1 week. The collected data consisted of the events detected by the system in the three rooms where the PIR motion sensors were installed , the identity of the resident, and the time of event detection. The first metric considered in the assessment of the AL of the older residents is the number of detected events (PIR sensor activation) per day. Since the residents carry out their activities at different times of the day, the HAPs for the whole week have been calculated. Considering Information on residents’ activities during the day can also be evaluated through the heatmaps shown in The main contribution of this work is the presentation of an innovative multi-resident activity detection sensor network that uses the BLE signal emitted by tags worn by residents and PIR motion sensors deployed in the house to locate residents and monitor their activities. The advantage of this system is that it guarantees remote monitoring, is non-invasive, low-cost, and respectful of privacy. Furthermore, this system can be easily extended to larger environments and contexts with more residents. Initially designed to be installed in the home of older people, the sensor network can also be used in nursing homes where the monitoring of older residents can help caregivers understand their habits and daily activities. By integrating the system with more Raspberry Pi Zero W devices and PIR motion sensors, we can cover more areas, and by using additional tags, we can track more people. A drawback of using BLE technology is that radio waves are influenced by the presence of external factors such as walls, objects, etc. , which cause the RSSI to fluctuate. The further away the tag is from the Raspberry Pi Zero W scanner, the more unstable the RSSI is and the less accurate the detection becomes. Another drawback of using the wearable approach can be forgetting to wear the tag. In this case, the sensor network can still detect the movements of the users, but it is not able to associate them with the users’ identities.We demonstrated that this system solves the problem of monitoring older people and measuring their activities in multi-resident contexts through a test carried out in a real multi-resident scenario where two older people were monitored during their daily life. Finally, the metrics defined to analyze and interpret the collected data revealed a useful and innovative approach to understand daily habits and measure the AL of older people: the number of detected events (PIR activations) reflects the activity level of the older people, while the number of events captured per location reflects the amount of activity performed by the older people assigned to execute a specific ADL. By analyzing the results obtained, it is thus possible to define a behavioral model for the older people. The assessment of this model can help to capture possible changes resulting from cognitive or physical health.For future works, a case will be designed to house each PIR motion sensor and the related Raspberry Pi Zero W. Furthermore, the use of wireless PIR sensors instead of wired ones will be evaluated in order to reduce the number of Raspberry Pi Zero W devices by connecting more than one PIR to each of them, thus reducing costs and eliminating wiring. We found that the proposed sensor network can achieve high accuracy in discriminating the identity of residents by using the DT classifier; thus, further investigation regarding its real-time application will be carried out.


Using time-reversible Markov models is a very common practice in phylogenetic analysis,because although we expect many of their assumptions to be violated by empirical data,they provide high computational efficiency. However, these models lack the ability toinfer the root placement of the estimated phylogeny. In order to compensate for theinability of these models to root the tree, many researchers use external information suchas using outgroup taxa or additional assumptions such as molecular clocks. In this study,we investigate the utility of nonreversible models to root empirical phylogenies andintroduce a new bootstrap measure, the  The most widely used method for rooting trees in phylogenetics is the outgroup method.Although the use of an outgroup to root an unrooted phylogeny usually outperforms otherrooting methods , the maiIt is possible to infer the root of a tree without an outgroup using molecular clocks . A stricOther rooting methods rely on the distribution of branch lengths, including Midpoint Rooting, MinimalOther less common rooting methods that can be used in the absence of outgroup are: rooting bygene duplication , indel-bAll the methods mentioned above, apart from the molecular clock, infer the root positionindependently of the ML tree inference. The only existing approach to include root placementin the ML inference is the application of nonreversible models. Using nonreversiblesubstitution models relaxes the fundamental assumption of time-reversibility that exists inthe most widely used models in phylogenetic inference . This inRegardless of the rooting method and the underlying assumptions, it is crucial that we areable to estimate the statistical confidence we have in any particular placement of the root ona phylogeny. A number of previous studies have sensibly used ratio likelihood tests such asthe Shimodaira–Hasegawa (SH) test  and the There is strong empirical evidence that molecular evolutionary processes are rarelyreversible , but fewrootstrap support value, which estimates the extent to which the datasupport every possible branch as the placement of a root in a phylogenetic tree. Unlikeprevious studies that used Bayesian methods with nonreversible substitution models to inferrooted ML trees . This study showed thatIQ-TREE performs competitively with a new rooting tool, RootDigger. In most simulated datasets, IQ-TREE slightly outperformed RootDigger in terms of root placements, but no comparisonswere made between RootDigger and IQ-TREE on empirical data sets. Although RootDigger issignificantly faster than IQ-TREE (Bettisworth and Stamatakis 2020), the former is limited tonucleotide substitution models. Since we are interested in both nucleotide and amino acidnonreversible models, we used IQ-TREE for tree and root inference in this study.rootstrap support for each branch in the ML tree, as the proportion ofbootstrap trees that have the root on that branch. Since the root can be on any branch ina rooted tree, the rootstrap support values are computed for all the branches includingexternal branches. The sum of the rootstrap  edit 1 <inline-formula><tex-math notation="LaTeX"              id="ImEquation1"><![CDATA[$\quad$]]></tex-math></inline-formula>  support values alongthe tree are always smaller than or equal to one. A sum that is smaller than one can occurwhen one or more bootstrap replicates are rooted on a branch that does not occur in the MLtree  are bounded by 100%, as tips always appear in all thebootstrap trees.rootbranchlength error distance (rBED) and root split errordistance (rSED). Since the nonreversible model infers the exact position of theroot on a branch, we define the root branchlength error distance (rBED)as the range between the minimum and maximum distance between the inferred root positionand the “true root” branch. If the true root is on the same branch as the ML tree root,then rBED will be between 0 and the distance between the ML tree root and the farthestpoint on that branch (root split error distance (rSED) as the number of splitsbetween the ML root branch and the branch that is believed to contain the true root .If the true position of the root is known  or assumed , we can calculate additional measurements of theerror of the root placement. We introduce two such measurements here: t branch . Since rThe rootstrap, rBED, and rSED assess different aspects of the root placement. While therootstrap offers an indication of the support that the data have for a certain branch tobe the root branch, rBED and rSED provide an estimation to the accuracy of the method inestimating the exact root position if the root position is known or assumed in advance. Inother words, the rootstrap value is a measure for the robustness of the root placementgiven the model and the data and can be used on any data set regardless of whether thetrue root position is known, while rBED and rSED are measures of the accuracy of thenonreversible model to find the root placement given the data, and require the rootposition to be known or assumed in advance.Because nonreversible amino acid models require the estimation of a large number ofparameters, and because we suspected that the information in any such analysis on theplacement of the root branch of a tree might be rather limited, we searched for empiricaldata sets that met a number of stringent criteria: Existence of both DNA and amino acid multiple sequences alignments (MSA) for the sameloci.Genome-scale MSAs to ensure that the MSAs have as much information as possible withwhich to estimate the nonreversible models’ free parameters and the root position.Since we do not know the number of sites required to correctly infer the rooted MLtree, we define 100,000 sites as the minimum number of required sites. This alsoallows us to subsample the data set to explore the ability of smaller data sets toinfer root positions.Highly curated alignments: since the quality of the inferred phylogeny is highlydependent on the quality of the MSA , we focuExistence of several clades for which there is a very strong consensus regardingtheir root placement. Since we are interested in evaluating the performance ofnonreversible models to infer root placements in an empirical rather than a simulationcontext, we need to identify monophyletic subclades for which we can be almost certainabout their root position. This enables us to divide the data set into nonoverlappingsubclades for which we are willing to assume we know the root positions. Furthermore,we define the minimum number of taxa in each subdata set as five.https://figshare.com/s/622e9e0a156e5233944b) under the name “Wu_2018_aa”and “Wu_2018_dna” for the amino-acid and nucleotide alignments, respectively.We initially identified a number of genome-scale data sets that contained largenumbers of nucleotide and amino acid MSAs. In many cases, it was difficult todetermine whether these alignments had been rigorously curated, and even morechallenging to find data sets for which the root position of a number of subcladescould be assumed with confidence. The only data set that met all of our criteria was adata set of placental mammals with 78 ingroup taxa and 3,050,199 amino acids . This dawell-defined clades” that fulfilled all of thefollowing criteria: Since our main objective in this study is to evaluate the effectiveness of nonreversiblemodels and the rootstrap value in estimating and measuring the support for a given rootplacement on empirical data sets, we must identify a collection of subclades of the largermammal data set for which it is reasonable to assume a root position. We acknowledge, ofcourse, that outside a simulation framework it is not possible to be certain of the rootposition of a clade. Nevertheless, it is possible to identify clades for which theposition of the root is well supported and noncontroversial, thus minimizing the chancesthat the assumption of a particular root position is incorrect. To achieve this, weanalyzed the root position of each order and superorder in the data set, and defined“(1)It contains at least five taxa. This ensures that the probability of obtaining arandom ML rooted tree to be at most 0.95%. For clades with four taxa, there are 15different rooted topologies, and therefore a 6.7% probability to get any particularroot position by chance. On the other hand, for clades with at least five taxa, thereare at least 105 different rooted topologies and a maximum probability of 0.95% torandomly get a particular root position by chance.(2)Supplementary Appendix Fig.SA.1 available on Dryad at https://doi.org/10.5061/dryad.fj6q573rx). This requirement ensures thatthere is strong support in the data set for the root position of the clade when theentire data set is rooted with an outgroup.The bootstrap support for the branch leading to that clade in the phylogenetic treecalculated from the whole data set is 100%: since the bootstrap value indicates thesupport the data have for a certain branch, we also require 100% support for the firstdirect descendants in the clade ((3)The site concordance factor (sCF) for the first direct descendants in the clade issignificantly greater than 33%. The sCF is calculated by comparing the support of eachsite in the alignment for the different arrangements of the quartet around a certainbranch. In other words, an sCF of 33% means equal support for any of the possiblearrangements. Therefore, we require that the sCF of the deepest two levels of branchesleading to that clade is significantly greater than 33%. Moreover, we require that thegene Concordance Factor (gCF) for the first direct descendants in the clade to besignificantly greater than 33% of the sum of the gene concordance factor and the twoDiscordance Factors (gDF1 and gDF2). The gCF of a branch is calculated as theproportion of gene trees containing that branch, and gDFs are calculated as theproportion of gene trees containing one of the two other resolutions of that branch.Since for each branch in a bifurcating tree, there are three possible arrangements ofclades around that branch, we ignore all gene trees that do not contain one of thesearrangements .Although there is no threshold regarding the required proportion of genes concordantwith a certain branch, for convenience, we define branches with gCF significantlygreater than 33% of the sum gCFlignment . To test(4)Supplementarymaterial available on Dryad).At least 95% of the studies that have been published in the last decade support thisclade: we searched google scholar for all published papers since 2009 that determinethe root of the addressed clade. We then checked if at least 95% of those papers agreethat the root position of the clade matches that in the ML tree we estimate from thewhole data set  and nucleotide (NR-DNA) sequences . This apTo assess the performance of the rootstrap and the ability of nonreversible models toestimate the root of the trees on smaller data sets, we also repeated every analysis onsubsamples of the complete data set. For each well-defined clade, we performed analysis onthe complete data set (100%) as well as data sets with 10%, 1%, and 0.1% of randomlyselected loci from the original alignment.rootbranches confidence set as the set of root branches that are not rejected infavor of the ML root placement. An important difference between the AU test and therootstrap support is that the AU test is conditioned on a single ML tree topology, but therootstrap support is not. Because of this, they provide quite different information aboutthe position of the root. The AU test assumes that the ML tree topology is true, and thenseeks to determine the confidence set of root placements conditioned on that topology. Theconfidence set for the AU test will always therefore contain at least the ML root branch.The rootstrap does not assume any particular topology and instead asks how many times aparticular root position appears across a set of bootstrap replicates. Because of this, itis possible for every branch in the ML topology to receive 0% rootstrap support. This canoccur if none of the branches in the ML topology appear as the root branch in any of thebootstrap topologies.In addition to the rootstrap support, we calculate the confidence set of all the branchesthat may contain the root of the ML tree using the approximately unbiased (AU) test . To do tAs it is common in many phylogenetic analyses to remove third codon positions from thealignment , we wantMoreover, although the NR-AA and NR-DNA models relax the reversibility assumption, theystill assume stationarity and homogeneity. To reduce the systematic bias produced byviolating these assumptions, we used the MaxSym test  to removIn addition to the well-defined clades, we used the methods we propose here to infer theroot of two clades of mammals whose root position is controversial; Chiroptera and theCetartiodactyla.There is a controversy around the root of the Chiroptera (bats) in literature. The twomost popular hypotheses are: 1) the Microchiroptera-Megachiroptera hypothesis; where theroot is placed between the Megachiroptera, which contains the family Pteropodidae, and theMicrochiroptera, which contains all the remaining Chiroptera families. This hypothesis iswell supported in the literature . HoweverSimilar to Chiroptera, the root of Cetartiodactyla remains contentious in the literature.The three main hypotheses regarding the root of Cetartiodactyla are: 1) Tylopoda as thesister group for all other cetartiodactylans; 2) Suina as the sister group for all othercetartiodactylans; 3) the monophyletic clade containing Tylopoda and Suina as the sistergroup for all other cetartiodactylans.To ascertain whether certain sites or loci had very strong effects on the placement ofthe root we follow the approach of Supplementary Appendix Figs. SA.2,SA.3 and Table SA.2 available onDryad) are consistent with the published tree (Supplementary Appendix TablesSA.1 and SA.2 available on Dryad). Trees in Newick format can be found ongithub: https://github.com/suhanaser/Rootstrap/tree/master/trees.The trees inferred from the whole data sets with the nucleotide-reversible model and theamino-acid-reversible model (hed tree . Five clSupplementary Appendix TableSA.3 available on Dryad). Moreover, for all the five clades, the true rootwas the only root placement in the confidence set of the AU test. The average running timeof the NR-AA model . Moreover, in three of five well-definedclades, 1% of the sites  was enough to give a very highrootstrap support value for the assumed correct root placement. Using only 0.1% of thesites  decreased the rootstrap support value noticeably inall data sets (Supplementary Appendix Table SA.3 available on Dryad). These values areshown for each data set in Figure 3, where the X-axis is plotted in terms ofparsimony-informative sites to allow for a more direct comparison between data sets, andto assist those applying these methods in deciding whether to use them on their own data.Although the rootstrap support for the true root improves as the number ofparsimony-informative sites increase, in some data sets  this is not the case .pt Our results show that using only 10% of the sites in the amino acid alignments  still gave very high rootstrap support values (>98%) forfour of the five well-defined clades  with noThe nonreversible amino acid models were strongly preferred to the reversible models onthe complete data sets , and for the data sets with 10% of loci subsampled , but the opposite was true forthe data sets 1% and 0.1% of the loci subsampled .We correctly inferred the root for four out of the five nucleotide data sets with theNR-DNA model, when all loci were used. However, the rootstrap support was generally lowerthan in the amino-acid data sets . Similar to amino-acid data sets, there is no correlation between rSEDand rBED and the size of the data set (Supplementary Table SA.4 available on Dryad). Theaverage running time of the NR-DNA model . In fact, the BIC values of the NR-DNA models are always worse thanreversible models regardless of the size of the nucleotide data set except for threeclades when all loci were included (Supplementary Table SA.8 available on Dryad). In twoof the data sets (Myomorpha and Primates) where the NR-DNA model was better than thereversible model, the root placement was inferred correctly with high rootstrap support(>95%). In fact, the Afrotheria nucleotide data set is the only data set in which thenonreversible model was better than the reversible model but the root placement wasinferred incorrectly.Our results show that removing the third codon positions does not improve the rootstrapsupport value. In contrast, in some data sets removing third codon positions decreased therootstrap support value and increased the rSED .As expected, our results show that removing loci that fail the MaxSym test improves therootstrap support values when the rootstrap support value was less than 100% and/or theroot placement was inferred incorrectly, as the case in some nucleotide data sets .Using the whole amino acid data set, our results show 65.5% rootstrap support for theYinpterochiroptera–Yangochiroptera hypothesis and 23.2% for theMicrochiroptera–Megachiroptera hypothesis. The remaining11.3% of the rootstrap supportgoes to supporting the branch leading to Rhinolophoidea as root branch of the bats . RemovinUsing the NR-DNA model gives 100% rootstrap support for the Yinptero-Yango hypothesis,and we can confidently reject the Micro-Mega hypothesis in favor of the Yinptero-Yangohypothesis using the AU test . Yet,removing nucleotide loci that fail the MaxSym test . Moreover, the rootstrap support value for the Yinptero-Yango hypothesis increasesand the rootstrap support value for the Micro-Mega hypothesis decreases as moreparsimony-informative sites are added to the alignment, for both nucleotide and amino aciddata sets  and Suina as the sister groupto all other cetartiodactylans with 71.8% rootstrap support . Yet, ThRemoving the amino acid loci that failed the MaxSym test  and that the rBED requires the rootingmethod to be able to accurately place the root in a specific position of the rootbranch.In addition to the rootstrap support value, we introduced another two metrics; the rootbranch-length error distance (rBED), and the In this study, we used these and other methods to assess the utility of nonreversiblemodels to root phylogenetic trees in a ML framework. We focused on applying these methods toa large and very well curated phylogenomic data set of mammals, as the mammal phylogenyprovides perhaps the best opportunity to find clades for which the root position is knownwith some confidence. As expected, our results show an exponential increase in the rootstrapsupport for the true root as we add more information to the MSA. Our results suggest thatnonreversible amino-acid models are more useful for inferring root positions thannonreversible DNA models. One explanation for this difference between the NR-DNA and theNR-AA models is the bigger character-state space of the NR-AA models. These models have 400parameters (380 rate parameters and 20 amino acid frequencies) whereas NR-DNA models haveonly 16 parameters (12 rate parameters and 4 nucleotide frequencies). This could allow theNR-AA model to capture the evolutionary process better than the NR-DNA model, potentiallyproviding more information on the root position of the phylogeny. This hypothesis requiressome further exploration though, and we note that the actual character-space of amino acidsis much smaller than accommodated in NR-DNA models due to functional constraints on proteinstructure .Another explanation for the difference in performance between the NR-AA and NR-DNA modelsis that higher compositional heterogeneity in nucleotide data sets may bias tree inference.The fact that each amino acid can be specified by more than one codon, and that synonymoussubstitutions are more frequent than nonsynonymous substitutions, makes amino acid data setsless compositionally heterogeneous than nucleotide data sets. In principle, this bias can bealleviated by removing loci that violate the stationarity and homogeneity assumptions . Our resWe used the nonreversible approach to rooting trees along with the rootstrap support toassess the evidence for different root placements in the Chiroptera and Cetartiodactyla.Using the amino acid data sets we found that in both cases, although there tended to behigher rootstrap support for one hypothesis, neither of the current hypotheses for eitherdata set could be rejected. These results emphasize the importance of the rootstrap supportvalue as a measure of the robustness of the root estimate given the data. In both theChiroptera and Cetartiodactyla data sets the root placement varied among subsamples of thedata set, and the rootstrap support reflects this uncertainty. However, checking thestability of root placement estimate by randomly subsampling from the whole Chiroptera dataset show an obvious trend towards the Yinpterochiroptera–Yangochiroptera hypothesis as thedata set increases in size. This trend is consistent with a small number of influentialsites or loci having their signal progressively drowned out in favor of theYinpterochiroptera–Yangochiroptera hypothesis as more data are added to the alignment. Inboth the Chiroptera and Cetartiodactyla cases, the amino acid data is inadequate todistinguish between certain root placements. On the other hand, in both the Chiroptera andCetartiodactyla, the nucleotide data sets appear to show stronger support for a single rootplacement.Comparing BIC scores of reversible and nonreversible models show that in most of thenucleotide data sets the reversible model was a much better fit to the data than the NR-DNAmodel. This is likely due to the limitations of the method we used to infer the NR-DNAmodel. Specifically, when inferring the trees with reversible DNA models, we used apartitioned model such that each partition was able to have an independent DNA substitutionmodel. On the other hand, when we inferred the NR-DNA model we estimated a single model forthe entire alignment. Thus, the NR-DNA model we inferred was unable to account forheterogeneity in the evolutionary process among partitions, possibly leading to its worsefit to the data when assessed using BIC scores. This suggests that using either mixturemodels or partitioned models may improve the fit of nonreversible DNA models to the data.The DNA results are consistent with results from previous study using the NR-DNA model andRootDigger , althougOur results demonstrate that the amino-acid nonreversible model can often be surprisinglyaccurate for inferring the root placement of phylogenies in the absence of additionalinformation (such as outgroups) or assumptions (such as molecular clocks). In all of thewell-defined clades that we examined, the nonreversible amino-acid model successfullyidentified the root that we identified a priori as correct, and with very high rootstrapsupport. Importantly, the nonreversible amino-acid models also tended to fit the data farbetter than their reversible counterparts did. Indeed, we show that root placements appearto be accurate even with data sets as small as 50 well-curated loci between fairly closelyrelated taxa such as orders of mammals. Nevertheless, the application of the nonreversibleamino acid models to two clades where the root position has previously been contentiousfailed to shed much additional light on the true root placement. Thus, while we show thatthe use of nonreversible models certainly has promise, we also show that it is no silverbullet.Where a reliable outgroup taxon can be found, without the issues that can confound theinference of root placements using outgroups , we suggOur work suggests a practical approach to inferring the root of a phylogenetic tree usingnonreversible models. First, estimate an unrooted tree topology using the best reversiblemodels available, excluding outgroup sequences. Next, fix the tree topology and use the bestnonreversible models available to infer the ML root position of that tree. Finally,determine to what extent the ML root position should be trusted. The degree of trust thatresearchers should put in an inferred ML root position should be influenced by three factors. First, the fit of the nonreversible model to the data should be betterthan the fit of the reversible model. This can be assessed using common criteria like AICcor BIC scores. A better fit of the nonreversible model provides some assurance that the datacontain sufficient signal that using a nonreversible model is advisable in the first place.Our results show that the root placement was inferred correctly with high rootstrap supportin 12 out of the 13 data sets in which the nonreversible model was preferable. In theabsence of a better fit for a nonreversible model, we do not think any inferred ML rootposition should be trusted. Second, root positions with higher rootstrap support should betrusted more, because a higher rootstrap support indicates less variance among sites in thesignal for the placement of the root. Third, ML root positions should be trusted more whenthe number of root placements included in the confidence set of an AU test is small, becausea smaller confidence set indicates that there is less uncertainty in the root placement whenthe analysis is conditioned on the full alignment and the unrooted ML tree topology. Aconservative approach to inferring root placements with nonreversible models would be toconsider any root placement that has a substantial fraction of the rootstrap support and/oris included in the set of possible root placements identified by the AU test as a possibleroot placement given the assumptions of the model.We hope that the combination of nonreversible models, rootstrap support, and AU tests willadd another tool to the phylogeneticist’s arsenal when it comes to inferring rootedphylogenies.http://www.iqtree.org/doc/Rootstrap. In addition, a python script is availableat https://github.com/suhanaser/Rootstrap.Availability and implementation: rootstrap support is implemented in IQ-TREE 2 and atutorial is available at the iqtree webpage