The Narrowband Internet of Things (NB-IoT) is a very promising licensed Internet of things (IoT) technology for accommodating massive device connections in 5G and beyond. To enable network scalability, this study proposes a two-layers novel mixed approach that aims not only to create an efficient spectrum sharing among the many NB-IoT devices but also provides an energy-efficient network. On one layer, the approach uses an Adaptive Frequency Hopping Spread Spectrum (AFHSS) technique that uses a lightweight and secure pseudo-random sequence to exploit the channel diversity, to mitigate inter-link and cross-technology interference. On the second layer, the approach consists of a clustering and network coding (data aggregation) approach based on an energy-signal strength mixed gradient. The second layer contributes to offload the BS, allows for energy-efficient network scalability, helps balance the energy consumption of the network, and enhances the overall network lifetime. The proposed mixed strategy algorithm is modelled and simulated using the Matrix Laboratory (MATLAB) Long Term Evolution (LTE) toolbox. The obtained results reveal that the proposed mixed approach enhances network scalability while improving energy efficiency, transmission reliability, and network lifetime when compared to the existing spread spectrum only, nodes clustering only, and mixed approach with no network coding approaches. The Internet of Things (IoT) has changed the way we view and use technology. As consumers, we have come to expect connectivity and access to information wherever we go. According to Information Handling Services (IHS) technology forecast, the Internet of Things (IoT) market is expected to grow to billions of devices by 2020 . MassiveThe NB-IoT is categorized as one of the licensed Low-Power Wide-Area Networks (LPWAN) cellular technologies based on cellular Technologies such as the Long-Term Evolution (LTE) and the 5th generation technology (5G) with long-range and low cost. In the LPWAN category, exists other licensed technologies, i.e., Long-Term Evolution Category M1 (LTE-M), and unlicensed technologies, i.e., Long Range (LoRa), SigFox, Ingenu, etc. ,4,5,6,7,The NB-IoT is one of the most prominent IoT technology within the licensed band. Despite all the challenges faced by NB-IoT such as energy efficiency because NB-IoT nodes run on batteries, coverage range, reliability of Uplink (UL) and Downlink (DL) communications, etc, there is still more and more devices being added to the network as the range of applications and needs keeps increasing. This poses a serious challenge when it comes to the NB-IoT network design considerations in terms of its different layers. This work focuses on proposing a novel spread spectrum and clustering combination approach as a mechanism to enhance network scalability of the NB-IoT. The challenge, however, remains to ensure that despite the use of a new mixed approach, the NB-IoT network still maintains an acceptable level of energy efficiency, network reliability, and data rates. Several clustering techniques for wireless IoT exist, however to the best of our knowledge, none simultaneously considers signal strength and remaining energy of nodes as a combined parameter for cluster head selection.The Frequency Hopping Spread Spectrum remains one of the most reliable and ruggedized options for modern Internet of Things communications. Therefore, despite the rapidly changing technology landscape, the FHSS remains a good choice for long-range communication and has the potential to accommodate the growing number of devices in the field thanks to its ability to efficiently and dynamically share network spectral capacity among devices. This work, therefore, proposes a new FHSS-based spread spectrum approach, adapted to the NB-IoT PHY layer characteristics, which, when coupled with a novel clustering approach, aims at enhancing the NB-IoT network scalability while preserving its energy efficiency and network reliability.Due to the limited available spectrum (roughly 200 kHz), as allocated to the NB-IoT within its three deployment modes ; one of the key issues is how to efficiently use these resources to support massive IoT devices. Because the NB-IoT technology is deemed to have a higher level of computational complexity at Base Station as compared to its unlicensed band competitors such as the LoRa and Sigfox; the concepts of time offset as well as transmission repetition have been proposed by previous work such as ,13. FinaThe above three reasons coupled with the fact that most NB-IoT technology designs, as currently proposed, mainly use simple ALOHA or CSMA based MAC protocols which do not scale well with a large number of connected devices; have been considered by the present study as a trigger for a need for a novel spread spectrum and clustering approach for the NB-IoT. Therefore, the present work proposes a mixed-strategy of frequency hopping spread spectrum and clustering approach to mitigate the inter-cell interference and reduce computational complexity at the BS. This is expected to enhance NB-IoT system designs as a way to enable network scalability, data rate enhancement, and also maintain energy efficiency and network reliability.The rest of this paper is organized as follows. The sharing of network resources remains a very crucial aspect of NB-IoT network designs. It is a key feature for enhancing network scalability and enabling the connection of massive NB-IoT devices to the network. Various approaches have been previously proposed among which tone allocations , PhysicaSpectrum sharing is a preferable solution for IoT due to the scarcity of available spectrum resources. In a very special manner, mobile operators are inclined to exploit the existing standards and infrastructures of current cellular networks and deploy licensed IoT such as the NB-IoT within the already congested licensed cellular spectrum . The reaThis study considers the use of a spread spectrum approach as part of the orthogonal frequency sharing techniques because of the nature of the NB-IoT spectrum which does not only comprise of the spectrum as provided by the cellular network to which it is attached but also comprises of other re-farmed cellular spectra. This characteristic provides an opportunity to mitigate interference and therefore, allows us to connect more NB-IoT nodes within a cell. This is possible provided that a smart and well-designed frequency selection algorithm is used. This is one of the objectives pursued by the present study.Other spectrum sharing techniques used for NB-IoT systems include spectrum sharing techniques similar to those in cognitive radio networks. In most of these cognitive approaches, the licensed IoT networks are considered as secondary cognitive transmitters while the primary transmitters are the UEs of the cellular network to which the IoT network is attached ,21. In tIt is expected that the ever-expanding number of connected NB-IoT devices will generate a massive volume of data, which has diverse requirements including network lifetime (≥10 years ), latenc−12.6 dB ), and th−12.6 dB ). Some o−12.6 dB . This is−12.6 dB . As moreClustering techniques have proven to be beneficial in terms of enhancing the energy efficiency of the Internet of Things networks. They have also proven to be very effective in balancing the energy consumption of the network. This is because they allow an offload of the Bases Station which, despite the increasing number of nodes, results in more reliable communications. However, one of the major challenges of clustering has been the fact that the use of fixed cluster heads has proven to shorten the life cycle of the network. This is because most NB-IoT nodes are energy limited as they run on batteries, Therefore, using some selected NB-IoT nodes as relays to a group of others poses a serious energy burden to these which eventually die. As the later die, this results in the inability of some NB-IoT nodes to reach the BS. This problem, also often qualified as a “hotspot problem” has triggered various research interests within the community of scientists. The present section takes us through the different clustering approaches as applied to the specific case of NB-IoT systems in an attempt to resolve this issue.Several energy-efficient techniques have been developed for IoT networks in general and NB-IoT networks in particular. Some of the key techniques include but are not limited to the following,P rounds, where P is the desired percentage of cluster heads. Thereafter, each node has a Low-Energy Adaptive Clustering Hierarchy (LEACH) and its variants: LEACH is s a hierarchical technique that consists of nodes transmitting data packets to cluster heads which first aggregate the received packets, compress the aggregated data before forwarding it to the base station (sink). In LEACH, every single node uses a stochastic algorithm at each transmission round to determine if it is its turn to become a cluster head. Nodes that have been cluster heads cannot become cluster heads again for protocol . The LEAprotocol . DF-LEACprotocol . In thisThe Energy-Efficient Hierarchical Clustering (EEHC): This clustering approach as proposed by  uses VorEnergy-Efficient Unequal Clustering (EEUC): Authors in  propose Power-Efficient and Adaptive Clustering Hierarchy (PEACH): PEACH is proposed by  to solveHybrid Energy-Efficient Distributed Clustering (HEED): The HEED (Hybrid Energy-Efficient Distributed clustering) protocol periodically selects CHs based on a hybrid parameter made of the node remaining energy as primary parameter and a secondary parameter, such as minimum average distance to a higher number of neighbor nodes. The HEED demonstrates a fairly uniform CH distribution across the network and a low computational complexity  process . The HEEDistributed Weight-based Energy-Efficient Hierarchical Clustering (DWEHC): The authors  propose The fact that the BS has to collect information from all the NB-IoT nodes, results in complex and multiple overheads. This results in additional energy consumption which although it is negligible for a few nodes, becomes significant as the number of nodes increases.The approach does not cater to the energy consumption caused by the distance measurement between NB-IoT nodes. This is because the CH selection parameter does not feature any signal strength consideration.Cluster Head Election using Fuzzy logic (CHEF): CHEF  proposesSeveral approaches have been previously considered to achieve energy efficiency within an NB-IoT system at various network layers . One sucSenseCode: This is a local network coding approach as proposed by  that focCodeDrip: This is a technique proposed by  to achieGiven the different identified issues as identified for the various existing approaches as described in the present section, this paper proposes a mixed approach strategy that first tackles the inter-cluster communication scalability issues through the use of an adaptive frequency hopping spread spectrum capable to mitigate interlink interference. Secondly, the proposed approach also proposes an energy-signal-strength-aware CHs selection method with the intent to reduce and balance the overall energy efficiency of the network while offloading the BS. The detailed explanation of this proposed mixed approach is detailed in the next section.T which varies depending on the number of NB-IoT nodes deployed in the cell. The random sequence is made known to other NB-IoT nodes as well as to the Base Station (BS) through an initial broadcast of interest packets. Assuming that only static NB-IoT nodes are considered, this broadcast of interest messages is performed during the initial network deployment. Because of the adaptive nature of the proposed cluster head selection algorithm and the need, therefore, to swap CHs, the broadcasting of interest messages also happens periodically intending to reconfigure the network by selecting new CHs based on a residual energy and signal strength mixed gradient.The study was based on the hypothesis that the proposed intelligent mixed approach is capable of switching a common carrier signal among many frequency channels, using a lightweight and secure pseudo-random sequence of a finite period The random sequence at a node must be known by neighboring nodes because of the multi-hop cooperative clustering aspect of the proposed method which dictates that some NB-IoT devices within the same cluster are selected to play a relay role to offload the BS. This is expected to reduce the transmission traffic, enhance the Up-link throughput and consequently the up-link data rate, mitigating the inter-cell interference, reducing the number of re-transmission attempts, therefore, enhancing network reliability and energy efficiency.It is also very important that the pseudo-random sequence generator algorithm is lightweight because of the limited computational, memory but also energy resources of the NB-IoT nodes . TypicalIn summary, this study was based on the hypothesis that the adaptive frequency hopping spread spectrum technique using a lightweight pseudo-random sequence generator, coupled with a cooperative adaptive clustering approach can result in an efficient way to offload the Base Station (BS), and therefore, allows for more NB-IoT nodes to be handled within a single cell. With an efficient way of grouping NB-IoT nodes within a cell and an appropriate cluster head selection based on a composite residual-energy aware and signal strength gradient, in smaller, there is a possibility for better handling of noise and interference mitigation within the cell. This is the hypothesis that this study verifies to primarily enhance network scalability of the NB-IoT network without degrading energy-efficiency performance.j NB-IoT nodes join the network, they send interest messages to the BS. The BS records the Received Signal Strength Indicators (RSSI) of all the j nodes and sorts them in ascending order. The lowest RSSI value which represents the NB-IoT node with the lowest level of access to the BS while the NB-IoT node with the highest RSSI represents the node with the highest access to the BS. The RSSIs of the j first deployed nodes in their order of arrival form the initial seed of a lagged Fibonacci generator (LFG) series that is used to generate the pseudo-random number sequence. The generated sequence is then used for frequency channel hopping access.As the first T of all nodes within the network, the number of elements of the initial seed j is selected as per the following considerations:A random number generator’s period is the number of new random values that it generates before returning to the start of the sequence.T.The period of the pseudo-random number generated is equal to the number of NB-IoT nodes Maximizing the period ensures that the random numbers generated for each node are from a different period of the LFG.j is selected to provide an optimal balance between the random number generation period and the amount of memory used. This is because the higher the period T, the higher the consumption of the already constrained memory space of the NB-IoT nodes.The value Given a number of j and k are the lag offsets of the past samples in such a way that the trinomial The proposed algorithm uses a multiplicative LFG (MLFG). For an NB-IoT node with T, an initial seed size of j is chosen such as T. The MLFG pseudo-random values For a number of NB-IoT nodes j seed elements on the other hand are selected by the BS upon reception of the interest messages from all the NB-IoT nodes during the initial deployment based on the computed and sorted RSSI values of the received interest messages.The The received RSSI values are computed as follows,d and f represent the path loss of the signal energy, the distance between the NB-IoT node and the BS and the transmission frequency being used respectively.Through the analysis of the RSSI, it is found that the RSSI value decreases as the distance from the NB-IoT node to the BS increases. Based on this consideration, the free space model (FSM) is used as the RSSI-Based Ranging Model. The FSM is expressed as follows,Furthermore, it is also found that the path loss decreases as the antenna height increases. Based on this consideration, a two-way ground model is proposed as follows,In this study, a two-way ground level model is considered with an average LTE BS effective height of 27.81 m  and an acenarios .Considering the NB-IoT antenna gain of 6 dBi , and a tThe obtained RSSI values are then sorted as follows,The initial seed is distributed as part of the downlink broadcast packet to all the NB-IoT nodes. The initial seed is used to form the MLFG sequence that is used by the selected CHs for dynamic frequency hoping and channel selection within the available 200 kHz narrowband bandwidth either deployed in standalone, in-band or guard-band mode. This dynamic frequency hoping and channel selection is performed by CHs as they relay aggregated data packets received from the respective member nodes (MNs) within their clusters to the BS. The proposed dynamic frequency hopping spread spectrum approach is used to mitigate the interlink interference within the cell as the various nodes attempt to reach the BS. It is further supplemented by the clustering approach as described and boosts the overall network security, energy efficiency and robustness of the network but most important, enhances the possibility for more nodes to join the network without compromising its overall network performance.This section describes the proposed cluster groups formation as well as the energy and RSSI aware CHs selection process. The BS in cooperation with two NB-IoT nodes with reliable source powers are equipped with GPS devices and are used as anchor nodes for localization of the other NB-IoT nodes within the cell as follows. The coordinates of the two selected anchor nodes and the BS are known and modelled as The solution of Equation  leads toAfter the geographical localization of nodes within the cell, the BS through its downlink reply packet does not only distribute the initial adaptive spread spectrum frequency hopping seed, it also sends information that geographically groups NB-IoT nodes that are close to each other into clusters. A typical illustration of a typical cluster formation within an NB-IoT cell is shown in The following steps are taken for clusters formation and CH selection.j nodes within the cell as modelled in [-It is observed that in an adaptive clusters formation and CHs selection, the process should be dynamic to ensure not only energy efficiency but also energy consumption balance. Because, as the overall average remaining energy efficiency reduces, the number of clusters should be reduced. This means much bigger clusters should be formed to ensure more energy balance and make sure some CHs do not die faster than others. It is also true that the bigger the transmission range threshold used for clusters formation, the smaller the number clusters. This means that the average residual energy is inversely proportional to the transmission range threshold -On the other hand, the higher the average RSSI value, the more eligible CHs as the more nodes can directly reach the BS. This also translates to a smaller number of clusters which means the higher the transmission range threshold. This means that the average RSSI value is directly proportional to the maximum transmission range threshold.Step 1: Determining the neighbor nodes of each node within the cell by the use of predefined transmission range threshold  problem  in energelled in  and Pt,PThe above considerations, lead to a maximum transmission range N, the set of neighbour nodes of node n denoted as b and n. This approach comes with the advantage that it avoids the chaining issue.Our proposed approach opts for the use of the threshold transmission distance to form the different network clusters. Considering the set of all nodes denoted as n, all the distances with the neighbors in its cluster is computed as followsStep 2: For every node i denoted as n belonging to cluster i.Step 3: The average residual energy of each cluster i, the BS computes as a mixed residual energy and signal strength aware cluster head selection gradient n in cluster i and n in cluster i with respect to the BS. i based on the following criteria,Step 4: For each formed cluster Equation . On the i  packet A is obtained as follows,Step 6: Each time a CH receives data packets form various member nodes in its cluster, it applies Random Linear Network Coding (RLNC) (data aggregation) before transmitting it to the BS. In the typical two-hops clustering approach as considered by this study, the focus of this study is mainly on the global network coding approaches as the data aggregation point is the CH after which the aggregated data is immediately forwarded to the BS where its decoding occurs. Furthermore, the proposed network coding approach is an intersession network coding scheme in which data packets form various member nodes within the network are aggregated together before being transmitted as a single packet to the BS. The m packets to occur. The decoding process on the BS side is performed by forming a m encoded packets and by reducing it using the row-echelon form. Considering that the linear independence of encoded packets, the set of m linearly independent equations is solved to retrieve each originally encoded packet. One key advantage of this proposed RLNC approach is that the choice of encoding coefficients form a GF, enhances the probability of linear independence among the encoded packets which then translates in high decoding reliability.The key challenge associated with the network coding technique is that the decoding of the encoded packet is performed by Gaussian elimination and, therefore, requires the reception of at least The combination of the proposed dynamic frequency hoping and the energy-RSSI aware clustering with network coding approaches form the mixed proposed strategy for enhancing the overall network scalability, energy efficiency and latency of the network. A R2019b MATLAB version has been used to implement and evaluate the performance of the proposed NB-IoT Mixed Approach with Network Coding (N-MANC) in comparison the NB-IoT with the adaptive frequency hopping spread spectrum at PHY layer only (N-AFHSS) , then wiThis section presents the performance evaluation process in terms of the simulation parameter considerations, the simulation scenarios as well as the obtained performance results and their discussion.The performance evaluation of the proposed N-MANC approach versus the N-AFHSS, the N-LEACH and the N-MANNC is performed under the network considerations as summarized in It is also important to note that eNodeBs of the LTE network are often designed to handle QPSK, 16 QAM and 64 QAM modulations schemes for the Downlink (DL) communications. However, due to the limited computational ability of the NB-IoT node, often micro-controller based in the majority of IoT applications such as smart water, electricity metering etc.), the modulation scheme of choice is the QPSK as specified in The realistic case scenario considered in our study is one of the smart water metering applications. As we know, the water meter, unlike the other utility meters, is very energy-critical because of the regulation in many countries like South Africa which forbids to bring electrical lines on a water utility. Most water supply companies request a guaranteed minimum of 10 years lifetime of smart water meter design. The smart water meters are fixed sensor nodes and the ones considered for this research are assumed to have a Tunnel Magneto Resistance (TMR) active magnetic sensor to detect and convert the turning of the mechanical register. Every turning cycle of the mechanical register corresponds to a well-defined water flow value and results in a digital pulse transition on the output of the TMR sensor. The digital processor of the smart water meter counts the number of digital pulses it receives. The water volume flow is computed based on the total number of counted TMR digital pulses. The smart water meter is also able to detect any tampering activities. There is an NB-IoT radio module in each smart water meter and the assumption that there is one smart water meter per household. The choice of 1000 to 150,000 NB-IoT nodes represents the number of smart water meters and other battery-powered environment monitoring smart devices such as geyser leak detectors, door sensors, room occupancy detection sensors, fire detection sensors, water tank level monitoring systems, smart fridges, smart TVs etc. The set of nodes considered in this study are normally static sensors and limited in terms of energy resources. With the increasing number of IoT devices applications, this study caters for up to 150,000 nodes within the geographical area of an NB-IoT cell. In terms of the data traffic, data packets are sent on an average of 1 packet per day by these devices. Hence our consideration of up to 3960 rounds of transmission packets for 11 years of operation. The daily data packet sent as an uplink packet by each smart water meter contains information about the device’s status . The other environment monitoring smart devices are also assumed to send their daily status once a day either in the case of a triggering event  or around midnight for a summary daily report. The traffic data model used in our simulation is therefore made up of a combination of deterministic daily packet transmissions coupled with a uniformly distributed stochastic event-triggered packet transmission. In essence, every single node has a minimum of 1 packet per day.This subsection presents the obtained performance evaluation results for the various considered simulation scenarios.P) are captured in First, the average energy consumption performance of the proposed N-MANC mixed approach as the network scalability is progressively increased, is assessed with respect to the N-AFHSS, N-LEACH and N-MANNC. The obtained results for a single communication period . This average rate of successful transmissions is computed as the various number of nodes per cell is increased to assess the impact of the network scalability on the network transmission reliability of the network. The obtained results are illustrated in It can be observed from the bar graph in It was also crucial to assess the contribution of the proposed approach towards enhancing the network’s lifetime. To assess the network lifetime performance, the number of simulation runs have been performed to compute the number of remaining active nodes as the rounds of packet transmissions is increased to evaluate at what rate the network dies when the proposed N-MANC approach is used as compared to the N-LEACH, N-MANNC and the N-AFHSS. The obtained results are shown in It can be observed in It can also be noticed in The results in The results in A novel NB-IoT Mixed spread spectrum and clustering Approach with Network Coding (N-MANC) is presented in this research article to enhance the NB-IoT network scalability while improving its energy efficiency, network reliability, and network lifetime performance. The N-MANC is proposed as a step towards enabling the massive connections of IoT objects to the cellular network (5G and beyond) within the licensed band. After establishing through literature survey that this enhanced scalability of the NB-IoT network is a cross-layers effort, the present research work has proposed a mixed approach which at the Physical layer (PHY) proposes an adaptive frequency hopping spread spectrum technique using a lightweight pseudo-random sequence generator coupled with the clustering and network coding techniques at the network layer. The proposed approach is described in detail in terms of its implementation and underlying concepts.After evaluation of the proposed approach, the obtained and discussed results lad to the conclusion that the proposed N-MANC approach helps mitigates the scarcity of the available spectrum coupled with the interlink interference to enable more NB-IoT nodes to join the network cell while ensuring an offloading of the BS using an energy and RSSI aware formation of network clusters and selection of CHs. It can be further concluded that the offloading of the BS together with the aggregation of data packets at each CH using the proposed RLNC technique, has demonstrated the ability to significantly improve the energy efficiency, network reliability and network lifetime of the network as compared to the N-AFHSS (with adaptive frequency hopping spread spectrum only), the N-LEACH (with energy-efficient clustering only), and N-MANNC (Mixed approach with no network coding) respectively.The proposed NB-IoT Mixed spread spectrum frequency hopping and clustering approach with network coding (N-MANC) has demonstrated enhancement of the network scalability while improving the overall energy efficiency, transmission reliability and lifetime of the NB-IoT. Therefore, our future research work will consist of firstly implementing and assessing the proposed N-MANC on-chip for the same considered application scenario of smart water metering coupled with other application scenarios such as environmental monitoring sensor network. This implementation will help demonstrate and validate the proposed approach.


Resting state functional MRI (rs-fMRI) creates a rich four-dimensional data set that can be analyzed in a variety of ways. As more researchers come to view the brain as a complex dynamical system, tools are increasingly being drawn from other fields to characterize the complexity of the brain’s activity. However, given that the signal measured with rs-fMRI arises from the hemodynamic response to neural activity, the extent to which complexity metrics reflect neural complexity as compared to signal properties related to image quality remains unknown. To provide some insight into this question, correlation dimension, approximate entropy and Lyapunov exponent were calculated for different rs-fMRI scans from the same subject to examine their reliability. The metrics of complexity were then compared to several properties of the rs-fMRI signal from each brain area to determine if basic signal features could explain differences in the complexity metrics. Differences in complexity across brain areas were highly reliable and were closely linked to differences in the frequency profiles of the rs-fMRI signal. The spatial distributions of the complexity and frequency metrics suggest that they are both influenced by location-dependent signal properties that can obscure changes related to neural activity. Resting state functional magnetic resonance imaging rs-fMRI;  is a popWhile rs-fMRI analysis has usually used correlation or similar metrics to calculate network-based properties like functional connectivity, it is also possible to look at voxel-level or parcel-level properties. The most prominent examples of this type of analysis are ALFF  and fALFAs interest in the brain as a complex system grows, researchers are increasingly applying more sophisticated analytical techniques to the rs-fMRI signal. Time-varying analysis  attempts to characterize the underlying low-dimensional structure of the brain’s dynamics . At the We approached this problem by asking whether metrics of complexity provide unique information beyond the most basic features of the BOLD signal: its mean, standard deviation, temporal signal to noise ratio (tSNR) and power spectrum. In the large dataset available from the Human Connectome Project (HCP), we look at the relationship between these fundamental features and the higher level metrics of correlation dimension, approximate entropy, and Lyapunov exponent. Finally, we examine variability in fundamental properties and complexity metrics across parcels and across individuals, and their reliability across scans and across days.The minimally preprocessed and FIX de-noised rs-fMRI of the HCP S900 release  were dowUsing the minimally-preprocessed data, the mean and standard deviation were calculated for the time course from each parcel. The tSNR was then given by the ratio of the mean to the standard deviation.The power spectral density of each time course from each parcel for each subject was calculated using the Matlab pwelch function . Because the power spectrum is a vector, it is difficult to compare it directly to measures of complexity. We chose to characterize each power spectrum using a single value, the weighted average frequency (WAF):WAF = x is the power in each frequency bin x and fx is the frequency of the bin. The power spectra of interest tend to be dominated by low frequency components in the HCP data, and so the weighted average frequency tends to reflect the relative contribution of higher frequencies for each time course. where PSeveral metrics of complexity were calculated for each parcel from each subject. The first step of the calculation involves the estimation of a lower dimensional space that is “embedded” in the inherently high dimensional signal and captures its most prominent features. We used Matlab’s phase Space Reconstruction to estimate the dimensionality and the appropriate lag for a low dimensional representation of the signal. This algorithm uses the first minimum of average mutual information to estimate the embedding lag, and the false nearest neighbor algorithm to determine the embedding dimension. These values are used in the calculation of correlation dimension, approximate entropy, and Lyapunov exponent. The correlation Dimension function first generates a delay embedding of the signal, then is calculated as the slope of the correlation integral vs. the range of the radius of similarity. It provides an estimate of the chaotic complexity of the signal. The Lyapunov exponent measures the rate of divergence of trajectories in phase space and can be used as a measure of chaos in the system (lyapunovExponent). Approximate entropy measures regularity in a time series to quantify the level of complexity (approximateEntropy). For each metric, these calculation resulted in a 817 × 360 matrix.To examine variability across subjects, the mean across all parcels was calculated for each individual, and the mean and standard deviation of the resulting distribution across individuals were recorded. To examine variability across parcels, the mean across all subjects was calculated for each parcel, and the mean and standard deviation of the resulting distribution across parcels were recorded.To examine consistency for both parcels and individuals, we first calculated the correlation coefficient between the full matrices from D1S1 and D1S2 (to examine short term consistency) and between the full matrices from D1S1 and D2S1 (to examine longer term consistency). We also assessed the reliability of the metrics between D1S1-D1S2 and D1S1-D2S1 using intraclass correlation (ICC). Specifically, we used a two-way mixed single score ICC , which iBecause variations in basic signal properties are more prominent at the parcel level than at the individual level, we also calculated the correlation between the values averaged across individuals for each of the scans.n = 12, requiring p < 0.0042 for significance with α = 0.05).To determine whether complexity metrics  provide additional information beyond the basic signal properties , we first calculated the average across individuals to obtain the values for 360 parcels. Correlation between each basic property and each complexity metric was calculated and tested for significance. Bonferroni correction was applied to control for multiple comparisons . These time series were bandpass filtered using three different passbands: 0.01–0.1 Hz, 0.1–0.2 Hz, and 0.01–0.2 Hz. We then calculated correlation dimension, approximate entropy, and Lyapunov exponent for each filtered time series to determine whether the average frequency or bandwidth of the signal affected the metrics of complexity. To examine the effects of noise, time series were bandpass filtered (0.01–0.1 Hz), then Gaussian noise was added with an amplitude of 0.2 or 0.5 times the standard deviation. Correlation dimension, approximate entropy, and Lyapunov exponent were calculated.1 and github2.All code is available on the lab websiteThe mean signal intensity, standard deviation, tSNR, and weighted average frequency for all subjects and all parcels are shown in To examine the source of the differences in WAF, the power spectra for the five parcels with the highest WAFs averaged across subjects  and the five parcels with the lowest WAFs averaged across subjects  are shown in To understand the relationship between basic signal properties and measurements of complexity, we first estimated the optimal dimensionality and lag for delay embedding of the signal from each parcel from each subject. The resulting embedding is the basis for the other metrics of complexity examined here. As shown in Using the delay embedding of each time course obtained with the estimated dimensionality and lag, we calculated the correlation dimension, Lyapunov exponent, and approximate entropy for each parcel from each individual . CorrelaTo examine the variability in basic metrics and complexity metrics, we compared the values from D1S1 to the values from D1S2 and D2S1 using Pearson correlation of the entire parcel by individual matrix and intraclass correlation [ICC]. The results are summarized in We further examined the variability in the metrics across subjects and across parcels in different scans. The results are summarized in Visual inspection of the parcel by individual matrices for the various metrics reveals commonalities and differences. To reduce the effect of noise and focus on parcel-wise relationships, we calculated correlation between each basic signal property and each complexity metric after averaging over individuals. Scatterplots are shown in The correlation between the weighted average frequency and all complexity metrics was significant and surprisingly strong . Given the strong relationship observed parcelwise, we also calculated correlation between the full parcel by individual matrices. For WAF and Lyapunov exponent, the correlation was −0.64; for WAF and approximate entropy, the correlation was 0.65; for WAF and correlation dimension, the correlation was 0.17.We also obtained complexity metrics directly from the minimally processed data . Their vFor the randomly-generated and bandpass filtered time courses, neither the WAF nor the bandwidth affected correlation dimension, approximate entropy, or Lyapunov exponent . HoweverWe showed that basic signal properties of the spontaneous BOLD fluctuations in the HCP data set vary across brain regions. Several metrics of signal complexity  exhibit similar variability. While mean, standard deviation, and tSNR are significantly correlated to complexity metrics, the weighted average frequency exhibits the strongest relationship. However, analysis of filtered randomly-generated time courses shows that the frequency distribution alone is not sufficient to account for differences in complexity.It has long been known that different frequency bands of the spontaneous BOLD fluctuations have differing contributions from noise and other processes. In an early paper, A number of other studies have shown that the frequency distribution within the low frequency band adds information about the functional organization of the brain. Using a wavelet-based analysis, A few research groups have begun to examine the relationship between measures of entropy and different rs-fMRI frequency bands. Using a modeling approach, Wang et al. found that multiscale entropy in the lower frequencies (0.02–0.087 Hz) in twenty subjects from the HCP data set was positively correlated with functional connectivity between those areas . This isThe brain can be described as a complex dynamical system. At the simplest level of analysis, researchers have examined the complexity of the time course of activity from a given region of interest, using measures like entropy. For methods that measure neural activity directly , these types of analysis are expected to reflect the complexity of the brain’s activity and provide insight into the underlying dynamic structure of the brain. For rs-fMRI, however, the signal time course is only loosely tied to the underlying neural activity , b, partThere is no inherent reason that WAF should affect metrics of complexity. This is clear from the measurements made on the simulated data filtered into different frequency bands and from the moderate correlation between the full complexity and frequency matrices (∼0.6). We suspect that the relationship instead arises from common factors that influence both metrics. In this context, it is interesting that the highest correlations (> 0.9) are found between complexity measures and WAFs that are averaged across subjects to obtain values for each parcel. This suggests that there is something specific to each parcel, whether in the frequency of the spontaneous BOLD oscillations, the contribution of particular noise sources, or the overall level of SNR, that produces reliably distinct frequency distributions and measures of complexity. It seems likely that a combination of these contributions is responsible for the parcel-wise differences, rather than any single factor, and the weighting of the contributions may vary across brain areas.Parcels with high WAFs are especially common in the limbic system and other areas near the base of the brain. These areas exhibit low signal intensity and low tSNR. In contrast, parcels near the top of the brain  tend to have lower WAFs. Given the known variations in both image quality and coil sensitivity across the brain, the location of the parcels with high WAFs suggests that signal dropout, reduced coil sensitivity, and physiological noise may all be contributing factors. If this is the case, it would suggest that the relatively broader frequency profile in these regions indicates higher noise contributions that may obscure the desired signal. Our analysis of simulated data demonstrated that the addition of noise can substantially alter complexity metrics. The relatively low tSNR in the limbic system, for example, could account for the higher WAF and greater entropy (less predictability) observed there, along with reduced correlation. The lower Lyapunov exponent in the limbic system, which describes the tendency for divergence in trajectories that start from nearby points, may be the result of the manifold structure being degraded by noise. Effects like these that may result from fundamental aspects of image acquisition like coil sensitivity and signal dropout should be considered when interpreting any secondary signal features.It is noteworthy that while WAF was very strongly correlated with complexity measures, tSNR was much more weakly related to complexity. This suggests that while complexity is affected by noise, it also reflects other processes that are captured by the WAF and which may be neuronal in origin. Further work with simultaneous measures of MRI and neural activity will be needed to better disentangle the relative contributions of noise and brain activity to metrics of complexity.The WAF used as a metric in this study is a simplistic description of the actual frequency distribution. The primary difference in power distributions across parcels was in the width of the low frequency peak, suggesting that the full width half maximum of the distribution may also prove a useful metric.There are arguments to be made for using unfiltered data for the calculation of correlation dimension, Lyapunov exponent, and approximate entropy, but for the primary analysis in this study, we have used data filtered to the range typically used for functional connectivity studies, with the rationale of minimizing noise. However, filtering undoubtedly discards relevant information in the higher frequencies along with the noise . It may Unsurprisingly, the full matrices of complexity metrics were less reliable across scans and across days than basic signal properties. The phase space reconstruction and subsequent calculation of entropy and Lyapunov exponent benefit from the use of large numbers of time points. A better estimate might be obtained by concatenating all four resting state scans, but at the expense of examining the reliability of the metrics.We have shown that the frequency distribution varies within the band of BOLD fluctuations used to map functional connectivity, particularly across parcels. Complexity metrics like Lyapunov exponent and approximate entropy show similar variation, suggesting that common factors impact both types of metrics. Given the location of the parcels with the highest WAFs, we believe that the differences partially reflect greater noise contributions. These fundamental sources of variation should be considered during the interpretation of measures of complexity.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation, to any qualified researcher.Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.SK designed and executed the study. EM created the figures. BY performed the preprocessing. YG advised on statistical analysis. EM, XZ, W-JP, MN, NX, TL, and YG discussed the ideas and gave critical feedback on methods. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Inertial Measurement Units (IMUs) have become a popular solution for tracking human motion. The main problem of using IMU data for deriving the position of different body segments throughout time is related to the accumulation of the errors in the inertial data. The solution to this problem is necessary to improve the use of IMUs for position tracking. In this work, we present several Machine Learning (ML) methods to improve the position tracking of various body segments when performing different movements. Firstly, classifiers were used to identify the periods in which the IMUs were stopped (zero-velocity detection). The models Random Forest, Support Vector Machine (SVM) and neural networks based on Long-Short-Term Memory (LSTM) layers were capable of identifying those periods independently of the motion and body segment with a substantially higher performance than the traditional fixed-threshold zero-velocity detectors. Afterwards, these techniques were combined with ML regression models based on LSTMs capable of estimating the displacement of the sensors during periods of movement. These models did not show significant improvements when compared with the more straightforward double integration of the linear acceleration data with drift removal for translational motion estimate. Finally, we present a model based on LSTMs that combined simultaneously zero-velocity detection with the translational motion of sensors estimate. This model revealed a lower average error for position tracking than the combination of the previously referred methodologies. Human motion tracking is any procedure that tries to obtain a quantitative or qualitative measure of human movement. It has seen remarkable progress in recent years and is one of the most relevant tasks of motion analysis. Examples of quantitative analysis include the measurement of biomechanical variables such as the angle between joints or parameters that describe the gait of an individual ,2.Nowadays, some of the most exciting applications of human motion tracking are medical evaluation, people monitoring, and activity recognition. The continuous recognition of the human activity is fundamental to provide healthcare and assistance services to populations that need to be continuously monitored, such as the elderly or dependent subjects . FurtherIn recent past, non-visual sensor-based solutions have become a common practice in ambulatory motion analysis since these are portable, can detect motion continuously—even in outdoor environments—and have a low computational cost allowing a real-time feedback ,5. With Inertial Measurement Units based on Micro-Electro-Mechanical Systems (MEMS-based IMUs) have become a popular solution for tracking human motion. IMUs are usually constituted by a set of three accelerometers and three gyroscopes . HoweverThrough the merging of data from three mutually orthogonal gyroscopes and three mutually orthogonal accelerometers of an IMU, it is possible to determine six degrees of freedom (DoF) motion which includes the calculation of both a 3D position and a 3D orientation. The tracking of translational motion is substantially more complicated than the orientation tracking. Since double integration is needed to calculate the displacement of the device, errors rapidly accumulate. A significant problem of this procedure is related to gravity ‘leaking’. A bad estimate of the IMU orientation leads to an incorrect removal of the acceleration caused by gravity. This incorrect removal corrupts the calculation of the velocity and position of the IMU ,9.A classification problem: The identification of stationary moments can be used in zero-velocity update algorithms allowing a better estimate of position by permitting the correction of the velocity estimates.A regression problem: After identifying the moments in which the IMU is moving, it is essential to understand the translational motion performed by the IMU during those periods. Again, relying solely on the double integration of the acceleration data can lead to wrong estimations of displacements. Using ML techniques, we aim to obtain more robust estimates of displacement.In this paper, we propose the development of Machine Learning (ML) techniques to obtain a better estimate of the 3D human translational motion using inertial data. To achieve these estimates, two different challenges were tackled:The application of an inertial navigation system (INS) is the most direct method for estimating the velocity, displacement, and orientation of an IMU using the measurements provided by its accelerometers and gyroscopes. However, the amount of noise present in those inertial measurements, and because the error quickly accumulates, the INS estimates are unreliable after short periods of time .Traditionally, sensor fusion has been used to obtain more accurate estimates. These algorithms include the Kalman Filter, the Complementary Filter, or the Gradient Descent Algorithm ,11,12. TOther approaches fused video and inertial data using neural networks to perform six degrees of freedom (6-DoF) motion estimate. Some examples of this kind of methodologies are presented in ,16. HoweA type of information that can also be used in sensor fusion algorithms is related to the stationary periods of the IMUs. By determining the periods in which the IMU is still, it is possible to adjust the velocity estimation eliminating the drift. This kind of assumption is also interesting for human motion tracking applications due to the characteristics of the human movement. The identification of the zero-velocity periods is traditionally conducted using threshold-based methods ,18. The Some AI methods have also been proposed for identifying these periods. Park et al. 2016) [16 [20] iOther methodologies look at the trajectory estimate problem using end-to-end approaches. These include methods that, using the inertial data, try to estimate both position and orientation of an IMU ,22,23. ORIDI (Robust IMU Double Integration) (2018) is an approach which regresses the velocity vector applying an SVM on inertial data. That velocity is used to correct the accelerometer readings which are integrated twice to provide a displacement estimate. In .In both equations, It is also important to understand the average prediction error of each model in the 3D trajectory. With that purpose, the absolute trajectory error (ATE) of the position was also assessed. The absolute trajectory error of the position can be regarded as a 3D RMSE. This way, ATE is defined as :(11)ATE=Additionally a 3D MAE was also calculated:The calculation of RMSE and MAE provides a single, easy to compare value. However, it is essential to mention that these metrics are time-sensitive: if a position estimate mistake happens at the beginning of a trial, then the error tends to be higher then if the same error takes place as the end of the trial . BesidesAnother interesting regression performance metric is Rlated as :(14)R2=Vp that the developed model can explain. If the developed models predict perfectly the desired variable, then RThis index allows calculating the proportion of the variance in  R2 is 1 .The results of some of the different implemented methods are presented in Firstly, it is important to analyze the importance of the application of the zero-velocity updates. There is a significant improvement in applying the zero-velocity update using the different classifiers compared with the simple double integration of the signals without any correction. The use of drift correction also improves RMSE and MAE of all the trajectory estimates with a zero-velocity update. RThe model which stopped moments identification leads to a trajectory estimate with a lower error and a higher RAnother interesting point is to note that the displacement estimates possess a lower average error in the z-axis. Provided that the IMUs are not always under high linear acceleration (which is true for human movement), then these are very accurate in terms of vertical measurements. In contrast, IMUs fail in horizontal measurements because there is no way to compensate the gyro drift around global Z (without proper magnetometer corrections). To remove gravity from the accelerometer readings, one needs to know the orientation of the sensor relative to Earth. Since both accelerometers and gyroscopes are affected by errors, the estimation of that orientation is never totally accurate. This inaccurate orientation estimation—that in this case was previously conducted by the Xsens MVN Link system—leads to errors during the gravity removal process which will affect less the measurements in the vertical axis than on the non-vertical axes. Besides, it is also important to consider that less movement occurs in the z-axis  and thus the removal of the gravity may also be optimized in axes in which less motion occurs.To compare the different regressors, the same 6-LSTM classifier was used to conduct zero-velocity detection. Afterwards, the different regression models were used to estimate the displacement between zero-velocity periods. Starting by comparing the double integration with drift correction results with the ML regressors, one can conclude that these could not improve the overall displacement estimate during an entire trial.By comparing the different regressors with each other, it is possible to observe that the “pre-trained” model achieves a lower average error on the x-axis while the “no initialization” model achieves a better performance on the y-axis. On the z-axis and for the 3D displacement estimate, the performance of these models is similar.The stacked neural network methodology was experimented using three different Equation )—0.5, 0.By comparing the stacked neural networks with different The stacked neural networks method allows to train networks that are capable of simultaneously detecting zero-velocity periods and translation estimate. This way, it is probable that, during the training of the neural network, the model is capable of understanding what are the detected stationary periods that will support an improved translation estimate. This way, some errors that could exist on the zero-velocity “ground truth” will be annulled by the other goal of the neural network that is to find a better translation estimate  when a subject is performing several movements using inertial data. For this purpose, two objectives were defined: (1) zero-velocity detection for identifying periods in which the IMU is stationary; and (2) regression of translations that took place in periods of movement. Lastly, an ML model that was capable of accomplishing both tasks (zero-velocity detection and displacement estimate) was also evaluated.The developed zero-velocity detectors provided an apparent accurate detection of IMU stationary periods which seems to have allowed constraining the error that accumulates during the integration of the noisy inertial measurements. The application of these detectors to correct the velocity estimation allowed an enhanced tracking when compared with the simple double integration of the accelerometer data. It is important to state that the improvement of this detection, as well as an enhanced definition of the ground truth, would allow a better tracking. The removal of the linearly estimated drift between zero-velocity moments provided an additional improvement.Relatively to the regression models, used to estimate translations between zero-velocity moments, no significant improvements in comparison with more straightforward methods—as the double integration with drift removal—were achieved. The approach that combined both zero-velocity detection and translations estimate showed promising results.The majority of the work existent in this area is dedicated to different variants of the problem tackled in this project. Some authors use inertial data to track subjects walking and estimate the trajectory of their path or their velocity. Regarding zero-velocity detection, the existent work concentrates into zero-velocity detection during gait using IMUs placed on the foot, lacking both movement diversity and data from several body segments. Others concentrate on evaluating 6-DoF pose estimate for tracking in areas as robotics and automation and, therefore, do not focus on the more complex task of tracking human motion. Besides, to the best of our knowledge, there is not any type of similar work developed using the same dataset which constitutes an additional difficulty for any kind of comparison. This absence of work in the area made the comparison of the developed methods with prior techniques difficult.This work provides several research opportunities. First, further optimization must be performed in all the developed models. Zero-velocity detection was performed with high accuracy, although it can be improved with more exploration and optimization. The regression estimate presented results with lower performances and, in this way, future work can be focused on this area. Some examples of work that can potentially be performed on this part following the methods implemented in this project include: initializing the weights of an LSTM-based model using simultaneously data from the three-axis to provide a warm start to the network and the ability to capture information at the same time from inertial data from different axes; conducting warm start for the stacked neural network methodology—particularly on the first layer responsible for detecting stationary periods using the weights optimized during in the zero-velocity detection LSTM classifier. The work should also be focused in improving and solving the capacity to adapt to negative values presented by the approach that combined simultaneous zero-velocity detection and translations estimate as well as filtering the results obtained in that section.Other future work includes the validation of the developed methods in different datasets. Although the used dataset presents wide motion variety and data from several body segments, it is vital to understand if the developed models can generalize to other parts of the body, different IMUs, and movements of different nature. The final goal should be to achieve a model that can improve human motion tracking being applied to data from any IMU that is placed in any body segment during the execution of any type of movement.


The global aging phenomenon has increased the number of individuals with age-related neurological movement disorders including Parkinson’s Disease (PD) and Essential Tremor (ET). Pathological Hand Tremor (PHT), which is considered among the most common motor symptoms of such disorders, can severely affect patients’ independence and quality of life. To develop advanced rehabilitation and assistive technologies, accurate estimation/prediction of nonstationary PHT is critical, however, the required level of accuracy has not yet been achieved. The lack of sizable datasets and generalizable modeling techniques that can fully represent the spectrotemporal characteristics of PHT have been a critical bottleneck in attaining this goal. This paper addresses this unmet need through establishing a deep recurrent model to predict and eliminate the PHT component of hand motion. More specifically, we propose a machine learning-based, assumption-free, and real-time PHT elimination framework, the PHTNet, by incorporating deep bidirectional recurrent neural networks. The PHTNet is developed over a hand motion dataset of 81 ET and PD patients collected systematically in a movement disorders clinic over 3 years. The PHTNet is the first intelligent systems model developed on this scale for PHT elimination that maximizes the resolution of estimation and allows for prediction of future and upcoming sub-movements. Pathological Hand Tremor (PHT) is a common upper-limb motor symptom of several age-related neurological movement disorders and is described as involuntary and pseudo-rhythmic movements6 affecting coordination, targeting, and speed of intended motions7.Age-related neurological movement disorders such as Parkinson’s Disease (PD) and Essential tremor (ET)8 and affects the performance of individuals in high precision tasks such as robotic surgery9, PHT represents higher amplitude motion occurring in the broader frequency range of 3–14 Hz10. The repetitive and oscillating nature of PHT differentiates itself from other involuntary movement disorders such as chorea, athetosis, ballism, tics, and myoclonus11. Upper-limb tremor significantly limits individuals in performing Activities of Daily Livings (ADLs)12. Thus, during the last decade, several techniques and technologies have been proposed in both rehabilitation and assistive domains14 to compensate for the involuntary movement while promoting the voluntary component of motion. The accuracy of a tremor compensation technology (such as sophisticated wearable exosuits) relies significantly on the efficacy and spectrotemporal resolution of the algorithm, as inaccurate or slow extraction techniques do not allow for proper compensation. PHT consists of the following types of tremor: Rest Tremor (RT) occurring when a limb is relaxed and supported against gravity (commonly observed in PD15); Action Tremor (AT) occurring during voluntary contraction of muscles and classified into the subcategories of postural, kinetic, and isometric16. Postural and kinetic tremors are commonly observed in ET patients15. While postural tremor occurs when an individual voluntarily maintains a position against gravity, such as an outstretched arm, kinetic tremor occurs when a voluntary movement is performed. On the other hand, isometric action tremor exists during muscle contraction against a rigid stationary object such as grasping a solid object that blocks the motion of the limb and changes the length of muscles. It is worth mentioning that although PD tremor is typically characterized by unilateral rest tremor in the spectral range of 4–6 Hz and ET patients commonly show symmetric postural and kinetic tremor in the range of 4–8 Hz, there are many atypical cases (for example PD patients having action tremor and ET patients having rest tremor) that PD and ET share overlapping features17.Unlike physiological tremor, which is identified with low amplitude vibrations occurring within the spectral range of 6 to 14 Hz18 focus on alleviating tremor severity and improving arm functionality21. Although efficacy is reported for these PHT interventions, about 25% of patients are unresponsive or experience short-term suboptimal response23, and adverse side effects are also commonly observed. The severity and characteristics of PHT are assessed and monitored34 through recording and processing of hand motions in clinical settings while performing different tasks. This information has also been used to tailor dosing and regimen of therapy, such as botulinum toxin type A (BoNT-A) injections35. However, a major remaining challenge in assessing action tremor is the processing and separation of voluntary and involuntary components, which is not accurate using conventional approaches. Thus, some of the recently-developed techniques and therapies (e.g. BoNT-A therapy)39 that finely tailor dosing and muscle selection for targeted therapy based on accurate signal monitoring, would not be feasible for patients with prominent action tremor.PHT therapies such as oral medication, therapeutic lesions, Gamma-knife radiosurgery, and Deep Brain Stimulation (DBS)44 have attracted a great deal of interest due to their promising performance in compensating the involuntary tremor in both rehabilitation and assistive settings. Such technologies46 are mainly developed to remove (damp out or compensate) the tremor and assist patients in performing their voluntary movements49. However, the performance and efficacy of such technologies is directly linked to the accuracy of tremor estimation, which is a nonstationary, nonlinear, and uncertain signal processing challenge. Thus, accurate PHT elimination (and possibly prediction) is important for both robotic rehabilitation and assistive technologies, and also for different clinical applications. This will allow proper delivery of the expected assistive actions at the right time of oscillation, and also guarantees the required level of safety. In particular, for these settings, it is essential to have zero or minimal phase lag in the estimation and extraction of tremor to effectively and simultaneously generate a counteracting force field. It is shown50 that a phase-lag as low as 20 ms in the signal filtering part noticeably degrades the system performance. As the frequency content of pathological tremor can be close to that of the voluntary component, having zero or minimal phase-lag is a major challenge. However, this would be feasible if a robust and generalizable model of tremor is designed that can encapsulate nonlinear temporal dependencies between sub-movements during task performance.Recently, robotic rehabilitation and assistive technologies55, including our previous work14, there is an unmet need for a reliable, adaptable, and generalizable processing framework that can be directly translated to clinical settings for estimation, extraction, and prediction of action tremor with high spectrotemporal resolution. Among previously published studies, Band-limited Multiple Fourier Linear Combiner (BMFLC)51, Extended-BMFLC (EBMFLC)13, and Wavelet Adaptive Kalman Estimation (WAKE)14 frameworks demonstrate the most successful current state of the available approaches. The FLC-based methods, e.g., BMFLC and EBMFLC, are aimed at deriving linear mixing models for the spectral contents of the motion signal to distinguish and separate the two components. The BMFLC employs fixed predefined values to identify the spectral range of PHT and is aimed at attenuating those contents to derive the voluntary component. The EBMFLC, on the other hand, tries to adaptively identify and remove the spectral range of the PHT, which has shown superior performance compared to its counterpart. Recently, the WAKE framework showed significant improvement in performance in comparison to all existing techniques for PHT elimination. WAKE exploits Wavelet transforms and Kalman filtering to obtain a spectrotemporal representation of motion signals to separate the voluntary and involuntary components. Existing methods including EBMFLC and WAKE, share a similar characteristic by assuming that the spectral contents of voluntary and involuntary components are distinct and one could be derived by removing the other from the measurement signal. However, this assumption is not always realistic and has resulted in limited performance of the designed techniques and also hindered their clinical translation. In fact, the frequency contents of the voluntary and tremor components are not completely distinct from each other, and the overlapping of spectral contents is quite natural. Thus, considering all high-frequency components as tremulous motion results in inferior performance of the technique, which may fail to follow voluntary changes in the direction and frequency context of motion. Such behaviour can dramatically degrade the performance of any assistive technology that is supposed to react to the changes in motor intent, in an agile manner. This can significantly affect the estimation of involuntary components (through misinterpretation as involuntary motion the high frequency content of voluntary motion caused by nonlinear and nonstationary changes). This can directly affect the regimen of advanced new therapies that are tailored based on such measures. We believe that the above-mentioned issue is an important contributing factor for the limited performance of previous methods.Despite the crucial need for PHT elimination techniques and the numerous research advancements on this topic56. Therefore, one sample ahead-of-time prediction can play a very important role for compensation of external disturbances and achieving the control goals. On the other hand, a major difference between the proposed intelligent data-driven tremor extraction model and conventional filters is that since it is supported by the deep modeled connectivity in the dataset, it not only can remove phase latency but also has a one sample prediction. It is worth noting that phase lag (even in the order of 10 ms) can easily make a high-gain rehabilitation robot unstable which can sacrifice safety59. Therefore, the proposed PHTNet, which not only removes the lag but also enhances the time resolution, provides a significant phase benefit, which is imperative for the control algorithm of rehabilitation and assistive technologies.Besides concerns regarding the computational power and capacity of existing frameworks for an ultimate predictive model, there is a need for characterizing tremor based on a sizable inclusive dataset, that covers possible pathological variations causing diverse types of tremor signals in terms of spectrotemporal behavior, dynamic nature, temporal dependencies, and sub-movements. Without such a data atlas, conservative and impractical assumptions would be considered to define a ground truth reference for designing and validating the techniques. We argue that due to the high degree of inter- and intra-subject variability of tremor characteristics, the solutions designed and validated based on a limited dataset may not be generalizable for translation to the clinic and ADL for PD and ET patients. Thus, building a representative and sizable dataset coupled with designing a predictive model with high spectrotemporal capacity are critical for developing a PHT removal and prediction framework that is robust to inter- and intra-subject variability of tremor characteristics. This unmet need is addressed and discussed in this paper by utilizing a novel data-driven deep neural network modeling technique with a unique tremor extraction capacity augmented by predictive power and trained based on our unique dataset. The dataset includes kinematic motion recordings of 81 PD and ET patients (about hours of recordings) collected in a movement disorders research laboratory based on a rigorous systematic protocol. This is the largest dataset known to date permitting the generation of a strong high-capacity model. Developing a PHT extraction framework by employing large datasets, which include recordings of an extensive number of patients over long periods of time, takes a significant range of possible variations in the characteristics of tremor into account, adds more generalization to the framework, and makes the technique adaptable to inter and intra-patient variabilities, nonstationary and nonlinear behaviours of tremor signals. In other words, utilization of such larger dataset allows the network to more securely avoid the curse of overfitting to the training samples by observing a more diverse range of possibilities in the PHT behavior. The proposed model, referred to as PHTNet, is a real-time and assumption-free neural model developed based on bidirectional recurrent neural networks. PHTNet can process measurement signals in both online and offline fashion and provides one-sample-ahead (one sample ahead) predictions on the voluntary component of hand motion signals, which is the ultimate temporal resolution for this application. It should be noted that the time resolution of signal processing plays a very important role when the control loop gain and the frequency of activations in a control system are high (such as the robotic rehabilitation devices)To address the need for a valid ground truth of the voluntary component of action tremor signals when training the PHTNet, through a novel design, we have employed a mixture of synthesized voluntary components and recorded rest and postural tremor signals from PD and ET patients to teach the network to distinguish between the two motor behaviours. It is worth noting that this study is not made based on any potential differences between the characteristics of tremor in PD and ET. The data from two population of patients is collected to get a large variety of tremor characteristics and to empower our algorithm for a large population of users. One of the differences between PHTNet and previous works in the literature is that we do not bound the characteristics of PHT into a few commonly used parameterized assumptions in the spectral domain, as this is a highly dynamic feature. Instead, we employ a data-driven approach, PHTNet, which learns how to internally evaluate the separation criteria and employ them for extraction and prediction of the voluntary component through several training samples, without explicitly introducing any characteristic of or assumptions on the tremor. Moreover, the devised training strategy of PHTNet teaches the network to model the nonlinear short-term and long-term temporal dependencies and minimize the error between the output sequence and the forthcoming samples of the ground truth signal to equip PHTNet with predictive behaviour, which, for the first time, presents an ultimate temporal resolution.60.It is worth noting that the PHTNet is not proposed as a tremor treatment procedure, rather, it is developed to enhance the efficacy of current tremor management methodologies and also improve the quality of services delivered to the patients by robotic assistive devices. In other words, PHTNet can be put into practice to extract the voluntary and involuntary components of action tremor signals with a high spatiotemporal resolution to help the neurologists with objectively assessing the characteristics of tremor over the course of action and time, needed to specify the dosage and plan of prescribed medications. In addition, PHTNet can be directly employed by robotic assistive and rehabilitative technologies to enhance the quality of delivered assistive and rehabilitative services by maximizing the performance in reducing the tremor and stabilizing the motion, and by minimizing the risk of amplifying the tremor component by the device. Failure in precisely removing the tremor component in the input signals to the assistive devices can result in abrupt and unpredictable force profiles generated by the device, which can sacrifice safety. Details can be found in our previous papers inThis section describes the basics of the proposed PHTNet, as well as the systematic data collection strategy employed to design and train the network.www.clinicaltrials.gov” registry (Identifiers: NCT02551848 and NCT02668497). All experiments were conducted in accordance with the Declaration of Helsinki, as well as the Tri-Councel Policy Statement of Ethical Conduct for Research Involving Humans in Canada. The ethics committee provided full board approval for this study protocol and the consent procedure was approved as required in the documentation checklist, submitted with the full study protocol. Demographics data of the PD and ET group are tabulated in the supplementary material. All participants were recruited through the Movement Disorder Centre, at the University Hospital, London, Ontario, Canada. All participants provided written informed consent regarding their participation in the study. The participants recruited met the inclusion/exclusion criteria39. First participant’s first visit and last participant’s last visit occurred in March 2014 and January 2018, respectively.This dataset was collected from 81 PD and ET individuals who participated in a single-centre. The study protocol was approved by the Western University’s Health Sciences Research Ethics Board (REB#: 104584 and 107433) at the London Movement Disorders Centre in London, Ontario, Canada. The study protocol is registered with the “A convenience sampling of 119 PD and 131 ET upper-limb tremor assessments were utilized to develop PHTNet. The PD group included 47 patients, 8 females, and 39 males, with an average age of 39 and illustrated in Fig. Kinematic analysis of upper-limb tremor was conducted by having participants perform a series of seven scripted tasks each held for 20 seconds over three trials, as previously described61, for full reconstruction of a sampled signal, one needs to set the sampling frequency at least to double that of the maximum meaningful frequency of the signal. It is worth noting that although downsampling may generally impose distortions on the signals, in the case of tremor removal with the frequency of interest generally being less than 15 Hz, downsampling to 100 Hz would impose minimum to no distortion to the spectral range of interest in the hand motion signals, while at the same time it avoids imposing excessive computational costs on the system. Thereafter, the dataset was divided into three sets for training, validation, and testing of the network. Table To prepare the dataset for development of the PHTNet, all the recorded signals were downsampled to 100 Hz. Based on the Nyquist sampling theoremBased on the aforementioned motivations in employing pseudo-synthesized data, we addressed the need for a valid ground truth in action tremor signals by synthesizing the voluntary component and mixing it with static tremor signals. The voluntary component was a sinusoidal signal with random amplitude, frequency, and phase, which was modeled as62 are introduced and developed such that the intrinsic patterns of input data are automatically inferred and extracted.Machine learning is defined as a study of statistical and mathematical models, which enable a computer to capture the behaviour of a certain phenomenon without explicit instructions. Conventional machine learning methods are based on hand-crafted and user-engineered techniques developed to transform and represent raw data in a format which is perceivable by mostly-linear, or linear-in-parameter mathematical models. Performance of traditional machine learning methods, however, is normally restricted due to their limited modeling/learning capability. In addition, conventional machine learning methods require domain expertise and careful system design in order to have an acceptable performance. Therefore, representation learning methods63 which are specialized in analyzing sequential data and detecting long-term and short-term temporal dependencies in signals based on nonlinear embedded memory. An RNN model consists of a sequence of hidden cells employed to process a stream of data. In RNN models, at each time instance, a combination of input sequence, i.e., hand motion signal, and hidden state vector of the previous time instance are analyzed together to update the state vector and pass it to the next time instance. This process continues until the whole sequence of hand motion measurements is analyzed and a meaningful representation is formed. RNNs have various designs to fit different applications, e.g., sequence to sequence RNNs are employed for machine translation tasks, and sequence to single-output RNNs are employed for classification tasks. Since in this work, we translate a tremor-contaminated sequence to a voluntary component sequence, we employ the sequence-to-sequence architecture. A typical RNN representation is depicted in Fig. b is the bias vector for the input nodes; W is the weight matrix for hidden-to-hidden connections; U denotes the input-to-hidden weights of the RNN; c is the bias vector for the output nodes; V denotes the weight matrix for hidden-to-output connections; and f models are a subcategory of representation learning methods65, the forward and backward hidden cells are usually followed by a mixing matrix which merges the outputs of the two paths. However, in this work, through an architectural modification of the model, the BRNN kernel is applied without the mixing matrix. This is done since the ultimate goal is to have two separate processing pipelines for both online and offline applications. Finally, Fig. Using an RNN it can be expected that the output of the network for the very initial input samples is inaccurate and as the information propagates across the network and more samples of the input sequence are analyzed, the output becomes more accurate. Consequently, the output sequence becomes more reliable  after a transient phase of initial inputs. As in this work, our goal is to develop an online and offline tremor extraction framework, we structure the processing pipeline in a bidirectional format which employs two parallel sets of recurrent cells for the two processing schemes. As shown in Fig. 66 were developed. The reset gate determines the degree of dismissing old information and considering the data from input in the current time. The update gate, on the other hand, defines the degree of updating a hidden state based on the newly arrived data67. Therefore, we can update Eq.  in Eq. .Rigorous performance validation of PHTNet is satisfied by grid-searching over potential hyperparameters of the network, i.e., the length of input sequence, the number of RNN features, the number of hidden layers in the deep architecture, and the learning rate for the optimization algorithm. To identify the number of hidden layers in the RNN architecture, a comprehensive grid-search approach is taken to compare the MSE value over validation and test sets across different number of hidden layers. In this regard, the error of network over validation and test sets is obtained and plotted as a function of the number of hidden layers. As shown in Fig. ) in Eq.  best absAs stated previously, providing predictions on the estimated voluntary component of hand motion signals is an important feature for robotic rehabilitation technologies and an unmet need in the literature. A contributing factor for the absence of this feature is the highly dynamic behavior of PHT in and across affected individuals. Predictive operation of a PHT elimination framework grants the robotic systems enough time to adjust their parameters for the subsequent tremulous events. To address this, we have modified the way we feed the training examples to the network. In fact, instead of normally feeding PHTNet with In this work, we argue that the existing techniques for PHT elimination may suffer from the absence of a generalizable and inclusive method of extracting the ground truth for the voluntary component of action tremor to reliably measure the performance of the system. Hence, instead of employing conventional methods to extract the ground truth and calculate the performance, an inverse evaluation method is implemented. To this end, pseudo-synthesized action tremor signals are generated by mixing real static tremor recordings with a synthesized atlas of voluntary components, which provides an opportunity to numerically assess the performance measurement of PHTNet in training, validation, and test phases, without specifying assumptions to extract the ground truth signal. Existing methods on PHT elimination show limited performance in estimating the voluntary component of motion signals due to their dependence on different assumptions on the characteristics of the tremor, which normally shows a high degree of inter- and intra-subject variability. As our devised solution for PHT elimination is based on data-driven methods, and we can potentially generate any possible foreseeable signal for voluntary movements, we believe that employing pseudo-synthesized signals adequately addresses this challenge and eliminates the need for making unrealistic assumptions on tremor behaviour. To quantitatively measure the performance of PHTNet, the MSE criterion was used:Moreover, we have employed a qualitative approach, to visually investigate the performance of PHTNet in PHT elimination and demonstrate its superior estimation accuracy compared to its counterparts. In visual inspection, we can verify if the output of PHTNet follows the low frequency trend of the measurement signal, i.e., the voluntary component, that we intuitively expect. The visual investigation is performed to check if the designed neural network is operating in the expected way. It should be noted that visual inspection is not a means of evaluating the performance of PHTNet and only serves as the preliminary verification of the network.In this section, the performance of PHTNet is evaluated in several scenarios. Also, supporting results on the effectiveness and capacity of the devised PHTNet for PHT extraction problem are also discussed.Here, the results of a quantitative evaluation over the pseudo-synthesized validation and test sets are reported. As stated previously, the benefit of employing pseudo-synthesized validation and test sets is the possibility of numerical performance evaluation. Static tremor recordings of 16 subjects for validation and 16 subjects for testing are combined with voluntary components to synthesize data for quantitative evaluation. Please note that the tremor assessments of one patient only belong to the training set, or the validation set, or the test set to avoid leakage of data from training set to the other two sets. The network’s output for different pseudo-synthesized inputs is shown in Fig. y-axis of the plots represents the label of each technique and the disjoint area between the dashed lines of each two methods verifies that each two methods have significantly different mean performance from each other.Although the examples presented in Fig. 69 to check if the data samples for each frequency and for each of the input and output groups follow a normal distribution. After verifying the normality of data samples, we employed the “Z-test” to extract the confidence intervals by setting σ/n represent the standard deviation and the sample size, respectively. As shown in Fig. p-value between the two populations at each frequency point and also to illustrate the position of the two spectral populations with respect to each other. As demonstrated in Fig. Furthermore, to statistically investigate the efficacy of PHTNet in manipulating the spectral contents of hand motion signals, the FFT of the input and output signals in the validation and test sets are derived and statistically compared. Due to the large number of samples in each of the validation and test sets , we employed “Z-test” for statistical comparison of the input and output groups. We performed D’Agostino and Pearson’s testAs mentioned earlier, one goal of this work is to predict the voluntary motion of an individual in a one-step-ahead-of-time fashion, which is of significant importance in robotic rehabilitation technologies. To address this goal, we devised a novel training strategy to equip PHTNet with predictive functionalities. Table 51, EBMFLC13, and WAKE14. While the FLC-based methods, e.g., BMFLC and EBMFLC, are focused on modeling the spectral contents of the measurement signal with a linear combination of spectral components, the WAKE method employs spectrotemporal techniques and Kalman filtering to decompose the measurement signal into the two components of motion. Moreover, to visually inspect the performance of the forward (predictive) path of the PHTNet over action tremor data, the action tremor signals shown in Fig. Qualitative inspection of the PHTNet framework was performed on the action tremor recordings due to the absence of a valid framework to extract the voluntary component of hand motion signals. As the performance of PHTNet over real action tremor signals cannot be numerically reported, performance monitoring is conducted through visual inspection, where one checks if the estimated voluntary component is aligned with the expected low-frequency trend in the signal or not. Figure To investigate the performance of the PHTNet over hand motion recordings from healthy individuals, 2 set of signals from 2 healthy volunteers were recorded. Two healthy male individuals aged 28 and 37 participated in the data collection procedure and the acceleration of their hand motion is recorded with Trigno Avanti Wireless System PM-W05 (Delsys Inc.). Trigno Avanti sensors have a built-in 9 DOF inertial measurement unit which can relay acceleration, rotation and earth magnetic field information. The sensitivity of the sensor is set to 70, respectively, which results in accumulation of power in frequencies around 5 Hz. In the qualitative performance monitoring part, the output of PHTNet is compared with well-regarded, recent works in the field of PHT estimation, i.e., BMFLC, EBMFLC, and WAKE. As shown in Fig. From the results of the experiments over the validation and test sets presented in Figs. This paper proposes the design and implementation of a novel voluntary motion prediction and tremor removal technique that can be used for enhancing assistive devices and clinical settings. Although the proposed trained technique significantly performed better than all existing approaches, it requires relatively stronger computational support to be implemented due to the deep neural structure. It should be noted that the training of the model is completed in this paper, and the model can be used as a ready, plug-and-play trained algorithm without the need for retraining. However, utilization of any deep neural network with memory gates, requires sufficient computational power, which can be a limiting factor if the computational resources are strictly limited. Thanks to the power of new processing technologies, this challenge will not be very concerning but should be considered when implementing. In addition, we are working on a cloud computing approach for this work, which can be used for minimizing the need for having on-site computational power. Also, in order to enhance this aspect of the technique, we have an ongoing research to optimize the design of the PHTNet and implement hybrid and shallower models with comparable performance. In addition to the above points, it should be also highlighted that the accurate predictability of the proposed technique is a novel and unique feature, which do not exist in conventional techniques. However, the achieved horizon of prediction was limited. We are planning to augment the input space with other biological modalities and biomechanical models while improving the prediction ability of the technique to further enhance the horizon. Lastly, another future direction for this work would be to expand the size of the dataset employed and investigate the performance of the framework when the kinematics of motion in all dimensions are jointly fed to the network to potentially enhance the perception of the network over the voluntary action and increase its predictive horizon.Supplementary information