In this work the effects of simple imputations are studied, regarding the integration of multimodal data originating from different patients. Two separate datasets of cutaneous melanoma are used, an image analysis (dermoscopy) dataset together with a transcriptomic one, specifically DNA microarrays. Each modality is related to a different set of patients, and four imputation methods are employed to the formation of a unified, integrative dataset. The application of backward selection together with ensemble classifiers (random forests), followed by principal components analysis and linear discriminant analysis, illustrates the implication of the imputations on feature selection and dimensionality reduction methods. The results suggest that the expansion of the feature space through the data integration, achieved by the exploitation of imputation schemes in general, aids the classification task, imparting stability as regards the derivation of putative classifiers. In particular, although the biased imputation methods increase significantly the predictive performance and the class discrimination of the datasets, they still contribute to the study of prominent features and their relations. The fusion of separate datasets, which provide a multimodal description of the same pathology, represents an innovative, promising avenue, enhancing robust composite biomarker derivation and promoting the interpretation of the biomedical problem studied. Integration of multimodal and multiscale data is of known importance in the context of personalized medicine and future electronic health record management. The quest for suitable data fusion schemes, which could ideally optimize the exploitation of the information residing in composite datasets, is an emergent area with numerous potential applications. In the context of Virtual Physiological Human (VPH), an integrated framework should promote the interconnection of predictive models pervading different scales, with different methods, characterized by different granularity. Such a framework consolidates system level information and enables formulation and testing of hypotheses, facilitating a holistic approach .In this work we propose a novel methodology on multimodal data fusion regarding separate datasets. As separate we define datasets where each has been obtained from a different technological source and from a different set of patients. Even the number of patients participating in each examination is not the same. The only common determinant of separate datasets is that they refer to the same disease. Such datasets are not amenable to ordinary fusion methods, as all the known methods deal with the same set of patients being examined by various instruments and techniques in sequence, thus producing the multimodal data. However, the majority of open-accessible data refers to the unimodal results of certain experiment relative to a specific disease. The suggested methodology is able to highlight biomarkers utilizing these separate unimodal outcomes and thus repurpose the existing data of accessible repositories.As proof of concept, we focus on the fusion of two separate unimodal datasets, one of molecular and one of imaging description, both concerned with the study of cutaneous melanoma (CM). Application of feature selection and dimensionality reduction algorithms on the produced unified dataset can contribute towards the extraction of better biomarkers, ruling out false positive findings coexisting, but with no causal association, with the investigated disease. The procedure can be applied to various cases and tackle separated datasets of other diseases as well.This paper is organized as follows: Information fusing algorithms can be classified as belonging to one of the following categories: Combination of Data (COD) or Combination of Interpretations (COI) ; COD metSi, where x1, x2,…, xk represent the k observations in a study  and i represents one of the N data sources, i ∈ {1,2,…, N}. While this could be the case for specific studies or electronic patient records, most available datasets are essentially unimodal. That is, there are single modal data for a set of patients and then other available data of a different modality from observations on a different set of patients; both sets refer to the same disease although possibly at different stage or phase of the disease. Hence, there are other sources Sj, j ∈ {1,2,…, M}, where in general l ≠ k and M ≠ N. Nonetheless, the description conferred by each technological source Si, Sj characterizes the same disease, implying common phenotypic manifestations. Furthermore, associations underlining causal biological actions could emerge due to associated physiological determinants. We name the datasets derived from the sources Si and Sj as separate datasets.GFF algorithms assume that we have raw data from sources Cutaneous melanoma (CM) is considered a complex multigenic and multifactorial disease that involves both environmental and genetic factors. It is the most life-threatening neoplasm of the skin, and its incidence and mortality are constantly increasing worldwide. CM tumorigenesis is often explained as a progressive transformation of normal melanocytes to nevi that subsequently develop into primary cutaneous melanomas (PCM). However, the molecular pathways involved have not been clearly elucidated, although considerable progress has been made . DespiteThe microarray studies that have been performed on CM by different groups exploit different microarray technological platforms applied in highly heterogeneous patient cohorts and pathological sample collections . These dRegarding the clinical methods for diagnosis of melanoma, there exist several standard approaches for analysis and diagnosis of lesions, For example, the Menzies, scale, the Seven-point scale, the Total Dermoscopy Score based on the ABCD rule, and the ABCDE rule . In these methods, digital images can serve as a basis for the medical analysis and diagnosis of lesions under consideration. As human interpretation of image content is fraught with contextual ambiguities, advanced computerized techniques can assist doctors in the diagnostic process . A revieFeature selection techniques do not alter the original representation of the input variables but merely select a subset of them, a contrast to other dimensionality reduction techniques like those based on projection  or compression . Thus, feature selection techniques preserve the original semantics of the variables, enhancing interpretability by a domain expert .The main objectives of feature selection are (a) to avoid overfitting and improve model performance, (b) to provide faster and more cost-effective models, and (c) to gain a deeper insight into the underlying processes that generated the data.Regarding the applied feature selection procedures in this study, at first, a wrapper type technique was employed  using the random forest (RF) algorithm , which uThe random forest algorithm, among other ensemble learning methods, is reported to be successful in variance reduction, which is associated with reducing overfitting . In addiNext, principal component analysis (PCA) and linear discriminant analysis (LDA) were employed to highlight top features with larger coefficients as potential biomarkers. PCA is an unsupervised method and a data reduction technique that allows the major sources of variation in a multidimensional dataset to be analyzed without introducing inherent bias. PCA as a manifold provides an isomorphic, direct mapping of high-dimensional data into a lower-dimensional space capturing and representing the information of the original data incrementally. PCA defines new orthonormal in-between variables, consisting of linear combinations of the original variables such that the axis of the first principal component (PC) highlights the dimension collinear to the most variation and the axis of the second component the dimension with the most of the remaining variation and so on. The coordinates of the samples in the new space created by the PCs are called scores .LDA uses class information to maximize the separation between various groups of observations. LDA presumes that the classification variables follow a normal multivariate distribution and the covariance matrices for the observations of each class are equal (homoscedasticity). When these working assumptions are not considered plausible, LDA does not represent the optimal classifier. However, it can still be considered a valid and accurate method for the screen of the multidimensional solution space, when the objective is the determination of separating hyperplanes that maximize discrimination between different classes. This is so, because the hypotheses on the form of the data distributions do not have any impact on the solution of the geometrical separation problem .The workflow of the methodology is shown in All workflow programming was implemented in R .The dataset derived from skin lesion images contained 972 instances of nevus skin lesions and 69 melanoma cases. Three types of features were analyzed: Border Features which cover the A and B parts of the ABCD-rule of dermatology, Color Features which correspond to the C rules, and Textural Features which are based on D rules. 31 out of the initial set of 32 possible features were used; one feature was removed due to having zero variation across the samples. The relevant preprocessing for all features is described in . The dimThe microarray dataset was taken from the Gene Expression Omnibus (GEO) , GDS1375P value 0.001 and 2-fold changes as thresholds. The dimensions of the microarray dataset were thus 63 (rows) × 1701 (columns).Data retrieval from GEO was performed using GEOquery  and concThe two tables containing the microarray and image data were merged to one block sparse matrix with dimensions 1104 rows × 1734 columns, marking the unavailable values as NA. The rows contain the microarray and image data samples and the columns microarray and image features plus one binary response variable (0 for nevus and 1 for melanoma).“mean value” imputation,“random normal” imputation,“uniform” imputation,“bootstrap” imputation.Although there are several software packages implementing advanced imputation methods , they com) and standard deviation (sd) of each feature  per class, we randomly filled the missing values sampling from an assumed normal distribution having as parameters: . The “uniform” imputation is conducted by sampling uniformly within the range of each feature per class and the “bootstrap” imputation by independent bootstrap of each variable separately per class, until all the NA values are replaced. The last two imputation methods are similar to the way random forests construct synthetic data, in order to provide for a similarity measure [plyr R package was employed [In the second case, after estimating the mean value ( measure . For theemployed .caret (classification and regression training) [The first feature selection workflow was built, using the R package raining) . The seacaret documentation, the colinearity filter computes the correlations between the microarray features and then screens the correlation matrix, in order to remove features with high pairwise correlation. If two variables have a high correlation , the function traces the mean absolute correlation of each variable and removes the variable with the largest mean absolute correlation. The default cut-off value was empirically selected as to reduce the number of high-correlated features (genes) up to a level where “enough” genes were left to continue with the analysis (~30%). The cut-off value is dataset dependent, as the correlation among the features of an experiment is inherent to the experiment itself. As to the tolerance-in-the-performance criterion, it allows the selection of a subset size that is small enough but without sacrificing too much performance and can yield good results with a performance plateau for larger subset sizes. The combination of prior-filtering and tolerance-threshold resulted in the examination of four distinct cases: “Filtered Tolerance,” “Filtered Best,” “Unfiltered Tolerance,” and “Unfiltered Best.”The setup of the insilico simulation involved the examination of the reported selected feature subsets when (a) applying a colinearity removal filter to the microarray dataset prior to the execution of the selection algorithm (marked as filtered/unfiltered), and (b) setting a 95% tolerance threshold to the best-obtained performance criterion (Tolerance/Best). As noted at the only the microarray data (marked as o.m),the unified dataset produced by the mean imputations (m.i),the unified dataset by normal random imputations for the NA values (nr.i),the unified dataset by the “uniform” imputations (u.i),the unified dataset by the “bootstrap” imputations (b.i),only the image data (o.i).For each of the four cases, a 10-fold cross-validation procedure was performed with 50 repetitions on six different datasets:Throughout all repetitions, the nr.i, u.i, and b.i datasets were reimputed, thus providing more sampling variations. Prior to the application of the repetitions, the datasets were centred and scaled.For each of the 50 repetitions, the cardinality of the optimum subset of predictors was recorded, along with the names of the predictors and the performance attained. The area under the ROC curve (auc) was used as a performance metric. The auc of a classifier is equivalent to the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance. This is equivalent to the Wilcoxon test of ranks and is also closely related to the Gini coefficient .prcomp R function and LDA by the lda function from the MASS R package [Statistical analysis was performed using multivariate techniques, specifically PCA, followed by LDA. For all the cases, the colinearity removal filter from the caret package was applied to the microarray data. PCA was performed by the  package .lda function has two working modes: one having the parameter CV = False (the default), implying no application of cross-validation methods, meaning it can obtain an object that includes discriminant scores, and the other with CV = True, where predictions of class memberships are derived from leave-one-out (LOO) cross-validation. In LOO CV a model is trained iteratively with all the available observations (rows) but one each time, and a prediction is made as regards the response value of the left out observation. This procedure provides an estimation of the overall performance of our model.The lda function on each of the examined datasets and retrieved the estimated scores as well as the Singular Value Decomposition (SVD) parameters. SVD parameters provide the ratio of the in-between- and within-group standard deviations on the linear discriminant variables. Their squares conform to the canonical F-statistics. The lda function (having CV = True) was executed 50 times on each dataset , in order to assess the variability of the attained accuracy performance for the melanoma class. Next, the lda function was run again 100 times  on each dataset in order to assess the stability of the suggested top performing features.Initially, we ran the Finally, the set of the 20 top scoring biomarkers was derived, in order to assess the prediction performance for the melanoma case. The LDA (CV = True) and RF  methods were applied 50 times on each dataset using only the biomarkers' columns and the class as response variable. The RF performance was assessed using the out-of-bag (OOB) error estimation. OOB are the rows of the dataset that have not taken part in the creation of a decision tree due to the bootstrap procedure, and therefore they can be used to assess the performance of the created decision tree.The results of the tests regarding the feature selection process are depicted in The unimodal dataset of the image data (o.i) exhibited lower performance scores together with the higher cardinality of the selected features set. The application of the colinearity reduction filter to the microarray data had little effect regarding the dispersion of the optimum subset cardinality. The execution time in the condensed dataset, derived after the application of the filter, however, was 4 times faster than before, in proportion with the reduced number of remaining features after the use of the filter . The results on the imputed datasets of filtering demonstrated a drastic reduction regarding the number of features required to build the classifier, as well as the constancy of the feature subsets, regarding the derivation of the classifier in the iterative framework.The four imputed datasets, despite their different mechanism of imputed value generation, produced classifiers with similar cardinality. As shown in the feature of Tables The features from the mean-imputation unified dataset presented higher instability than all other methods and so proved to be the least preferable approach for the imputation procedure. All other imputation schemes  performed equally well in terms of stability.The noted weaknesses in the use of solely performance indicators for marker discovery, without considering the stability of the proposed marker set, has been raised in the literature  and is iNotably, none of the image-derived features were present in the top selected features of the unified datasets, as shown in Tables The majority of the image features were ranked as less important when compared to the features from the microarrays; this implies their lower informative power concerning the total observed variation in the integrated dataset. The lower informative power of the image features could be attributed to the phenotypic complexity of the image feature space, as well as the technical covariance and the size of its feature space. These factors mark their fingerprint in the integration process, despite the application of normalization techniques, thus impacting the response vector of the disease. When using the nr.i, u.i, or b.i method, however, the image features perform better, demonstrated through their more frequent presence in higher positions of the classifier's vector. This contrasts with the results of the m.i method. The mean imputation process relegated all image features to the lowest positions of the complete feature set, considering them less informative compared to the microarray features. In this sense, it is obvious that the three imputation methods, nr.i, u.i, and b.i, yield a more impartial effect. This can be surmised from the improved score of the image related features, providing practical value to its application in the integration process. The simulated dataset thus derived has a more balanced representation of features from the two modalities (microarray and image).In order to assess whether the small size of the image features (31 variables) compared to the 482 microarray features  is the reason that RF algorithm consistently favors microarray predictors as the best performers, we performed a series of simulations similar to those described in http://dx.doi.org/10.1155/2014/145243) we note that while in unimodal image data more than 2 PCs are needed in order to discriminate the two classes, in all other cases PC1 can separate the classes, with the exception of 2-3 samples, all of which come from the microarray dataset (noncrossed symbols). The three imputation methods  perform similarly regarding the extent of separation they attain between classes, as well as the percentage of variation captured by the two PCs.In The plot of the scores after application of the LDA after the nr.i imputation method is shown in lda function 50 times in LOO cross-validation mode for each of the datasets  enables measurement of the accuracy of the predictive model for the case of melanoma class. The accuracy is measured as the percentage of correct guesses in all predictions made by the model related to the dataset. As it is shown in The execution of the lda function 100 times without cross-validation for each integrative dataset and recorded the LDA coefficients (loadings). Then we introduced two performance indicators that rank feature performance as follows: the top-20 features with the largest mean coefficient (top-means) or the top-20 features which appeared most of the times having the top-20 largest coefficients (top-20). The results are shown in As with the high performing features derived by the application of the random forest algorithm, the stability of the features suggested by the LDA is of high importance. In order to test the stability of the features derived by the LDA, we executed the top-means) are retrieved after only one run, since there is no variation in the LDA coefficients as the dataset is stable in these cases. The top-20 features for o.m, o.i, and m.i have been included so as to have a full view of the best variables at each dataset. As it is obvious from the frequencies columns, the three imputation methods  show similar distributions. In order to assess the method which presents the better stability, we used as stability indicator (si), the number of common features between top-means and top-20 columns for each method. The si for nr.i, u.i, and b.i was 2, 5, and 1, respectively.In top-means), looking at the LDA and RF algorithms, to all the datasets. For the cases of the original datasets  only the relative part of the biomarkers was used as predictors. The results are shown in As a last step, we assess the predictive performance of the top features , where each source comes from a different set of patients, connected however by the same pathological mechanism. Although this study focused in the CM disease, the method could be applied, in general, to any disease where separate datasets exist, such as those residing in microarray expression, proteomic and genomic data repositories. Of course, special care has to be given to the selection of matching experiments regarding the disease, so samples, preparation methods, and the subsequent analysis are in the same context. In this work regarding CM, for instance, the microarray experiment selected was carefully screened among dozens residing in the GEO related to melanoma in order to match comparable state, progress, and tissue taken for the disease .In this work we propose the fusion of two The application of imputation methods per feature and per class fills the gaps in the block sparse matrix of the unified dataset and produces a dataset which is amenable for processing by multivariable machine learning methods. The imputation process as performed here is class dependant and so inserts a significant bias at testing time towards achieving high performance accuracies. As a heavy biased procedure still, it is able to integrate the different datasets at the data level and highlight features and relations amongst them which would otherwise be impossible.The three imputation methods (apart from the mean imputation), where each relies on a given statistical distribution scheme, perform similarly (auc or accuracy) regarding the predictor sets (biomarkers) they propose for the 2-class classification problem. These methods also performed equally well regarding the stability of the predictors, seen especially in the RF results. In all cases the unified datasets were able to produce biomarkers of higher predictive performance . In particular, when the stability of the proposed biomarkers was taken into account, the unified datasets had a superior performance, especially when compared with the solely microarray related features.The two joined datasets are characterised by class imbalance, features imbalance, and imbalance of the number of observations between the modes. An additional caveat is that the columns in both datasets are covariant. The random forest algorithm, applied in stratified bootstrapped samples, was capable of tackling these issues and performs almost perfectly in most cases, with the exception of the image-only data, which had an auc score of 0.8. The linear approaches (PCA and LDA) did not perform highly in the cases of the unimodal datasets (image or microarrays). For the unified datasets, however, both RF and LDA attained top scores.top-means favours microarray features , and top-20 favours image features . Moreover, as surmised from the svd values (Supplementary 2) of the unimodal datasets, both perform similarly and so the proposed top predictor sets contain variables from both modes. Additionally, in the same linear perspective, the image data points present better class separation in the unified datasets, as denoted by the crossed marks in the PCA and LDA scores plots. The selection of the 20 u.i top-means set as biomarkers , the effect of noise, in the form of false positives in each layer of description, is seriously limited. By transforming the initial datasets into a unified phantom set the effect of arbitrary feature covariance due to noise is confined in this layer of organization, downgrading their impact in the unified dataset and thus disqualifying them from candidates for the predictor set.These findings support the notion that the expansion of the feature set, through the use of the imputation methods, benefits the classification process. Proposing geometric spaces where appropriate separating hyperplanes can be derived incurs good performance in a methodologically similar way with methods as the Support Vector Machines. It is worth noting the dramatic improvement of the predictor sets inferred by the unified datasets , in terms of classification performance as well as informational content regarding the explained variation. Of paramount importance is the fact that this superior performance is rescued even by subsequent application of drastic data reduction techniques, such as the top-means and top-20, proposes equivalent feature sets capable of classifying robustly and accurately human samples, probed either through molecular (gene expression profiling) or imaging  examination. The fact that these signatures are top performers, not only in terms of efficiency in classification but also regarding the informational content of the observed variation that they manage to explain in the integrative dataset, renders them ideal starting points for functional interpretation of critical determinants of the CM pathophysiological mechanism. As previously said those signatures bear complementarity regarding the modules they select in order to perform the classification task, in terms of functional implication. Interestingly, the top-means indicator, which comprises the 20 features with the largest mean coefficients, is heavily enriched with features coming from the molecular layer, while the top-20 is basically comprised of image features. The HPCAL1.1 gene was encountered in all of the unified signatures with the exception of b.i, where CDC37L1 was observed.The analysis of the composite multimodal signature sets presented in Regarding the imaging features, they can contribute to the construction of reliable operators, that consolidate the wealth of morphological information, critical for the task of the dynamic description of the undergoing transformations of biological procedures in relation to disease manifestation. Moreover they may represent reliable macroscopic candidates for assessment of disease molecular subtypes, something that could be strengthened if covariance-based association of those markers with the molecular markers is undertaken. However in order that this analysis is biologically insightful and of practical use, large populations of cohorts are needed to check the consistency of the findings.The molecular part of the feature set comprises genes which imply promotion of tumorigenesis, angiogensis, and protein endoplasmic misfolding, that induces stress response repair signals, xenobiotic metabolism, and so forth, namely, involvement of cellular modular functions. These functions are known to be associated with the carcinogenic aberrant course in particular for aggressive cancers like CM. Indicatively, FOX1 possesses an established role in myogenic growth, differentiation, and blood angiogenesis. Defective function of FOX1 is incriminated for rhabdomyosarcoma type 2, a highly malignant tumour of striated muscle derived from primitive mesenchymal cells, which is a cancer model that is evolutionary close to skin cells . CDC37L1top-means and top-20. Still, this approach could be further developed to implement automated feature selection, exploiting statistically derived decision cut-offs. It can also be extended to enable juxtaposition with the automated filtering that occurs in the feature selection with RF, which is depicted at the 3rd column of Tables In this study, the direct comparison between RF and the linear information-oriented methods, regarding the stability of the proposed predictor set for biomarker discovery, was based on an empirical cut-off, namely, the top-20 features, as surmised by the LDA method. Subsequently, mode membership (image or microarrays) in the 20 feature set was examined for the two schemes: Paragraph description for Supplementary 1: PCA representations for the 6 different datasets, classifying melanomas from healthy donors, namely 2 datasets corresponding toeither the microarray(o.m) or the dermoscopic data(o.i), plus 4 integrated datasets, applying the missing value imputation schemes, defined in page 5.Paragraph description for Supplementary 2: LDA representations for the 6 different datasets, classifying melanomas from healthy donors, namely 2 datasets corresponding to either microarray (o.m) or dermoscopic data (o.i), plus 4 integrated datasets, applying the missing value imputation schemes, defined in page 5.Click here for additional data file.Click here for additional data file.


Their properties divide weighted networks in two broad classes: one is characterized by small hierarchically nested holes, while the second displays larger and longer living inhomogeneities. These classes cannot be reduced to known local or quasilocal network properties, because of the intrinsic non-locality of homological properties, and thus yield a new classification built on high order coordination patterns. Our results show that topology can provide novel insights relevant for many-body interactions in social and spatial networks. Moreover, this new method creates the first bridge between network theory and algebraic topology, which will allow to import the toolset of algebraic methods to complex systems.The statistical mechanical approach to complex networks is the dominant paradigm in describing natural and societal complex systems. The study of network properties, and their implications on dynamical processes, mostly focus on locally defined quantities of nodes and edges, such as node degrees, edge weights and –more recently– correlations between neighboring nodes. However, statistical methods quickly become cumbersome when dealing with many-body properties and do not capture the precise mesoscopic structure of complex networks. Here we introduce a novel method, based on persistent homology, to detect particular non-local structures, akin to  Complex networks have become one of the prominent tools in the study of social, technological and biological systems persistent homologyTo avoid this pitfall, given a weighted network filtration of the original dataset: data points are usually embedded in a metric space in order to extract from their configuration a sequence of growing simplicial complexes, which approximates with increasing precision the original dataset. Studying the changes of the topological structure along such filtration provides a natural measure of robustness for the topological features emerging across different scales. In analogy to the metric example, we call the set graph filtration: considering the set of all filtered networks captures the link weights and connectivity structure over all weight scales, without the need to resort to any assumption on an eventual metric structure underlying the graph structure. The graph filtration of a network Persistent homology is a recent development in computational topology designed for robust shape recognition and data-discovery from high dimensional datasets Rank the weights of links from At each step weighted network holes, that relate the network’s weight-degree structure to its homological backbone.A weighted network hole of weight By unearthing their properties, we obtain the main contribution of this paper: the statistical features of weighted network holes yield a classification of real-world networks in two classes, depending on the compatibility or lack thereof with null models generated by graph randomisations. Furthermore, this classification is defined by mesoscopic homological structures that cannot be reconduced to local properties alone.weighted clique rank homology, is the second novel main contribution of this paper. It allows to recover complete and accurate long-range information from noisy redundant network data, by building on persistent homology The method used for the classification itself, which we call deathEach weighted hole Similarly to stratigraphy, each step of the filtration is a topological stratum of the network, where the edge weight rank plays the role of depth. Intuitively, We applied this analysis to various social, infrastructural and biological networks (see SI for a detailed list). In order to compare datasets, indices are normalized by the corresponding filtration length  The statistical distributions obtained for the  classes :cycle distributions are markedly different from the randomized versions ;cycle distributions are very close to their random versions .solid than the randomized versions, while class II networks resemble more closely the randomized instances. Second, since the cycle abundance ratio between real and random instances is the same in the two groups, the differences between class I and II does not depend on cycle abundance, but rather on their properties.The short cycles of Class I networks nest hierarchically and appear and die over all scales while those in the randomized counterparts are born uniformly along the filtration but are more persistent, producing largely hollow network instances. The implications are twofold. Since cycles represent weaker connectivity regions, this results in class I networks being more network hollownesschain-length normalized hollownessk-th homological group This can be seen easily by compressing the whole information within two scalar metrics which do not depend on the number of generators in a given network filtration. We define the Interestingly, the hollowness values for the nstances , as oppointegrativenessNaturally, the two network classes do not represent a binary taxonomy and should be considered as two extremes of a range over which networks are distributed. For example, we find networks that interpolate between these classes, e.g. the online messages network has short persistence intervals, but also late cycle appearances and short length cycles. However, classes do not appear to display uniform behavior for local and two-body quantities: degree- and weight-distributions and correlations are mixed within the same group and do not provide a direct answer for the nature of the two classes. Similarly, a recently proposed measure of structural organisation, assortativity: for example, class I includes the gene network (assortative) and the airport networks (disassortative), while class II includes the assortative co-authorship networks and the disassortative Twitter data. Therefore, assortativity cannot be the discriminating factor between classes.Finally, the classes do not show a consistent pattern in perturbativeBecause homology is essentially a non-local property, it was expectable that the local measures mentioned would not be able to explain the observed homological patterns. Network homology can be seen in fact as the weighted complement to the A simple artificial network helps illustrating this point: Random Geometric Graphs (RGG) have been recently shown to display long-range many-body correlations For the latter and the airports, this organisation can be thought as the result of the non-local constraint imposed by the metric of the underlying space Further evidence of this behavior can be found by zooming on specific cycles which convey information about underlying constrains hidden in the network weight-link connectivity patterns. For example, the cycle structure of the air passenger network detects the expected reduced connectivity over oceans in the form of strong persistent cycles– and the strong backbone of US airport hubs, which is then filled by the local (intra-community) links . AnotherAt the opposite extreme of local quantities lie the spectral properties of networks. It is very important therefore to investigate whether it is possible to highlight peculiar spectral signatures of the two classes. Network eigenvalues, especially those of the Laplacian matrix, figure prominently in a number of applications, ranging from spectral clustering Interestingly, we find that class I networks have significantly larger spectral gaps local approaches.Among the many possible applications, two very relevant ones for social and infrastructural networks are the study of the weighted rich club’s geometry beyond the aggregate measure This work therefore provides a stepping stone towards understanding the coupling between network dynamical processes and the network’s homology.multi-persistent homology Finally, the filtration’s construction rule is flexible and can be readily adapted to other problems. Similarly to changing goggles, different edge metrics can be used . Individual flights between airports were aggregated on routes as defined by origin and destination cities. The weight reported is the yearly aggregated passenger traffic.The networks refer to the years 2000, 2002, 2006 and 2011. The years were chosen to provide snapshots of the air traffic situation at 4–5 years intervals, plus one extra (year 2000) just before the events of 9/11 which significantly affected the air transportation industry. The data used are publicly available from the website of the Bureau of Transportation Statistics .The online messages network consists of messages in a student online community at University of California The gene interaction network used in the paper is a sampling of the complete human genome dataset available from the University of Florida Sparse Matrix Collection. Each node is an individual gene, while the edges correlates the expression level of a gene with that of the genes (using a NIR score http://wiki.gephi.org/index.php/Datasets). Weights are proportional to the number of interactions between a pair of users.The dataset consists of a network of mentions and retweet between Twitter users and is available online on the Gephi dataset page (http://www.sociopatterns.org/) and analysed in The dataset contains two days of recorded face-to-face interactions in a primary school. Each node represents a child, with the edge weight between two nodes being proportional to the amount of time the two children spent face to face. We analysed the two days separately, yielding two networks. The dataset has been collected by the Sociopattern project  and the High-Energy Theory E-print Archive between 1995 and 1999 (hep-th) The graph edgelists used in the paper are available online as part of the code package we developed Finally, for comparison we use Random Geometric Graphs (RGG) The networks analysed in this article are undirected and weighted, because the weighted clique filtration finds a natural application in such case. However, schemes for directed networks can be easily devised and tailored to specific case studies, e.g. one could adopt the definition used in the directed clique percolation method The method we use to uncover weighted holes is persistent homology of the weight clique rank filtration. In this section we will briefly explain persistent homology and its realization through the weight rank clique filtration.simplicial complex and homology. A simplicial complex is a non empty family Persistent homology is a technique from computational algebraic topology that can be viewed as parametrized version of simplicial homology a subset of a face in the intersection of any two faces in We assume that the vertex set is finite and totally ordered. A face of Morphism between simplicial complexes are called simplicial maps. A simplicial map is a map between simplicial complexes with the property that the image of a vertex is a vertex and the image of a Simplicial Homology with coefficients in a field is a functor from the category of simplicial complexes to the category of vector spaces The construction that leads to the vector space The subspace The Persistent homology is the homology of a filtration, i.e. an increasing sequence of simplicial complexesIt assigns to a filtration the homology groups of the simplicial complexes For each homology group, the information about the filtration is collected in a barcode: the set of intervals In classical applications, the filtration is obtained from a point cloud using the Rips-Vietoris complex and persistent homology used to uncover robust topological features of the point cloud. We instead use the clique weight rank filtration to uncover properties deriving from the topology and weighted structure of weighted networks.clique complex is a simplicial complex built from the cliques of a graph. Namely there is a Recalling that an Weight Rank Clique filtration on a weighted network The Rank the weights of links from At each step For each graph The clique complexes are nested along the growth of In particular, persistent one dimensional cycles in the weight rank clique filtration represent weighted loops with much weaker internal links.There is a conceptual difference in interpreting witness complexComputing the filtration of a large dataset can be extremely demanding computationally. The identification of the maximal cliques requires in general exponential time, although algorithms exists for special cases that allow solutions to be obtained in polynomial time. In addition, the javaPlex library In the case of non-metrical discrete spaces, for example networks, one cannot easily construct a witness complex through a controlled sub-sampling of the network. Luckily, it is still possible to reduce the computational complexity in different ways: first, one can limit the analysis to the first File S1(PDF)Click here for additional data file.


In the Netherlands, absenteeism and reduced productivity due to work disability lead to high yearly costs reaching almost 5% of the gross national product. To reduce the economic burden of sick leave and reduced productivity, different employability interventions for work-disabled employees or employees at risk of work disability have been developed. Within this study, called 'CASE-study' , five different employability interventions directed at work disabled employees with divergent health complaints will be analysed on their effectiveness and cost-effectiveness. This paper describes a consistent and transparent methodological design to do so.Per employability intervention 142 participants are needed whereof approximately 66 participants receiving the intervention will be compared with 66 participants receiving usual care. Based on the intervention-specific characteristics, a randomized control trial or a quasi-experiment with match-criteria will be conducted. Notwithstanding the study design, eligible participants will be employees aged 18 to 63, working at least 12 h per week, and at risk of work disability, or already work-disabled due to medical restrictions. The primary outcome will be the duration of sick leave. Secondary outcomes are health status and quality of life. Outcomes will be assessed at baseline and then 6, 12 and 18 months later. Economic costs will consist of healthcare costs and cost of lost production due to work disability, and will be evaluated from a societal perspective.The CASE-study is the first to conduct economic evaluations of multiple different employability interventions based on a similar methodological framework. The cost-effectiveness results for every employability intervention will be published in 2014, but the methods, strengths and weaknesses of the study protocol are discussed in this paper. To contribute to treatment options in occupational health practice and enable the development of guidelines on how to conduct economic evaluation better suited to this field; this paper provides an important first step.NTR2886), Health and Motion (NTR3111), Guidance to Excel in Return to Work (NTR3151), Care for Companies/Second Care (NTR3136).Four trials involved in the CASE-study are registered with the Netherlands Trial Registry: Care for Work ( Many recent studies have focused on reduced productivity and work absence due to work disability in Western countries -5. Work Different return-to-work (RTW) interventions have been developed and evaluated to reduce sickness absence due to work disability. The effect, and occasionally the cost-effectiveness, of RTW interventions have been demonstrated in several observational studies -24, randThis paper is the first of its kind to describe a transparent, uniform methodological design for the economic evaluation of several different trial-based interventions in the field of employability, disability and return-to-work. Known as the 'CASE-study' , the project's aim is to evaluate at least five different employability interventions. All of the interventions share the common aim of reducing absenteeism and optimising the employability of employees with a known disability or those at risk of disability due to musculoskeletal complaints, mental health problems or a combination of both musculoskeletal and mental health complaints. In contrast with previous studies, the project design has no population restrictions and allows for the economic evaluation of any employability intervention regardless of the population they were developed for. Furthermore, the project also includes interventions provided in the primary and specialist health sectors as well as those delivered within the workplace. It also includes interventions aimed at reducing health care costs, wage replacement costs and other intervention costs in addition to improvements in return to disability duration and work return rates as measured by self-reported return to work dates, total duration of absence and subsequent recurrences, etc. Other outcomes include quality of life, quality of work life and general health status, etc. In addition, we cast a wide net to include a range of study designs since RCTs are not always possible in the economic evaluation of employability interventions. This paper describes the patient inflow in the event of an RCT or a quasi-experiment with a matched controlled design being conducted. For every employability intervention, an effect study will be conducted. In addition, from a societal perspective, a trial-based economic evaluation for every employability intervention will be carried out in a consistent manner. The results of these analyses will be discussed elsewhere in the future, but we hypothesize that the employability intervention is preferable in terms of costs, effects and utility compared to usual care. This paper will therefore demonstrate the strength of the CASE-study project in prescribing a methodological design which allows for the assessment of several different types of employability interventions within varying populations.We will carry out an economic evaluation designed for at least five Dutch employability interventions. Those undertaking the evaluation are not connected with the organisations or individuals who have developed, or are in the process of developing the employability interventions so no competing interests arise. Within the CASE-study each existing or innovative intervention will be compared with usual care for work-disabled employees or those at risk of work disability due to medical restrictions. At the start of the CASE-study, nine programmes were interested in participating in the project. Unfortunately four programmes could not collaborate because they were unable to provide the minimum required number of participants eligible for the study, or could not provide a comparable control group. The remaining five employability interventions have the ability to comply to the minimal requirements and focus on primary care, secondary or specialist care, and/or on the interaction between the employer and employee. Two existing employability interventions  and three newly developed employability interventions are included in the study. The employability intervention focussing on primary care aims to acknowledge the importance of employability in general practice. This intervention is under development at the University of Nijmegen (UMC St Radboud). The employability intervention in specialist care is being developed jointly by the VU University Medical Centre Amsterdam (VUMC) and the Institute for Health and Care Research (EMGO+) and focuses on employability among patients who suffer from rheumatoid arthritis. The third newly developed intervention aims at improving the interaction between the employee and his/her employer by appointing an independent mediator. The intervention is being developed by a separate team at Maastricht University. One of the 'Best Practice' interventions, developed by Health & Motion Nederland, offers patients with physical health complaints the ability to implement a range of exercises learned at the physiotherapeutic setting in their working environment by means of a workplace-intervention. The fifth and final intervention is developed by Second Care/Care for Companies (Best Practice) and aims at achieving sustainable employability among patients with physical and/or psychological health complaints using a multidisciplinary approach. An overview of these interventions is provided in Figure It is beyond the scope of this design article to describe the precise content of each employability intervention in detail. This will be done when the results of the economic evaluation per employability intervention are published. What is of importance in this paper is that the employability interventions use an innovative, systematic approach in order to support work participation in employees with health complaints. These are innovative because both the healthcare system and the private/work life of the employees are considered in the intervention. The features of the interventions differ according to the patient population, but the aim remains the same: to reduce sickness absence and increase the potential of achieving sustainable employability.The protocol for this study has been approved by the Ethics Committee of Psychology (ECP107) at Maastricht University, the Netherlands.The Gatekeeper Improvement Act stipulates that the responsibility for work resumption lies with the employer and the employee. The employer is obliged to start an employee's work resumption programme as quickly as possible, so the employee can resume his/her own work or other adequate work. In return, the employee should accept the work activities the employer provides. Furthermore, and unique in the world, the Dutch reintegration process provides the employer with salary and mandatory reintegration obligations for a period of two years . ParticiEmployees eligible for participation in the study will be work-disabled or at risk of work disability due to medical restrictions. In order to identify these employees, we will use employer registration forms, registrations with health care organizations and insurance companies, and registration forms from those providing the intervention. Employees eligible for inclusion will be 18 to 63 years of age, perform paid labour for at least 12 h per week, and are able to communicate in Dutch. Reasons for exclusion are absenteeism for 1.5 years or longer, fulltime students with a student job, informal caregivers and/or volunteers.In order to recruit a sufficient number of eligible employees, one or more health care providers providing an employability intervention will screen service users for inclusion in the study. When appropriate, information on the research will be provided to the patient. This will be done orally by the health care provider, possibly supported by a handout containing contact information for the researchers, what is expected of participants, privacy concerns, and so on. A self-completion card will be attached to each flyer (handout) consisting of a short checklist for inclusion and informed consent form for signing. The employee is requested to fill in the short checklist, sign the informed consent form, and send these back to the researcher. Participants will have at least 1 week to consider their participation in the study before signing the informed consent form. Participants will be screened for exclusion by the researcher on the basis of answers provided on the checklist. If an employee meets all the selection criteria and is willing to sign the consent form, randomization or matching will then place. Before the intervention starts, the first (baseline) measurement occurs. An email with a login code is sent to all participants, and they will be asked to fill in the baseline questionnaire. The questionnaire for the intervention group is identical to the one for the control group.As suggested by Drummond et al. 2001), a randomized controlled trial (RCT) remains the standard design for evaluating the relative effect of candidate interventions . However001, a raIn the case of randomization, the minimization method will be applied. Minimization aims to ensure an excellent balance for several prognostic factors in small groups . EmployeDuring recruitment, employees will be asked whether they want to participate in the research by one or more health care providers operating in the employability intervention. Participation requires employees to fill in a self-reported online questionnaire at time points T0 (baseline), T1 (6 months after admission), FU1 (12 month follow-up) and FU2 (18 month follow-up). After 18 months the study results for sustainable employability and quality of life are expected to extend into subsequent years. The timeline of the study can also be found in Figure The outcome assessment and data collection is identical for every employability intervention. Questions derived from different questionnaires, such as the 'National Working Conditions Survey' (NEA) , 'EuroQoThe primary outcome measure in the CASE-study is reduced sickness absence duration in calendar days. Reduced sickness absence will be measured subjectively via patient self-report records. The duration of sick leave will be calculated as the time in calendar days since reduced productivity occurred (baseline). This will then be compared with the productivity time of a person at 6, 12, and 18 months. The success of an intervention will be defined by comparing the average reduction in sickness absence between the intervention group and the control group. The employability intervention is effective when the employees in the intervention group return to work twice as quickly as employees in the control group. Data on sickness absence duration will be gathered by using the productivity and disease questionnaire (PRODISQ) .Data on secondary outcome measures will be gathered by means of self-administered questionnaires such as, NEA , EQ-5D-5After commencing a treatment intervention, a follow-up measurement will take place at 12 and 18 months to analyze the sustainability of employability. An intervention will be deemed to be unsuccessful when absenteeism of more than fourteen consecutive days after receiving the productivity intervention occurs. On the basis of previous research, it will be assumed that the long-term effect of the employability intervention stabilizes after 18 months .Self-related health will be assessed using one item from the Short Form Health Survey (SF-36). This widely used measurement tool describes how a respondent values and assess his/her general health . Data ga5) 3125 possible health states. Furthermore, the EQ-5D-5 L includes a visual analogue scale (VAS) rating from zero  to 100  [An explorative analysis of productivity costs and quality of life showed a clear relationship between both variables . Therefoh state) .Self-reported questionnaires will capture the costs and benefits from a societal perspective, thus independently of those who bear the costs, and those who receive the benefits. Self-reported questionnaires appear to be a reliable alternative for cost-diaries, at least for a recall period of 6 months, since participants will still remember significant life events ,47-49. CFor the economic evaluation of the three newly developed employability interventions, the study sample size was pre-specified by the researchers and the developers of the particular interventions. For the sample size calculation of the existing 'Best Practices', we expect that a Hazard Ratio (HR) of 2.0 indicates a relevant difference in the average number of hours/days of lost work due to work disability between employees in the intervention group and employees in the control group after 18 months. This HR is based on findings of recent studies in occupational health care ,52-55. WAll statistical analyses, performed to distinguish differences between the control group and the intervention group, will be performed according to the intention-to-treat principle. This means that the participants will remain in the group they were allocated to at baseline. All data regarding primary and secondary outcomes will be analysed using descriptive statistics. To control for protocol deviations likely to cause bias, data will be analyzed using per-protocol analyses whereby outcomes will be compared between those participants who received at least one or a partial intervention session and the control group. With the aim of correcting for possible regression to the mean, the baseline value of the particular outcome variable will be added to the model. Days of sick leave in the year before the inclusion, measured at baseline, will be added as a potential effect-modifier.Diagnosis and prognostic parameters can be confounders, thus the Cox regression analysis for confounders will be used to adjust for differences between control and intervention groups. If necessary, differences in baseline will be adjusted for prognostic dissimilarities. Missing data will be handled using SPSS missing value analysis on item level. Completely missing measurements will be handled using multiple imputation.The primary dependent variable is days of absence from work. In order to estimate the number of absence days, survival analyses will be used. There are two reasons why we opted to use survival statistics. Firstly, the time to sustainable employability is unlikely to be normally distributed which will make parametric statistical analysis difficult or inappropriate. Increasing productivity can occur quite early, possibly within months, but can also, as expected in the care as usual situation, may not be prolonged or may even decrease again. Secondly, survival analysis accounts for dropout by the technique of censoring. Participants who drop out will have the same hazard of sustainable employment than those that remained in the study. The Cox Proportional hazard model will therefore be used to analyse the HR of the days of sick leave duration. Reduced sickness absence duration in calendar days will be analysed with the Kaplan Meier survival method, and the differences in absence duration between both groups will be tested with a log rank test.To analyse the secondary outcomes, longitudinal multilevel analyses will be used to assess the differences in health and quality of life between the control group and the intervention group.Data will be analysed in SPSS 17.0.The economic evaluation will be carried out according to the intention-to-treat principle. Non-parametric bootstrapping (1000 replications) will be used to obtain 95% confidence intervals to estimate the uncertainty surrounding the cost differences, because cost-data are usually skewed to the right .For the cost-effectiveness analysis, the incremental cost-effectiveness ratio (ICER) will be calculated by assessing the ratio of the differences in costs between the groups to the differences in effects between the groups. The ICER indicates whether the additional investments needed for the interventions gain at least one extra unit of effect compared with care as usual . The cosBootstrapping with 5000 replications will be used to estimate the sample-uncertainty around the point estimates (ICER and ICUR). The bootstrapped ICERs and ICURs will be graphically presented on a cost-effectiveness plane. Furthermore, acceptability curves will show the probability that the employability intervention is cost-effective at a specific ceiling ratio, which is the maximum amount of money society is willing to pay to gain one extra unit of effect or a gain in quality of life . FinallyThis study protocol provides detailed information on how to assess the effectiveness and cost-effectiveness of employability interventions among employees with work disabilities or employees at risk of becoming work-disabled due to health complaints. The results derived from these economic evaluations can benefit the individual employee, the employer and the broader occupational health practice sector because illness duration, care consumption and costs of lost productivity are expected to decrease if the employability intervention is cost-effective. For society as a whole, cost-effective employability interventions are attractive since they are expected to increase labour participation and public health. Furthermore, we feel that productivity is the key to higher economic growth as we are facing an ageing population.A limitation of this study-design is the generalisability to other countries. The employability interventions are specifically tailored for the Dutch context in which they will be implemented. When applying these interventions in other countries, specific characteristics of the population in relation to the societal, political and cultural context in which the interventions are to be used should be taken into account. Although international generalisability might be restricted and require adaptation to the intervention, lessons can be learned and implementation advice given to those engaging in future work.Selection bias cannot be ruled out in this design because we ask the healthcare provider to select employees to participate in our study. As a consequence, healthcare providers cannot be blinded. Furthermore, the consultation with the health care provider might change the participants' point of view on work disability. This 'Hawthorne effect' will be born in mind and might overestimate or underestimate the effect of the intervention. However, we feel that minor interactions such as the provision of information about the study will not be an important source of bias.The strengths of this pragmatic design looking at the cost-effectiveness of employability interventions versus usual care, is the comparison under 'real life' circumstances of the intervention and control variables. Resource use is self-reported by participants and will be collected separately from the unit costs or prices of those resources. We will keep in mind the possibility that health care and intervention providers in different settings may wish to apply their own prices to the units of resource use. Furthermore, due to the standard recall period of 6 months, an adequate estimation of resource use is linear over time. This linear time trend restricts overestimation and underestimation of resource use between two consecutive measurement moments. An additional strength of this design is the inclusion of losses due to sick leave and the costs of using health care facilities together with productivity losses during working periods whilst unwell (presenteeism). In developing the methodology, we kept in mind the possibility that the employability and usual care interventions may have different effects on the component of reduced productivity while at work and therefore capture these. Besides internal validity, external validity of the results of the economic evaluation is reached by analysing the data from a societal perspective. We can split the calculated cost-effectiveness from the societal perspective to other perspectives, e.g. employee perspective, organisational perspective, etc.To conclude, the research design provided in this article will be applied to each employability intervention included in the CASE-study. The CASE-study is the first study assessing the cost-effectiveness of different employability interventions. In addition, this design might contribute to new and better guidelines in the conduction of future economic evaluations in the occupational health sector. The results of the CASE-study will become available between 2012 and 2014.CASE-study: Cost-effectiveness analysis sustainable employability; CEA: Cost-effectiveness analysis; CUA: Cost utility analysis; DDD: Daily defined doses; ECP: Ethics committee of psychology; EQ-5D-5 L: EuroQol 5dimensions 5levels; HR: Hazard ratio; ICER: Incremental cost effectiveness ratio; ICUR: Incremental cost utility ratio; NEA: National working conditions survey; PRODISQ: PROductivity and disease questionnaire; QALY: Quality of life; RCT: Randomized control trial; RTW: Return to work; SF-36: Short form (36) health survey; TIC-P: Trimbos/iMTa questionnaire for cost associated with psychiatric Illness; VAS: Visual analogue scale.The authors declare that they have no competing interests.CN is responsible for the data collection and drafted the manuscript. CN developed the study design. All authors participated in discussing the design of the study and developing the study protocol. All authors have read and corrected draft versions of the manuscript and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/12/43/prepub


Fast reaction to sudden and potentially interesting stimuli is a crucial feature for safe and reliable interaction with the environment. Here we present a biologically inspired attention system developed for the humanoid robot iCub. It is based on input from unconventional event-driven vision sensors and an efficient computational method. The resulting system shows low-latency and fast determination of the location of the focus of attention. The performance is benchmarked against an instance of the state of the art in robotics artificial attention system used in robotics. Results show that the proposed system is two orders of magnitude faster that the benchmark in selecting a new stimulus to attend. For successfully interacting with the environment in daily tasks, it is crucial to quickly react to ubiquitous dynamic stimuli. However, reaction time of state-of-the-art robotic platforms is limited by the low temporal resolution of sensory data acquisition and by the time required to process the corresponding sensory data.In conventional robotic systems, sensory information is available in a sequence of “snapshots” taken at regular intervals. Highly redundant data are received at fixed frame-rate. High dynamics can be sensed only by increasing the sampling rate, at the cost of increasing the quantity of data that needs to be transmitted, stored and processed.Additionally, the available bandwidth limits the amount of information that can be transmitted, and the available computing platforms limit the speed at which data can be processed, forcing a compromise between resolution and speed. As a result, current robotic systems are less efficient in reacting appropriately to unexpected, dynamic events  system has been implemented for the humanoid robot iCub which will therefore be able to quickly orient its gaze, scrutinize and act on the selected region and react to unexpected, dynamical events. Additionally, it can be of generic interest for robotics systems with fast actuation.The output of the In the following, we will describe EVA, show the improved latency in the selection of salient stimulus and compare its performance with the well-known state-of-the art frame-based selective attention system from the iLab Neuromorphic Vision C++ Toolkit (iNVT),www.icub.org) and is entirely based on the input from non-standard sensors. Such sensors use a new way of encoding information based on a custom asynchronous communication protocol Address Event Representation (AER). In the following we shortly describe the custom hardware and software modules developed for the attention system implementation.The selective attention system described in this work has been developed on the iCub humanoid robot (Dynamic Vision Sensors (DVS)  and an embedded processor specialized for asynchronous data, the General Address Event Processor (GAEP)  is composed of the emitting pixel address and the corresponding time-stamp. The application sends the received collection of events on the gigabit network where distributed processing takes advantage from middleware YARPaddress event, polarity, timestamp and type. The structure transparently manages events from the DVS sensor, as well as generic events such as complex objects deriving from clustering and feature extraction mask the AER stream into a dedicated format. The AE format consists in: Buffers of asynchronous data are handled with a two-threads method. N-buffering is used to guarantee concurrent access to data in process, thus avoiding conflicts and allowing each module to run at the desired rate irrespective of the incoming flow of event. Examples of developed modules are used to: display DVS activity, generate feature maps, perform weighted combination of multiple feature maps.EVA is an event-driven reduced implementation of the saliency-map based attention model proposed by Koch and Ullman  and Ittimapping. In the mapping, a RF is defined as a look-up table. The level of activation of the RF increases when it receives ON-spikes from the DVS pixel located in the ON-region of the RF and OFF-spikes in the OFF-region. If the neuron does not receive any spike over time, the activation decreases. When the neuron activation crosses a threshold, it generates a new event in the corresponding location of the feature map. Figures In EVA a number of features are extracted from the DVS output to populate diverse feature maps. As the DVS does not convey information about color or absolute intensity, we implemented a subset of feature maps from Itti and Koch : contrasThe visual field is covered with RFs following a multiscale approach. In the current implementation, we use two different scales with 4 × 4 and 8 × 8 pixels receptive fields. Figures mapping is less computationally demanding than a traditional convolution operation. Additionally, with this approach, the feature maps are only updated at the arrival of a new spike, without calculation of the RF activation for the complete image at each time step. To further reduce the computational load, we implemented non-overlapping receptive fields, at the cost of reducing the output resolution. However, in EVA the final goal is to obtain short latency in relation to saliency map resolution that guarantees reliable gaze shift. As a result the selected region is focused in the sensor's fovea for detailed inspection.The F):The final saliency map is obtained through weighted linear combination of the computed contrast (I), orientation (O) feature maps and flicker feature map . This mechanism prevents the system from immediately re-select the current winner, and allows for a scan of many points of the saliency map in order of decreasing conspicuity. However, in our setup neither EVA nor iNVT implement IOR, rather, the shifts of the focus of attention are determined by intrinsic noise in the system.A dedicated module implements saccades or gaze shifts toward salient regions selected by EVA. Tremor and microsaccades are used to generate motion of static visual stimuli on the DVS sensor focal plane, to elicit activity of the pixels that only respond to stimulus changes. This approach is similar to the mammals visual system, where small eye movements counteract photoreceptors bleaching adaptation Kowler, . Tremor The absolute novelty of EVA is in the short latency of the attentional shifts that guarantees fast reaction times. The proposed processing of the attention system generates short latency that hardly compares with the performance of frame-based attention systems. The selected attended location can be communicated to the oculomotor controllers to direct the robot's gaze toward salient regions with a saccade command. It continuously updates the saliency map and the resulting focus of attention location, allowing for fast reaction to unexpected, dynamic events and for a more natural interaction of robots with the environment.The improvement in computation latency is obtained thanks to many factors, among which the asynchronous low-latency and low-redundancy input, efficient sensory encoding, and efficient computing strategy (the mapping).iLab Neuromorphic Vision Toolkit (iNVT) .Figure The focus of attention locations selected by EVA and iNVT and their hit frequency are shown in Figures In EVA, the data rate depends on the lighting condition and on the stimulus, under these conditions it is about 7 kAE/s. Conversely, the data rate produced by a traditional color camera only depends on intrinsic parameters of the systems such as number of pixels, color resolution and frame rate, being independent from the stimulus; for the Dragonfly used on the iCub this amounts to 530 Mbits/s. The lower amount of data corresponds to lower processing demand and, hence, in a faster computation of the focus of attention location. Consequently this results in higher shifts frequency generated by EVA with respect to iNVT, and higher number of attention relocation, as shown in Table Figure One of the advantages of using the logarithmic encoding in the DVS is the wider dynamic range with respect to traditional sensors. Thus, we tested the attention systems in the scenario described above, but with reduced ambient light. The resulting focus of attention scan path is shown in Figures Figure We used a chaotic pendulum to test EVA with fast unpredictable stimuli. The chaotic pendulum shown in Figure The stimulus is so fast that neither the Dragonfly, nor the human eye, can successfully perceive its full motion. In this scenario, iNVT hardly relocates the focus of interest on the pendulum without introducing an evident delay. In iNVT, such shift is clearly shown in the video provided in Technical Materials.The fast movement of the pendulum generates a higher data rate with respect to the previous scenario. The resulting performance parameters are listed in Table To estimate the quality of the attention system, in Figure In this scenario, we assess how the performance of EVA scales with increasing number of events. In EVA, the number of events can increase for cluttered scenes and for higher resolution sensors, increasing the computational demand of the system. This happens also for iNVT, when higher resolution sensors are used. We estimate how the computation demand expressed in CPU utilization scales with the processed information (number of bits). In order to perform such analysis we determined how the computation demand of the two systems change when the quantity of information scales. For EVA, we control the number of generated events (and then quantity of information) by increasing the number of black edges printed on a white disk rotating at constant speed. We use five different configurations, where the edge is repeated every 360°, 180°, 90°, 45°, and 22.5°. Figure Figure To estimate these numbers, we select speed of rotating bar that covers typical use (4.2284 rad/s). Even though in normal scene operation the DVS activation is about 30%, in this test, we consider the worst case scenario where all the pixels in the sensor show the maximum level of activation (27 events per pixel).Thus, we estimate the maximum computation load associated to sensors that have dimension 128 × 128, like the DVS, 320 × 240, like the Dragonfly used for iNVT and 640 × 480. As the processing required by EVA sets well below iNVT, we conclude that for any possible situation, the required processing of EVA results less impacting on the performance than iNVT.This confirms the assumption that relevant saving in computation demand is associated to the design of the processing in EVA and it is not limited to the hardware of the system.mapping mechanism. The overall performance of the developed system takes advantage of the efficient information encoding operated of the sensor, its high dynamic range, low response latency and high temporal resolution. The use of non-conventional sensors coupled with an efficient processing results in a unprecedented fast attentional selection. We report the behavior of the system in three different experiments that highlight performance in detail. The three experiments give insights on the three major benefits of EVA: low computation demand, high responsiveness to high frequency stimuli and favorable scalability. The attention system EVA requires lower computation utilization up to one order of magnitude with respect to iNVT when stimulated by identical stimulus. This positive characteristic does not degrade the quality of the generated attention shifts. The characteristic of low response latency and high temporal resolution resulting from the efficient design of the attention system EVA allow remarkable performance in highly dynamic scenarios. The attention system accurately and swiftly redeploys the attentional foci on the most salient regions in the visual field even in situations where frame-based algorithms of visual attention fail in obtaining clear interpretation of the stimulus. The second scenario shows that the high temporal resolution allows the attention system to track very fast stimuli, expanding the application range of the system from humanoid robotics to even more demanding use cases. We presented a solution that, by sensing via efficient event-driven hardware sensor, provides outperforming selective attention mechanism with low latency and high dynamic range. In addition, for increasing the information load, e.g., for higher resolution sensors, EVA's CPU utilization increases with lower rate than iNVT's. The design of efficient processing in EVA guarantees, when compared with iNVT, relative superior performance of growing effectiveness with the amount of processed information. Finally, the last benchmark shows that the computational advantage of EVA is not restricted to the specific stimuli and sensor dimension used in this experimental setup, rather is more general.In this manuscript we described EVA, a real-time implementation of selective attention based on the bio-inspired model proposed in the foundational work of Itti and Koch , that usMost attention systems designed for real-time applications report the computational cost in terms of time needed to process a frame. The relative saliency map is often obtained in about 50–60 ms, slower than typical image frame-rate (30 frames per second) (Frintrop et al., EVA has been developed to equip the iCub with a fast and low weight attention system, exploiting the event-driven vision system of the iCub (Bartolozzi et al., The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Drosophila melanogaster has been established as a model organism for investigating the developmental gene interactions. The spatio-temporal gene expression patterns of Drosophila melanogaster can be visualized by in situ hybridization and documented as digital images. Automated and efficient tools for analyzing these expression images will provide biological insights into the gene functions, interactions, and networks. To facilitate pattern recognition and comparison, many web-based resources have been created to conduct comparative analysis based on the body part keywords and the associated images. With the fast accumulation of images from high-throughput techniques, manual inspection of images will impose a serious impediment on the pace of biological discovery. It is thus imperative to design an automated system for efficient image annotation and comparison.Drosophila gene expression images. The spatial sparse coding approach is used to represent local patches of images in comparison with the well-known bag-of-words (BoW) method. Three pooling functions including max pooling, average pooling and Sqrt (square root of mean squared statistics) pooling are employed to transform the sparse codes to image features. Based on the constructed features, we develop both an image-level scheme and a group-level scheme to tackle the key challenges in annotating Drosophila gene expression pattern images automatically. To deal with the imbalanced data distribution inherent in image annotation tasks, the undersampling method is applied together with majority vote. Results on Drosophila embryonic expression pattern images verify the efficacy of our approach.We present a computational framework to perform anatomical keywords annotation for In our experiment, the three pooling functions perform comparably well in feature dimension reduction. The undersampling with majority vote is shown to be effective in tackling the problem of imbalanced data. Moreover, combining sparse coding and image-level scheme leads to consistent performance improvement in keywords annotation. Drosophila melanogaster has been established as the canonical model organism for the study of the fundamental principles of animal development , where The BoW performs hard assignment for descriptors; that is, it chooses the closest visual word in the codebook to represent each descriptor. Then each image can be represented by a histogram of the visual word. Assume the number of patches for a given image is A recent study  shows thH as the histogram of an image generated by a non-spatial BoW; the spatial histogram bag can be written as Sn= where n is the number of spatial sections. In our work, we partition patches into 2 by 2 sections on each image, which enlarges the non-spatial BoW representation of the same image by a factor of 4.The spatial BoW can be obtained via augmenting the simple BoW representation to a larger vector with extra spatial information. We implement the spatial BoW by adopting the spatial pyramid matching scheme . Denote The BoW approach assigns each patch to the closest visual word in the codebook, which involves solving the following optimization problem: y to the codebook, which means only one visual word can be picked to represent the patch represented by y. The hard assignment ignores the relationship between y and other visual words, while the soft assignment method overcomes the limitation with assigning each descriptor to a limited number of visual words with different weights simultaneously.It is clear that BoW is a vector quantization from Denote the codebook matrix as λ is a parameter that controls the sparsity. This is essentially a linear regression problem with ℓ1 norm regularization, known as Lasso [where as Lasso  in the mas Lasso . We alsoX be the sparse coding representers of SIFT descriptors. We compute the image features by a pre-defined pooling function: Next, we apply different pooling methods to transform a collection of representers to one numerical vector (the image feature). After feature detection and description, each image is represented by a collection of representers  of the same dimension. Pooling is used to achieve more compact representations and better robustness to noise and variance. Let f is applied on each column of X. Recall that each column corresponds to the responses of all the descriptors to one specific item in the codebook. Therefore, different pooling functions construct different image statistics. We transform a collection of representers into one vector serving as the feature of the image using three different pooling functions: average pooling, the max pooling and the square root (Sqrt) pooling.where For any image represented by a set of descriptors, we can compute a single feature vector based on some statistics of the representers. For the BoW, a common choice is to compute the histogram: xi is the representers generated through BoW and M is the number of patches we have created. In this method, we have actually taken the average value of all the BoW representers. For the more sophisticated spatial BoW, the representation z for one image is the concatenation of histograms associated with various locations. In this case, z can be seen as again a histogram after normalization. The average pooling is the most commonly used pooling functions [where unctions -23 and iX to its max element: Average pooling can be used on both hard-assignment and soft-assignment representers. Due to the intrinsic continuity property of soft assignment, there are other ways to proceed the pooling operations. The max pooling function was introduced in -26, and The max pooling basically uses the strongest signal among multiple patches to represent the information of the image, and the characteristics of that image are potentially well captured by the max pooling. Max pooling has been shown to be particularly well suited to the sparse features .P=2). Mathematically, it is defined as: The P-norm is one of statistics that continuously transitions from average to max pooling, of which a special case is the square root of mean squared statistics (Sqrt) pooling . The Drosophila embryogenesis is partitioned into 6 stage ranges  in BDGP. We focus on the later 5 stage ranges as there are very small number of keywords appeared in the first stage range.The Drosophila embryos are 3D objects [D images that are taken from different views  [The  objects , and the-dorsal) . As majo-dorsal) , we focuSince most keywords are stage-range specific, we build a codebook for each stage range. Based on the codebook, the spatial BoW features and the spatial sparse coding representers are computed. We adopt both group-level and image-level annotation schemes to compare the performance. Various pooling functions are applied to generate the features for annotation.Trp, the remaining 20% as testing set Tstp. We also select a subset of samples Tstn(|Tstn|=|Tstp|) which do not contain the term. The test set Tst=Tstp+Tstn is kept untouched during our experiment. As the data distribution is imbalanced, we apply undersamplings for 21 times on the remaining negative groups to form Trn(|Trn|=|Trp|). Thus, we have 21 training sets Tr(i)=Trp+Trn(i), i=1,…,21, which are made up of the same positive set and different negative sets.Given the feature representation, we carry out the undersampling for prediction. We focus on the most frequent 10 terms for each stage range. The groups that contain a particular term are selected as our positive samples. We use 80% of the positive samples as the positive training set Trp and Trn and produces the predictions for Tst. The average and majority vote are used to summarize multiple undersamplings   to annotWe first carry out the comparison between spatial BoW and non-spatial BoW based on the traditional group-level schemes. We fix the same settings in both cases, including the training samples, test samples and the pooling functions. The only difference between the two cases lies in the length of SIFT descriptors: the spatial descriptors is 5 times the length of those for non-spatial representation. The extra part of the descriptors captures the location information of multiple patches for the images. As the positive samples for most terms are less than the negative samples, we perform 21 under-samplings to balance the training samples for the annotation. We choose four measurements: accuracy, AUC, sensitivity and specificity to evaluate the performance of different approaches.Comparison results for all 5 stage ranges by weighted averaging top 10 terms are shown in Table In this experiment, we compare the group-level scheme and the image-level scheme. For the group-level scheme, a group of images serve as one sample, and the corresponding keywords act as the labels. For image-level scheme, each image of the group is treated as one sample, and the keywords of the group are assigned to all the images within the group (for training). Given a group of images, the group-level models will predict the terms associated with the whole group; the image-level models will predict the terms associated with each image within the group and the union of predicted terms from all images within the group will be used as the final prediction.To support our image-level scheme, we implement another image-level scheme without union which evaluates the prediction on the image-level. The results are reported in Figure Our empirical study also shows that the image-level scheme with union outperforms the group-level scheme, which can be seen in Figure λ via cross-validation. We use average pooling for both BoW and sparse coding.In this part, we compare the performances of spatial BoW and spatial sparse coding based on image-level scheme, which produces the best result in the previous study. The BoW method assigns each SIFT descriptor to its closest visual words, and uses a histogram to summarize all of the representers. On the other hand, the sparse coding assigns each SIFT descriptor to multiple visual words with different weights, and different pooling functions are used to summarize the representers. For the sparse coding, we use a subset of images to tune We compute the weighted average for all of the 10 terms for comparison. Overall, spatial sparse coding performs better than spatial BoW (see Table X, the average pooling takes the mean value of each column of X, and the Sqrt pooling take the square root of ℓ2 norm of each column of X. Different pooling functions utilize different statistics to summarize the representers.To study the difference of various pooling functions, we conduct another experiment to compare three different pooling methods based on spatial sparse coding. Recall that the max pooling takes the max element of each column of Overall, the experimental results show that our image-level max-pooling approach achieves the best performance among different combinations of image representation methods and keywords annotation schemes. In particular, all three pooling methods perform comparably under the same condition. Furthermore, the max pooling and the Sqrt pooling produce slightly better results than the average pooling (see Table There are three main differences between the approach in reference  and our Drosophila gene expression pattern images. In our study, images are represented by BoW and spatial sparse codes. To transform the representers to sample features, different pooling functions are adopted. In addition, an image-level annotation scheme is presented to boost the performance. The random undersampling mechanics are applied to deal with the imbalanced data in our study. Results on images from the BDGP database demonstrate the effectiveness of our proposed approach.This article proposes an image-level undersampling scheme for annotating Drosophila embryo images in lateral view. There are also images in dorsal view and lateral-dorsal view which are not used in our study. We plan to extend the proposed method to other views of images in the future.The current approach is only applied to the http://www.flyexpress.net/.Images used in this paper are available at The authors declare that they have no competing interests.All authors analyzed the results and wrote the manuscript. JY conceived the project and designed the methodology. QS and SM implemented the programs. QS and LY drafted the manuscript. SN, SK and JY supervised the project. All authors have read and approved the final manuscript.