The platform offers prioritizations of putative disease genes, supported by functional hypotheses. We show that BioGraph can retrospectively confirm recently discovered disease genes and identify potential susceptibility genes, outperforming existing technologies, without requiring prior domain knowledge. Additionally, BioGraph allows for generic biomedical applications beyond gene discovery. BioGraph is accessible at  High-throughput methods for large scale and genome-wide identification of disease-related genes often result in large sets of potential targets requiring expensive and arduous experimental validation . For theBased on the availability of large volumes of curated biomedical databases, various methods for gene prioritization have emerged in recent years . These cPrioritization platforms commonly lack an easily accessible user interface for the formulation of queries and the intelligible interpretation of the results. One common problem is that most of the data mining platforms are supervised, that is, they require prior domain knowledge from the user. For example, in disease gene prioritization techniques, it is commonly required to define a set of known disease genes on which the system can be trained for the identification of new genes. Since these training gene sets are subjective, they will consequently vary between users and outcomes are strongly dependent on them, and the robustness of the predictions becomes impaired. These platforms offer rankings of possible susceptibility genes, but often lack comprehensible support for these prioritizations. Often, rankings of research targets are offered without references to the literature, inhibiting the user from evaluating the rationale behind the predictions. Still, platforms that offer rationale and incentives for researching functional support are mostly limited to a specific domain of interactions. A common paradigm, for example, is to adopt protein or gene interaction networks for the construction of functional hypotheses, which excludes alternative functional explanations in support of the predictions. Here, we propose BioGraph, a user-friendly computational platform that strives to overcome such deficiencies by applying novel data mining techniques on integrated databases of diverse types of biomedical knowledge.Summarized, BioGraph provides an online resource and data mining method for the automated inference of functional hypotheses between biomedical entities. Assessment of these hypotheses can consequently be used for the ranking of targets in the context of a research domain, such as a disease. BioGraph's resource is a knowledge base that integrates many biomedical databases into a common network of heterogeneous relations. These databases are selected based on their practices of manual curation by experts, guaranteeing that the integrated knowledge is accurate and valid. Our methodology generates a map of relations linking biomedical research subjects to potential targets, such as diseases, genes, ontology annotations, pathways, and so on, and offers literature support for these putative functional hypotheses. Assessment of these hypotheses' plausibility and specificity to source and targets allows for various applications in the identification of promising research targets. Here, we focus on the genome-wide identification of susceptibility genes for heritable disorders. The overall framework of BioGraph's methodology is schematically represented in Figure BioGraph is based on the data integration of 21 publicly available curated databases containing biomedical relations  Curated databases  and various protein-protein interaction databases) constructed by manual extraction of published, peer-reviewed information about a specific type of information, guaranteeing the quality of the relations in these databases. (2) Curated ontology databases  and Medical Subject Headings) using hierarchical classifications of subjects. (3) Curated annotation databases  and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway database) that relate biomedical entities or concepts to ontology terms.With regard to the integration of diverse databases with diverse identifiers for the concepts, each concept is provided with a distinct accession number, based on the Unified Medical Language System (UMLS) , to guarProvided with the integrated network, one can intuitively conjecture that nearby concepts in the integrated network are related. Indeed, since functionally related concepts are connected in the graph, we may assume that concepts that are close but only indirectly related in the network may also be functionally related in the real world. However, empirical analysis of the network shows that most of the concepts in the network are interconnected in only a few steps. This indicates that the network shows so-called small-world properties. Indeed, there is a considerable abundance of highly connected nodes. For example, interactions of proteins with water and ATP compounds or functional annotations such as the location of a protein in membranes or protein binding are prevalent. These unspecific hubs serve as ubiquitous connections mediating short path lengths between functionally unrelated concepts. This characteristic of the network prevents successful prioritization using simple shortest path methods. Still, our prioritization technique relies on the detection of nearby concepts in the network with respect to a source concept, but we correct the ranking of concepts for their global accessibility in the network.a priori importance or accessibility of concepts in a graph. This technique determines the global centrality of concepts in our integrated network. For this purpose, we compute the limit distribution that yields the probability of visiting the concepts when performing an infinite random walk on the integrated network. Google's PageRank algorithm [We provide a short technical summary of the methods here, but refer the interested user to the full implementation details in the Additional materials and methods in Additional file lgorithm  adopts aa posteriori accessibility of each concept from the source concept, measuring the probability of visiting each target concept from the source disease, pathway, and so on. Concepts are scored by their posterior probability, divided by the square root of their respective prior probabilities and ranked with respect to this resulting score. In practice, for a gene prioritization query, a user of the web application provides a 'research subject'  and a list of 'research targets'  that need to be ranked in relation to the research subject. Our algorithm then assesses and ranks the relations between the source concept and each of the target concepts as above. Since any type of concept can be provided as the subject or target of a prioritization, our method does not require prior domain knowledge from the user, that is, there is no need to define a gene set of known disease-causing genes for the identification of related genes, which results in a more reproducible and robust user experience.For computing the vicinity of targets to a source concept in similarity to the prior probabilities, we compute the limit distribution of a stochastic model of random walks with restarts in the source concept (with probability 0.25 at each step). As such, we compute the The method of performing random walks to determine the accessibility of target concepts implicitly generates ensembles of indirect paths between source and target concepts, which may serve as functional hypotheses for highly ranking targets. We can heuristically determine highly probable simple paths, that is, paths that do not contain cycles, of the random walk that starts in the source concept and ends in the target concept by adopting backtracking Figure . The bacThe resulting set of paths is presented to the user as a network with putative hypotheses linking the source to the target. Each directed edge represents a supporting relation among intermediate concepts, with annotated semantic meanings and literature references intelligibly supporting the relation for evaluation by the user. In cases where the target is highly ranked, specific and relevant connections and concepts are included in the constructed hypotheses. If the functional hypotheses linking concepts is limited to visiting general hub concepts, this is usually a sign that the linked source and target concepts can be considered unrelated, reflected by a bad ranking score.In order to assess the quality of BioGraph in prioritizing interesting research targets, we study its application in the identification of genes known to be associated with disease. Test sets of proven disease-related genes were selected from the OMIM Morbid Map database and Comparative Toxicogenomics Database (CTD). OMIM Morbid Map contains several thousand diseases and disease genes with a proven underlying molecular basis, manually selected and indexed from the peer-reviewed medical literature by experts . SimilarWe have adopted the BioGraph framework to prioritize all human genes in the context of diseases selected from these databases and evaluate the positions of the diseases' proven susceptibility genes in this ranking. We then compute sensitivity and specificity values and observe the area under the receiver operator characteristic (ROC) curve (AUC) as the standard performance measure for analyzing the quality of prioritizations or classifications . A perfeAs a first application, we analyzed the performance of our platform in prioritizing known disease genes among all genes in our integrated knowledge base. For testing a known disease-gene association, we first removed the link between the disease and its susceptibility gene from the knowledge base. We then ranked all genes in the network in relation to the disease and evaluated the ranking of the test gene. If the test genes ranked high, BioGraph allows retrieval of these genes as valid disease genes based on integrated information linking the disease to its susceptibility gene. For this test, we adopted published benchmarks and compared our prediction performance to that of Endeavour.Endeavour is a related and mature technology for gene prioritization , which aWe have computed the performance of BioGraph's prioritization method for the disease-gene prioritization benchmark initially published to evaluate the performance of Endeavour. This benchmark consists of 627 genes known to cause 29 diseases, selected from the OMIM database, of which 609 disease genes are present in our integrated knowledge base [Benchmarking a disease gene with BioGraph requires that each disease gene is evaluated by first removing the direct relations between the gene and the disease from the integrated network to ensure that the relation to be prioritized is not already in the network. Moreover, variants of the disease  are also disconnected from the disease gene. In order to identify these related diseases, we have selected the diseases for which at least one of its UMLS synonyms has the original disease's name as its substring. For example, we identify Charcot-Marie-Tooth disease, type 4C as a related disease to Charcot-Marie-Tooth or Alport's disease as related to Deafness, since a synonym of Alport's disease is Nephritis with nerve deafness. This method provides an objective interpretation of the benchmark by guaranteeing that no prior direct information can be exploited by our prioritization algorithm. Subsequently, for the prioritization of a disease gene, we perform a ranking of the integrated network's 16,912 known human genes with respect to the disease concept.For benchmarking Endeavour, each disease-gene relation was tested by removing the gene from the disease's known gene set, by training Endeavour on the remaining disease genes and by ranking the gene among a set of 99 random test genes. For both platforms, we adopt the AUC for analyzing the quality of these prioritizations.The mean AUC for BioGraph's prioritization of disease genes among all human genes is 92.92%, where the reported AUC for Endeavour in prioritizing disease genes among 99 random genes is 86.6% [The benchmark indicates that our prioritization approach yields a considerable improvement over mature technologies. There are two noteworthy differences in the experimental benchmarking design. Our platform does not require a training set of known disease-causing genes since it will implicitly base prioritizations on integrated disease-gene associations in addition to other heterogeneous types of integrated knowledge of the disease. This has a major advantage for the user since no prior knowledge of the disease is required. Secondly, our platform provides a ranking of the disease gene in relation to all known genes, where Endeavour ranks disease genes among a random set of 99 non-disease genes.As a quality control of the integrated databases, we have assessed the effect of each database on the benchmarking results by leaving out one database at a time and by assessing the prioritization algorithm on the Endeavour benchmark. This experiment showed that none of the included databases significantly harms the overall prediction capabilities. Conversely, it should be noted that some databases  are essential for successful prioritization, since leaving out these databases has a significantly negative impact on the benchmarks. More information on these quality checks is available in the Additional materials and methods in Additional file In the above benchmark tests, well-known disease genes are expected to rank high. Indeed, important susceptibility genes usually become the subject of intensive research efforts. Consequently, a literature and database bias may exist toward indirect evidence linking a gene to a disease in the integrated databases. Since BioGraph is capable of using this indirect evidence, the literature bias of important disease genes may strengthen the predictive power of our algorithm. To remove this bias, we can more objectively evaluate the platform by ranking recently discovered disease-gene relations that are not present in the knowledge base.Provided with the integrated network for which the resource datasets were frozen in March 2010, we identified all recently curated additions of human disease-gene relations from the July 2010 releases of the OMIM Morbid Map (15 new disease genes) and CTD  that are not present as direct relations in the knowledge base from March 2010. This yielded 845 recent disease-gene relations for which the ranks in the disease's genome-wide prioritization have been determined based on the integrated network of March 2010.Figure The above benchmarks demonstrate that BioGraph is capable of retrospectively finding or confirming existing disease genes, indicating that we can adopt the method to predict putative susceptibility genes for heritable diseases. Feasible applications of the framework are the identification of functionally interesting genes from sets of candidate genes - for example, in the identification of promising genes in linked regions, copy number variation regions or for the identification of genes through genome-wide association or expression studies.Additionally, the automated construction of hypotheses is of interest to explore genetic/genomic findings in peer-reviewed functional support. Collecting functional support for newly discovered disease-gene associations is not always obvious, especially when the functional evidence is indirect and spans several fields of interest. With the advent of high-throughput methodologies and torrents of published material to substantiate these findings, detecting relevant information has become a laborious process where computational techniques, such as those presented here, allow for these processes to be automated.in silico exploration of drug-drug interactions or the prioritization of identifying compounds in screening pipelines. Another example application is the computational inference of clinical biomarkers related to pathways, biochemical functions or disease processes, building on the various integrated types of concepts, relations and integrated literature references to detect promising candidates.Beyond applications in genetics and genomics, the framework can similarly be adopted to prioritize or to determine functional support for biomedical relations other than disease-gene associations - for example, in linking drug compounds, annotation terms, pathways, and so on - making the framework a very versatile tool in the discovery of diverse types of biomedical knowledge. In one feasible application, BioGraph can be adopted to determine functional interactions between drug compounds and for the To illustrate possible applications of the framework, we have employed the platform to predict candidate genes for schizophrenia (SZ) and substantiate the top predictions with support adopting the automatically generated functional hypotheses.SZ is a common neuropsychiatric genetic disorder with approximately 1% prevalence and with 64% heritability. It is characterized by a constellation of symptoms, including hallucinations and delusions, and symptoms such as severely inappropriate emotional responses, disordered thinking and concentration, erratic behavior, as well as social and occupational deterioration ,14.The newly identified genes are indirectly inferred from the integrated knowledge, but not directly associated in our gene-disease resource databases. The predictions in this section are based on a dataset freeze of the integrated databases from September 2009. This data freeze allows us to test if the predicted genes have been observed in genetic studies since the data freeze. Table PRL, the top inferred gene that is not a known disease-causing gene for SZ, encodes the prolactin hormone, of which the most commonly known function is to stimulate lactogenesis. Prolactin's relation to SZ is important, especially due to the effects of dopamine-regulating drugs aripiprazole and risperidone on the expression of prolactin and their adverse hyperprolactinemia-associated side effects [PRL is located on chromosome 6p22.3, which is linked to SZ through DTNBP1 [ATXN1 [PRL as a likely candidate gene for SZ. effects  where th effects . Additioh DTNBP1  and ATXN1 [ATXN1 . AlthougPRL for SZ are along the lines of current understandings and are schematically shown in Figure PRL are through the antipsychotic compounds aripiprazole and respiredone, which are both dopamine antagonists affecting the expression of prolactin. This hypothesis also shows that both compounds are adopted as drugs for attention deficit disorder, Asperger syndrome and autistic disorders. Additionally, PRL is associated with autistic disorder, strengthening the importance of it for psychiatric disorders. Additional paths from SZ visit SZ-associated genes and commonalities among these genes with PRL; TAAR6 and PRL are both genes in the neuroactive ligand-receptor interaction pathway; CCL2 and PRL are both regulated by 8-bromo cAMP, a derivative of cyclic AMP; DRD3 and PRL share the GO annotation 'Regulation of multicellular organism growth'. These relations may serve as indicators for determining the putative functional involvement of PRL in the etiology of SZ.The automatically inferred hypotheses by BioGraph that support the high ranking of HTR1A (serotonin receptor 1A) with SZ. The main hypothesis is driven by the receptor's interaction with the antipsychotic drugs aripiprazole and chlorprothixene. HTR1A is additionally linked to its paralog HTR2A, a known susceptibility gene for schizophrenia, via GO annotations on serotonin binding activity. Although our integrated disease-gene databases (OMIM Morbid Map and CTD) have not indexed HTR1A as a schizophrenia susceptibility gene, variants in the gene have previously been shown to be associated with schizophrenia and other psychopathologies [Figure hologies . This exHTR1A [DRD2 [DNMT3B [ATXN1 [DNMT3B with SZ was only reported in October 2009, where the data for our predictions are based on integrated databases from September 2009, demonstrating the usefulness of the currently proposed prioritization technique. Additionally, the highly ranking OMG gene has been shown to be associated with SZ, warranting replication studies for confirmation [SYN3 is the only gene in the top 20 for which several association studies have been performed, but where the findings show no support for SYN3 as a SZ susceptibility gene [Significant associations with SZ and polymorphisms in 4 of the top 20 ranked genes, namely HTR1A , DRD2 [2 [DNMT3B  and ATXNB [ATXN1 , have prirmation . SYN3 isity gene .For the remaining 15 of the top 20 genes, we did not find published association studies to support or contradict possible roles of these genes in SZ, although for some genes associations with SZ-like symptoms or related psychiatric and various neurological disorders have been shown, supporting the putative role of these genes in SZ  show that our prioritization method outperforms leading technologies and notable differences in the rankings are supported by comprehensible hypotheses that confidently support the prioritization. In experimental cases where an accountable gene needs to be identified in a set of 100 genes, BioGraph prioritizes the gene among the top 10 genes in 73.73% of the cases. We showed that BioGraph is able to retrospectively confirm recent disease-gene associations to the integrated databases (AUC 86.14%). Additionally, relations that have been confirmed in recent publications were successfully predicted. For example, BioGraph ranked a priori defined gene sets to determine related ontology terms, compounds or protein domains.Finally, we would like to note that, although the focus of the applications of BioGraph in this paper is in the ranking of disease-gene relations, the presented methodology is generic and applicable in various biological research settings requiring the construction of intelligent and intelligible hypotheses among interrogated concepts. One may use the platform, for example, to identify diseases related to a pathway of interest, or to enrich AUC: area under the ROC curve; CTD: Comparative Toxicogenomics Database; GO: Gene Ontology; GOA: Gene Ontology Annotations; KEGG: Kyoto Encyclopedia of Genes and Genomes; OMIM: Online Mendelian Inheritance in Man; ROC: receiver operator characteristic; SZ: schizophrenia; UMLS: unified medical language system.The authors declare that they have no competing interests.WD, PDR, JDF and BG conceived the project. AL, JDK and PDR created the BioGraph resource, data miner and hypothesis generator, designed and carried out the performance tests and built the web service. All authors have read and approved the manuscript for publication.Additional materials and methods and Additional Tables. Detailed methods describing technicalities of the database integration and algorithms, with the following sections. Knowledge integration: detecting hub nodes by computing a priori probabilities with random walks; computing a posteriori probabilities and ranking relations; backtracking heuristic for the automated generation of functional hypotheses; additional results. Additional Table 1: top 50 hubs or highest ranking concepts of the computation of the a priori rank score in the integrated network. Additional Table 2: area under the receiver operator characteristic (ROC) curve (AUC) for the prioritization of disease genes in the Endeavour benchmark. Additional Table 3: effect on the Endeavour benchmark after leaving out each separate database from the data integration process.Click here for file


Clustering is a widely used technique for analysis of gene expression data. Most clustering methods group genes based on the distances, while few methods group genes according to the similarities of the distributions of the gene expression levels. Furthermore, as the biological annotation resources accumulated, an increasing number of genes have been annotated into functional categories. As a result, evaluating the performance of clustering methods in terms of the functional consistency of the resulting clusters is of great interest.In this paper, we proposed the WDCM (Weibull Distribution-based Clustering Method), a robust approach for clustering gene expression data, in which the gene expressions of individual genes are considered as the random variables following unique Weibull distributions. Our WDCM is based on the concept that the genes with similar expression profiles have similar distribution parameters, and thus the genes are clustered via the Weibull distribution parameters. We used the WDCM to cluster three cancer gene expression data sets from the lung cancer, B-cell follicular lymphoma and bladder carcinoma and obtained well-clustered results. We compared the performance of WDCM with k-means and Self Organizing Map (SOM) using functional annotation information given by the Gene Ontology (GO). The results showed that the functional annotation ratios of WDCM are higher than those of the other methods. We also utilized the external measure Adjusted Rand Index to validate the performance of the WDCM. The comparative results demonstrate that the WDCM provides the better clustering performance compared to k-means and SOM algorithms. The merit of the proposed WDCM is that it can be applied to cluster incomplete gene expression data without imputing the missing values. Moreover, the robustness of WDCM is also evaluated on the incomplete data sets.The results demonstrate that our WDCM produces clusters with more consistent functional annotations than the other methods. The WDCM is also verified to be robust and is capable of clustering gene expression data containing a small quantity of missing values. The ay metric ,11. A clSince clustering is proposed, an increasing number of clustering approaches have been developed and improved for the analyses of gene expression data. The common clustering methods include k-means ,13, hierIn this article, we propose a Weibull distribution-based clustering method called WDCM. The assumption of this method is that the gene expression of each gene can be considered as a random variable following unique Weibull distribution , and thaThe following sections of this paper are organized as 'Results', 'Discussion and conclusion' and 'Methods'. In section 'Results', we first introduced three cancer gene expression data sets we used, and then visually demonstrated the clustering results obtained using the WDCM for the three data sets. Second, to assess the performance of the WDCM, we compared the functional consistency of the gene clusters produced by the WDCM to those of the k-means and SOM methods for the same data sets. We also used the external measure Adjusted Rand Index to establish the performance of the WDCM, and the comparisons with the other algorithms were conducted simultaneously. Finally, we tested the robustness of the WDCM on clustering the incomplete data sets. In section 'Discussion and conclusion', we first summarized the main work of this study, discussed the strength and limitation of the WDCM. In the end we briefly mentioned the improvement of the WDCM and the future study. In section 'Methods', we introduced the WDCM together with the algorithm used for clustering the Weibull distribution parameters, the functional consistency assessment method of the clustering result, and the external validation index Adjusted Rand Index of the clustering performance. Moreover, Robustness test of the WDCM on clustering the incomplete data set was also presented in this section.gij be the jth expression value of gene i, i = 1, ...,m, and j = 1, ...,n. We here treat one gene expression as a random variable, and construct the distribution of the gene expressions of gene i. We then choose a subset of genes whose distributions of the gene expressions belong to the common Weibull distribution [In this section, the WDCM is described as follows: Given a m × n gene expression matrix, let ribution . Due to xi1, xi2, ..., xin are the gene expressions of gene gi, i = 1, ...,m, and sort them in ascending as x ∈, define the empirical distribution of gi asFirst, we construct the empirical distribution of each gene expression , and theI(∙) is the indicator function.Where We utilize the Weibull distribution type to fit The probability density function of a Weibull distribution is defined as:a >0 is the scale parameter and b >0 is the shape parameter of the distribution. The scale parameter a determines the range of the distribution. The shape parameter b is what gives the Weibull distribution its flexibility. By changing the value of the shape parameter, the Weibull distribution can fit a wide variety of data.where i)((x) is a certain Weibull distribution with known parameters, and a Kolmogorov-Smirnov test is conducted to determine if the sample xi1,xi2, ..., xin comes from the Weibull distribution Fi)((x). The null hypothesis is that the random sample of gene expressions of gi comes from the Weibull distribution Fi)((x). If the null hypothesis is true, the deviation of Fi)((x) and Fi)((x) is small. Construct the Kolmogorov-Smironov statisticLet Fα if Kα is the critical value of the Kolmogorov distribution. Given α = 0.05, we here select the appropriate parameters for Fi)((x) in order to the null hypothesis is accepted , that is, the random sample comes from the certain Weibull distribution Fi)((x), i = 1,2, ...,m. Following the above procedure, we can obtain the Weibull distributions of m gene expressions, denoted by F(1)(x),F(2)(x),...,F(m)(x).under the null hypothesis, ribution . The nulθi denotes the parameter of the Weibull distribution Fi)((x), j = 1, ...,m. Here θi consists of double-parameter pair , we then cluster the m parameters θ1, θ2,..., θm using a certain clustering algorithm based on the hub points. This algorithm presented by Robert Clason designates a single point as a hub for each cluster and then finds the distance from each remaining point to each hub, as well as assigns this point to the hub to which it is closer [Let s closer . The merIn order to evaluate the performance of the proposed WDCM, we also apply the K-means and Self Organizing Map (SOM) clustering algorithms to the same gene subsets as the WDCM and obtain the gene clusters, respectively. We compare the functional consistency of the gene clusters produced by WDCM to those produced by the other methods. For this purpose, we consider the biological annotations of the gene clusters in terms of Gene Ontology (GO). The Gene Ontology (GO) project provides three structured, controlled vocabularies that describe the gene products in terms of their associated biological processes (BP), cellular components (CC) and molecular functions (MF) . The annARI) is a measure of agreement between two partitions of the same set of objects [X is the partition based on some external criteria and C is the clustering result obtained by some clustering method. Let a,b,c,d respectively denote the number of gene pairs that are in the same cluster in both X and C, the number of gene pairs that are in the same cluster in X and in different clusters in C, the number of gene pairs that are in different clusters in X and in the same cluster in C and the number of gene pairs that are in different clusters in both X and C. The Adjusted Rand Index ARI is defined as follows:The Adjusted Rand Index  index is introduced for assessing the overlapped degrees at individual missing percentages.n gene clusters C1,C2,...,Cn for the complete data set and m gene clusters I1,I2, ... Im for the incomplete one. The Cluster Overlap Ratio (COR) index is then defined as follows:Suppose wherepi represents the proportion of genes in the gene cluster Ii. Here xi denotes the maximum of overlapped gene numbers between Ii and each individual Ck  divided by |Ii|.|∙| denotes the number of genes in the cluster, and thus p-values produced by Kolmogoriv-Smirnov goodness of fit test for the 402 genes were reported in Additional file We applied the WDCM to cluster the lung cancer data set. It consists of expression levels of 675 genes across 156 tissues, which include 17 normal and 139 carcinomas lung tissues . Using tIt is evident from Figure p-values produced by Kolmogoriv-Smirnov goodness of fit test for the 471 genes were reported in Additional file We tested the WDCM on another follicular lymphoma data set consisting of expression levels of 798 genes in 19 B-cell follicular lymphoma specimens . We utilFrom Figure p-values produced by Kolmogoriv-Smirnov goodness of fit test for the 1040 genes were reported in Additional file The bladder carcinoma data set contains 1203 genes measured over 40 different experimental conditions . Using tARI for the WDCM, k-means and SOM algorithms on these three data sets were also contrasted (see Methods).To show the performance of the WDCM, we applied the K-means and Self Organizing Map (SOM) algorithms to the same gene subsets clustered by the WDCM and compared the functional consistency of the gene clusters produced by WDCM to those of the gene clusters produced by the other methods (see Methods). Simultaneously, the values of Among these three tested algorithms, the WDCM show the highest functional annotation ratios on both lung cancer and follicular lymphoma data sets. The detailed comparisons for the lung cancer data set are given in Figure ARI for algorithms WDCM, k-means and SOM on these three data sets. Note that among the three methods, WDCM provides the consistently best ARI values. Specifically, the ARI value for the proposed WDCM (0.5365) is much better than those for k-means and SOM (0.2478 and 0.3681) on lung cancer data set. Although these three ARI values  are close on B-cell follicular lymphoma data set, the ARI value for WDCM is better than the other values. For bladder carcinoma data set also, the proposed WDCM outperforms the other algorithms in terms of ARI. The values are reported in Table Table The above comparative analyses on the functional annotation ratios of the three algorithms have demonstrated that the genes in each cluster obtained using the WDCM show not only the similar expression patterns, but also more consistent functional annotations, which means these genes are more inclined to be involved in the same biological functions together. Also, the Adjusted Rand Index comparative results indicate the superiority of the performance of the proposed WDCM compared to the other algorithms.To test the robustness with which the WDCM clusters the incomplete gene expression data, we applied the WDCM to cluster the above three gene expression data sets containing missing values and compared the overlapped degree between the gene clusters for incomplete data sets and the ones for complete data sets. These three data sets were preprocessed by randomly removing 5-25% of the data in order to create the incomplete gene expression data sets, and the WDCM then was applied to these data sets. Table The results of the cluster overlapped degree comparison tests indicate that the WDCM gave a high overlapped degree of the gene clusters compared with those of complete data set at low missing value, highlighting the robustness and potential of the WDCM. We think that the results might stem from the fact that the missing gene expression values of individual genes have little influence on constructing their corresponding Weibull distribution parameters at low missing values.In this article, we propose a robust approach based on Weibull distribution (WDCM) for clustering gene expression data. It is based on the idea that a group of genes tend to be clustered together if the distributions of gene expressions of these genes belong to the common Weibull distribution and have the similar distribution parameters. Consequently, we cluster the genes by clustering the distribution parameters of their gene expressions. A hub nodes-based dynamic clustering algorithm is utilized in the distributions clustering process. The clusters number in a gene expression data set is automatically determined in this clustering algorithm. The performance of the proposed WDCM has been compared with those of K-means and SOM clustering algorithms by the biological annotation ratios to show its effectiveness on three cancer gene expression data sets. The results show that the WDCM is more capable of grouping the genes with similar expression patterns and strong functional consistency together. We also used the external measure Adjusted Rand Index to validate the performance of the WDCM. The comparative results demonstrate that the WDCM provides the better clustering performance compared to k-means and SOM algorithms. Moreover, the WDCM can be applied to cluster the incomplete gene expression data set without imputing the missing values. The results have demonstrated that there is high overlap between the gene clusters for the incomplete data set and those for the complete data set, which illustrates the robustness of the WDCM on clustering the incomplete data set at low percentage of missing values.In general it is known that due to the complex nature of the gene expression data sets themselves and the experimental errors in detecting the gene expression data, it is difficult to discover an acknowledged best clustering approach. In clustering process, the WDCM disregards a few genes whose gene expression distributions fail to fit the Weibull distribution. In future study, we will consider replacing the single Weibull distribution with the mixture distribution in order to cluster the whole data set. Besides, we will also increase the robustness of this approach on clustering the incomplete gene expression data set containing the missing values of moderate percentage. For the gene clusters found by WDCM, we would like to investigate which gene clusters and genes are correlated with some cancer phenotype, and which biological processes or molecular functions these genes in the clusters are concerned with. Our study may be helpful to gain insights into the complex diseases.The authors declare that they have no competing interests.HKW and ZZW jointly proposed this approach and conducted the data experiments. XL gave the statistical idea of the method. BSG modified this paper. LXF partly wrote the program codes. Testing was done by YZ. All authors read and approved the final manuscript.A clustering algorithm based on "hub nodes". A clustering algorithm used to cluster the Weibull distribution parameters.Click here for fileP-values of tests for the three data sets. This file consists of three spreadsheets, each lists the gene numbers and p-values of Kolmogorov Smirnov test for one data set.Click here for file


The role of cerebellar plasticity has been increasingly recognized in learning. The privileged relationship between the cerebellum and the inferior olive offers an ideal circuit for attempting to integrate the numerous evidences of neuronal plasticity into a translational perspective. The high learning capacity of the Purkinje cells specifically controlled by the climbing fiber represents a major element within the feed-forward and feedback loops of the cerebellar cortex. Reciprocally connected with the basal ganglia and multimodal cerebral domains, this cerebellar network may realize fundamental functions in a wide range of behaviors. This review will outline the current understanding of three main experimental paradigms largely used for revealing cerebellar functions in behavioral learning: (1) the vestibuloocular reflex and smooth pursuit control, (2) the eyeblink conditioning, and (3) the sensory envelope plasticity. For each of these experimental conditions, we have critically revisited the chain of causalities linking together neural circuits, neural signals, and plasticity mechanisms, giving preference to behaving or alert animal physiology. Namely, recent experimental approaches mixing neural units and local field potentials recordings have demonstrated a spike timing dependent plasticity by which the cerebellum remains at a strategic crossroad for deciphering fundamental and translational mechanisms from cellular to network levels. Recent evidences show that the cerebellum plays a key role in motor and nonmotor domains through a great number of cerebro-cerebellar closed loops  Figure  that susThe crucial position of the cerebellum in brain circuitry and its involvement in sensorimotor and cognitive processing make it an ideal structure for studying the possible role of neuronal plasticity in a translational perspective on learning. The attractiveness of the cerebellum in the context of learning results is enhanced because it can be modeled as a neuronal network in terms of a “wiring diagram” , in whicIn order to establish a comprehensive theory of learning, we need to determine a chain of causalities linking together neural signals, plasticity mechanisms, neural circuits, and behavioral learning . To usefWhatever the type of the involved paradigms, the output of the cerebellar cortex is always determined by the PC firing. This major signaling can be electrophysiologically identified by two types of firing patterns, namely, complex spikes (CS)  and simpSS firing presents a resting state at about 50 Hz in alert animal , 43–48. vice versa [The specific discharge frequencies of CS and SS are reciprocally organized, that is, an increase in CS is accompanied with a decrease in SS, and ce versa –62. Thisce versa . Three min fine implies one site of plasticity in the cerebellum and another one in the brainstem. This conclusion has been accepted following a protracted controversy between cerebellum [Since Ito's original proposal  that therebellum  and brairebellum  “supportThe VOR stabilizes retinal images during self-induced and artificially induced rotational head movement by generating smooth eye movements that are opposite in direction and nearly equal in amplitude to head movement. When the compensatory eye movement is not adequately adjusted a retinal-slip error signal is generated . These eSince the pioneering recordings of the PC in the floccular region in the behaving monkey by Lisberger and Fuchs , 68, thein vitro or in anesthetized preparation about 50% of PCs present three-mode state of firing  [After the determination of the different inputs reaching a microzone, the output signals elaborated by the PC must still be identified in order to discover the input-output transformation realized by the cerebellar cortex. This approach is complicated by the marked complexity of PC behaviors of the same microzone compared to input signals, despite similarity of the various MF input signals conveying head velocity and eye velocity-position signals , 80 Fig. Althougiescent) –91, switiescent) , 92–94 aiescent) . Lisberger  proposedBy adapting this motor perturbation paradigm to the eye pursuit system, Medina and Lisberger  were theAs different levels of plasticity are possibly involved in the final operation exerted by the cerebellar cortex, it is difficult to identify the contribution of each level. As regards the granular layer, all GCs are required for the acquisition of new memories, whereas only a minority of GC is sufficient for the maintenance of basic motor performance –105. YetClassical eyeblink conditioning constitutes one of the most used experimental models for investigating the neural mechanisms underlying motor learning. The conditioning of eyelid/nictitating membrane response was first studied in humans   and an unconditioned stimulus (USt)  . Two priOne of the factors that have contributed to considering eyeblink conditioning as one of the most important experimental models for cerebellar motor learning studies partially relies on the knowledge of the anatomical and functional pathways conveying CSt and USt information to the cerebellum , who assDuring the last decade, the cerebellum has been considered the principal structure associated to eyeblink conditioning containing, together with the brainstem, the entire essential circuitry involved in classical conditioning . AccordiA receptor agonist muscimol [Several evidences point to the importance of the IN in classical eyeblink conditioning. Thus, it has been early demonstrated that ablation of the lateral cerebellum and the electrolytic lesions of the dentate and interpositus cerebellar nuclei caused a near-complete abolition of the CR with no effect on UR , 142. Inmuscimol  in naivemuscimol . Electromuscimol –149. Nevmuscimol , 147, otmuscimol , 149. Thmuscimol , 150–152γ-isoform of protein kinase C, the PF-PC LTD was converted into long-term potentiation (LTP) [As mentioned before, the second cerebellar structure, where CSt and USt signals seem to convey, is the cerebellar cortex. PC constitutes the only output of the cerebellar cortex, modulating through its inhibitory synapse the activity of DCN . Althougon (LTP) . This alon (LTP) ), demonsA receptor antagonist gabazine and GABAA chloride channel blocker picrotoxin has been explored [Because DCN constitutes the only target for cerebellar cortex output, the pharmacological disconnection of these two structures seems to be an important approach for understanding their respective roles . In partexplored –165. Picexplored , 164. Inexplored  to a decexplored , respectexplored , 169 andexplored , 170. Muexplored . This reexplored , 170. Puexplored .As commented before, the cerebellum and its relation with the pontine nuclei have classically constituted the focus of attention in eyeblink conditioning studies even when a large number of brain regions are implicated in the performance of eyeblink . Thus, tComprehensive sensory information from the skin collected from the numerous specific types of sensors can be represented as a cutaneous envelope, which is the physical site of interaction between the body and the environment. This envelope is well represented in the cerebellar mantle, where sensory information conveyed both by the CF and MF is integrated. There is a functional relationship between this sensory envelope and the proprioceptive afferents from the underlying muscles. In turn, it has been hypothesized that the cutaneous envelope is functionally linked to the control of the related joint movements , and thiAs in the case of the control of eye position by the flocculus described above, elemental movements allowing the optimization of joint position are assumed by distinct cerebellar areas. One of the most studied circuits concerns the forelimb movement control in the cat. The functional organization of cerebellar regions contributing to this control has been studied with regard to input-output anatomical connections. Each cerebellar module has thus been shown to receive CF afferents from a specific IO region and projects the PC axons to a specific part of the DCN. Each cerebellar module  can be fThe cutaneous receptive fields of the CF projecting onto the C3 zone possess a detailed spatial organization divided into eight functional classes . The cutCerebellar LTD was first demonstrated in response to conjunctive stimulation of the vestibular nerve and the IO in decerebrated rabbit by Ito et al. , and in The same plasticity paradigm was reactivated more recently by Jörntell and Ekerot  and enriin fine the PC receptive field. A reversed effect could be produced at the same level when an LTD is induced at the PC reducing its inhibition on the Golgi cell, which can in turn inhibit the GC decreasing the PC receptive field.The main result was that unpaired burst stimulation of the PF beam durably transformed (~2 hours) the original restricted SS receptive field to a wide receptive field resembling the extended field described for the MF input . In contIn line with the functional proposal of the Swedish group, the possibility that the PC output could modulate the muscle tension  and the proprioceptive input to the MFs also needs to be revisited. Two main points of this experimental approach need to be discussed: (1) the unphysiological character of the PF stimulation paradigm and (2) the decerebrate preparation. Concerning the first point, Jörntell and Ekerot  have int2+-activated K+ (BK) channels [A receptors stabilization and clustering at postsynaptic densities of their inhibitory synapses [Still, the recent studies of our group , 201 shochannels  and on tchannels . This tisynapses . This clsynapses . The absConcerning the use of decerebrate preparation, the recent data of Huang et al.,  demonstrThe next step will be to fill the gap between animal and human studies about gating and learning of the central representation of the cutaneous envelope. For example, in human the N30 component of the somatosensory evoked potential following electrical stimulation of the median nerve can be suppressed when the stimulated hand is moving , 207 or


Normal human cells require a series of genetic alterations to undergo malignant transformation. Direct sequencing of human tumors has identified hundreds of mutations in tumors, but many of these are thought to be unnecessary and a result of, rather than a cause of, the tumor. The exact number of mutations to transform a normal human cell into a tumor cell is unknown. Here I show that male gonadal germ cell tumors, the most common form of testicular cancers, occur after four mutations. I infer this by constructing a mathematical model based upon the multi-hit hypothesis and comparing it to the age-specific incidence data. This result is consistent with the multi-hit hypothesis, and implies that these cancers are genetically or epigenetically predetermined at birth or an early age. Tumors originate from a single cell after the cell accumulates a series of mutations Direct DNA sequencing of tumors has established an upper limit on the number of mutations required to transform a cell. Sequencing of breast and colorectal cancers identified about 80 mutations in a typical tumor A lower limit on the number of mutations to transform a normal cell has been established in the laboratory. A human tumor cell was synthesized from normal human cells  by altering the expression of only three genes, which effected four biochemical pathways It is widely believed that colon tumors require four to six mutations Testicular gonadal germ cell cancers differ from other solid tumors in a number of ways. First, the incidence of gonadal germ cell tumors is highest at about 30 years of age and declines to just a handful of cases diagnosed in men in their 70's. In comparison, the incidence of many other solid tumors increases with age. Second, combination chemotherapy is particularly effective against testicular gonadal germ cell tumors as compared to other solid tumors. Finally, most solid tumors originate in somatic cells, while most testicular cancers arise in germ cells.The cause of testicular gonadal germ cell tumors is not known The strongest association of testicular cancer with any other medical conditions is with cryptorchidism, where the testicles do not descend into the scrotum at birth. About 5 to 10% of those who develop testicular cancer had undescended testicles at birth, compared to about 2 to 5% in the general population in utero or at an early age Two hypotheses exist for the origin of testicular cancers. The first suggests that testicular cancers are determined Tests of these hypotheses are mixed. A retrospective study of Swedish males found that those who underwent surgery before the age of 13 to correct undescended testicles had a slightly lower risk of developing testicular cancer than those who did not undergo surgery KITLG and SPRY4, that are known to play a role in testicular development. The estimated per allele odds ratio for these are among the highest found for any genome wide association study of a cancer Two genome wide association studies identified several mutations that predispose to the development of testicular tumors Previously, others have sought to understand the age-specific incidence of cancers with different approaches. Including using a Weibull distribution for lung cancer The objective of this paper is to determine how many mutations are required to develop testicular cancer. The approach is to compare the expected age-specific incidence, based upon the multi-hit model, with the measured age-specific incidence for testicular cancers.The age-specific incidence for testicular cancers is accurately described by Equation (1) with four mutations. I measured the probability of advancing to the next stage at The approach presented here implicitly assumes that the probability of advancing to the next stage, which could be associated with a mutation rate, is constant. Complexity could be added to the model by modifying this assumption. At least two different mechanisms could modify the mutation rate in a pre cancerous tissue. First, the mutator phenotype hypothesis suggests that one of the first mutations on the path to a tumor must result in a higher mutation rate The mutations are most likely chromosomal additions or deletions, not single base alterations. Cytogenetic studies of seminoma and non-seminoma testicular cancers have shown consistent alterations to several chromosomes. In particular, amplification of a region of chromosome 12p containing several known genes is often present Although four mutations are required for the development of testicular cancers, these mutations may alter more than four genes and biochemical pathways. In addition, other mutations that are not rate limiting may occur. Non rate limiting mutations would not alter the age-specific incidence data.One potential problem with this analysis is that it assumes no significant long term change in the rate of testicular cancers. The SEER-9 data show that the age-adjusted testicular cancer rate has increased by about 7% per year from 1973 to 2008. The standard way for dealing with temporal variation in cancer rates is to first analyze age models, then age plus drift, then age-period and/or age-cohort, and finally age-period-cohort models Additional complexity to the model could be added in different ways. To account for inherited mutations, the model could consist of two independent terms similar to Equation (1). The first term would require The agreement between the age-specific incidence data and Equation (1) implies that testicular cancers have a single potential progenitor cell. This contrasts with most other types of solid tumors which are thought to have many, many potential progenitor cells.The age-specific incidence data implies that testicular cancers are pre-determined before the age of 10 and possibly at birth either through genetic or epigenetic The multi-hit model describes a series of independent Bernoulli trials. A random number is drawn between zero and one. If the number is less than Under these assumptions, the probability distribution for the age at which testicular cancer is diagnosed should be given by the solution to the series of independent Bernoulli trials, the negative binomial distribution One assumption in the derivation of Equation (1) is that a single progenitor cell exists in the tissue. If many progenitor cells exist, as is thought to occur in most tissues, then cancer is diagnosed when the first of these many cells develops into a tumor. In this case, the first order statistic, or distribution of the minimum, of Equation (1) is the proper equation to compare to the age-specific incidence data. This would follow a Weibull distribution I tested the hypothesis that the age-specific incidence data on testicular germ cell tumors is accurately described by Equation (1). I performed a least squares fit to determine the parameters of the equation for all eight years in the dataset. Then, I calculated the reduced chi-squared value and the associated p-value, given the number of degrees of freedom, 53. The p-value, shown in In the United States, the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute (NCI) collects data on cancer cases. It is considered the gold-standard for data quality for cancer registries. It collects data from 17 different geographic regions that encompass just over 26% of the population of the United States I obtained testicular germ cell tumor age-specific incidence data using SEER*Stat (version 6.6.2) I compared Equation (1) to age-specific incidence data collected by the SEER-17 cancer registries from 2000–2007 on the incidence of testicular cancers, both seminomas and non-seminomas. I compared these equations with the data from ages 13 to 70 years old and with the number of mutations, I calculated error estimates by measuring the parameters


Irreducibility of the knee following complete dislocation is a rare event determined by the interposition of various capsulo-ligamentous structures in the joint space. Such cases often require urgent surgical treatment. We report the case of a healthy 70-year-old man with a sprain of the left knee that occurred after a sports trauma. The patient showed knee dislocation with multiple ligamentous injuries and articular block due to interposition of a portion of the vastus medialis muscle. After arthroscopic evaluation, we performed surgical treatment to free the muscle, regularize the medial meniscus and suture the posterior and medial capsule and ligaments; the cruciate ligaments were not treated. The most interesting aspect of the articular damage in this case was a wide detachment of the vastus medialis muscle with intra-articular dislocation. The decision to treat only the posterior lesions and allow the healing of the front ones by rehabilitation treatment was supported by full functional recovery and return to sports activity. Knee dislocation is a complex trauma often characterized by association of several capsulo-ligamentous lesions; its treatment remains controversial . For ratOther cases of knee irreducibility determined by interposition of various capsulo-ligamentous structures have been previously described. Nevertheless, to our knowledge, there are very few reports of interposition of the vastus medialis muscle.G.R., a healthy, sporty (skiing), 70-year-old man, had a sprain of the left knee with a valgus-external spin trauma after a crash against another skier at low–medium speed. He reported immediate pain and loss of function at the knee and was immediately taken to the emergency room.The clinical appraisal showed that the left knee was fixed in a valgus-flexed attitude, with pain on the medial compartment and a large skin depression on the medial side of the joint line Fig. . Severe The radiographic evaluation showed a subluxation of the knee with abnormal wideness of the medial joint space. The MRI showed anterior subluxation of the medial femoral condyle with intra-articular dislocation of the anterior-medial soft tissues. It also showed: complete lesion of both anterior and posterior cruciate ligaments (ACL and PCL), alteration and fragmentation of the medial meniscus (MM), rupture of the medial collateral ligament (MCL) and, finally, lateral subluxation of the patella. The MRI also showed, in the medial side, a muscular-density-like structure, which turned out to be intra-articularly located and to enter in contact with the medial femoral condyle Fig. .Fig. 2MRGiven the diagnosis of knee dislocation with multiple ligamentous injuries  and muscle interposition, surgical treatment appeared to be necessary.During the arthroscopy, a wide medial capsular lesion with intra-articular dislocation, a complex bucket-handle injury of MM and proximal disinsertion of PCL and ACL were observed. Specifically, dislocation of the vastus medialis was revealed Fig. .Fig. 3ArBecause it was impossible to reduce the dislocation of the vastus medialis by arthroscopy, a medial access arthrotomy was performed. It was thus possible to appreciate the following lesions: a distal complete laceration of the MCL, of the popliteal oblique ligament (POL) and of the posterior-medial capsule and a complete laceration of the meniscus-femoral ligament. Moreover, there was a dislocation of the vastus medialis muscle, whose distal portion was dislocated into the intra-articular space through the capsular lesion.Firstly, the release of the muscle belly from the notch was performed. This restored the mobility and the alignment of the knee joint and gave a better view of the capsular and ligamentous lesions. Then, selective partial regularization of the MM was performed. Finally, plastic suture of the posterior-medial and medial capsule, re-insertion of the POL with detached points and suture of the MCL with small anchor type “T-corkscrew” were performed Fig. .Fig. 4InAfter the surgery, the knee showed a correct alignment and the varus-valgus stress tests were negative. The rehabilitation program was immobilization with 20° of flexion for 15 days, then progressive recovery of range of motion in an articulated knee brace for 30 days. Partial weight-bearing was allowed after 15 days, and full weight-bearing after 45 days.We evaluated the patient at 6 months after the trauma through a clinical and instrumental evaluation with MRI and ultra-sonographic examination (US). The MRI showed the rupture of ACL, an altered morphology of PCL and a thickness increase with residual oedema of the MCL that appeared completely re-inserted Fig. . To obtaThe clinical examination showed a complete range of motion  with no lateral instability; Lachman and posterior drawer tests were minimal positive. This minimal instability was completely asymptomatic in the patient’s daily life. The quadriceps fully recovered after the rehabilitation. The patient walked, hiked, cycled and ran after 6 months and returned to alpine skiing after 10 months, as an active, healthy, 70-year-old man can do.During the most recent clinical examination, the patient gave his consent for publication of this case report.The interesting aspect of these types of knee dislocations is a wide lesion of the medial capsulo-ligamentous structures and their interposition between the femoral condyles, which causes the irreducibility of the dislocation itself. It is usually the result of a sprain in posterior-lateral sense. The presence of a cutaneous retraction on the medial joint line and eventual signs of cutaneous suffering are characteristic clinical signs , 8.The few cases described in the literature generally refer to the interposition of various medial capsulo-ligamentous structures, and their authors propose diverse surgical solutions without general agreement, particularly concerning the repair of ACL and PCL .The particularity of our case is a wide detachment of the vastus medialis muscle with intra-articular dislocation. This lesion clinically corresponded to an articular block with a forced valgus deviation of the limb and, specifically, to a cutaneous retraction at the level of the joint line. The intra-articular muscular interposition was detected by MRI and confirmed by the arthroscopy.The first arthroscopic step is advised in the literature, because it allows direct appraisal of the lesions and their eventual repair , but genTo our knowledge, very few cases presenting a similar condition can be found in the literature. Kilicoglu et al. in 2001 reported the case of a patient with an irreducible knee dislocation with interposition between the femoral condyles of the vastus medialis muscle. The clinical appraisal and the radiologic findings were similar to our case. In contrast to our approach, their surgical choice was to section the muscle to allow reduction of the knee dislocation and to perform the reconstruction of the ACL . SilverbThe difference in our case is that we chose to repair the rupture of the vastus medialis muscle and to relocate it in its anatomical site. This procedure allowed us to reduce the knee dislocation and to repair the medial capsulo-ligamentous lesions. In addition, in our case a lesion of the popliteus oblique ligament (POL) was also present and was repaired, in accordance with the literature . AnotherIn the end, what we have found in our experience is that, in this kind of lesion, lateral and rotational stability must be achieved immediately with reconstruction or suture of the ligaments, while reconstruction of the cruciate ligaments can be postponed and even avoided in elderly patients or those not engaged professionally in sports. In this case the rehabilitation protocol and the stability provided by the reinforcement of the muscles play a fundamental role.


In experimental and theoretical neuroscience, synaptic plasticity has dominated the area of neural plasticity for a very long time. Recently, neuronal intrinsic plasticity (IP) has become a hot topic in this area. IP is sometimes thought to be an information-maximization mechanism. However, it is still unclear how IP affects the performance of artificial neural networks in supervised learning applications. From an information-theoretical perspective, the error-entropy minimization (MEE) algorithm has newly been proposed as an efficient training method. In this study, we propose a synergistic learning algorithm combining the MEE algorithm as the synaptic plasticity rule and an information-maximization algorithm as the intrinsic plasticity rule. We consider both feedforward and recurrent neural networks and study the interactions between intrinsic and synaptic plasticity. Simulations indicate that the intrinsic plasticity rule can improve the performance of artificial neural networks trained by the MEE algorithm. Artificial neural networks with nonlinear processing elements are designed to deal with the troublesome problem of nonlinear and nonstationary signal processing. In a supervised learning problem, we are provided with a training data set containing the input, intrinsic plasticity (IP). IP is hypothesized to maximize the information capacity while maintaining an individual neuron's homeostasis of its mean firing rate level So far, experimental and theoretical studies on neural plasticity have mostly focused on synaptic plasticity, which is in accordance with Hebb's idea that memories are stored in the synaptic weights and learning is the process that changes those synaptic weights. Interestingly, recent experimental results have revealed that neurons are also capable of changing their intrinsic excitability to match the dramatic change of the level of received synaptic input synergistic information-theoretic learning. We use the resulting synergistic procedure for training feedforward neural networks (FNN) and recurrent neural networks (RNN) and test them on two benchmark applications. For simplicity and clarity, we focus on the prediction problem in the presentation, but the learning algorithm can also be used for solving problems of regression, classification and so on. Simulations indicate that Bell and Sejnowski's algorithm is appropriate for the proposed synergistic learning scheme and that the MEE algorithm combined with IP outperforms the MEE algorithm considered in isolation.The two plasticity mechanisms, intrinsic plasticity and synaptic plasticity, have been studied mostly separately. We are wondering how these two plasticity mechanisms would cooperate in artificial neural networks to learn complex mappings. To this end, we combine Bell and Sejnowski's information-maximization algorithm Studying the effects of intrinsic plasticity on various neural functions and dynamics relies on modelling intrinsic plasticity. In We apply this information-maximization rule as the intrinsic plasticity rule for artificial neural networks in this paper. Note that the original information-maximization rule in As mentioned above, the MSE criterion considers only the first two moments of the error distribution, thus it is ill-suited to non-linear applications in which the errors are not normally distributed. Recently, the error entropy (EEC) criterion based on ideas from information theory has been proposed as an alternative cost function for learning In applications, Renyi's quadratic entropy, We now introduce our synergistic information-theoretic learning algorithm, which is the simple combination of the IP rule of Eq. (2) and the synaptic plasticity rule of Eq. (7).From the perspective of information maximization, there are potential advantages of intrinsic plasticity in training artificial neural networks. For traditional weight update learning algorithms (synaptic plasticity rules), the activation functions of neurons are fixed during the training procedure. However, an invariant nonlinear activation function might be unsuitable for the input distribution. In an extreme case, the output of the neuron may be constantly found at saturation, and therefore carry very little information about the input. For real-world applications, the desired output distribution of a single neuron is far from these distributions with very low information. As we stated in the previous section, the information-maximization algorithm (the intrinsic plasticity rule) can adjust the shape of the activation function to match the input distribution and consequently increase the mutual information between the input and the output. Therefore, we hypothesize that the intrinsic plasticity rule might be beneficial to learning in artificial neural networks.The MEE algorithm requires several samples to accurately estimate the information potential. We therefore perform batch (epochwise) learning iterations, whereby the weights One may think that the effects of synaptic plasticity and intrinsic plasticity merely superpose in the learning process. In fact, we argue that they interact, which is why we call this combination “synergistic learning”. Indeed, the weight updating procedure affects the input of the neuron, and further influences the IP learning; the IP learning procedure affects the output of the neuron, and further influences the weight updating.It has been noted that, in reservoir networks, due to the incremental nature of intrinsic plasticity  as an example. As illustrated in The difference between the desired output, On the basis of the algorithm description in Choose a random set of small values for the The epochwise training procedure begins with Update the weight matrices Update the parameters If the stopping criterion is satisfied, the training procedure is stopped; otherwise, set In this section, we continue to study the proposed synergistic learning algorithm in a general class of recurrent neural networks The Following the approach of The proposed synergistic learning algorithm for the RNN is summarized as follows:Choose a random set of small values for the The epochwise training procedure begins with Let Repeat the calculation until Update the weight matrix Update the parameters As one epoch ends, if the stopping criterion is satisfied, the training procedure is stopped; otherwise, set The FNN and RNN are widely applicable to a set of problems such as regression and classification. As a typical example, we test the proposed synergistic learning algorithm on the single-step prediction of time series. For comparison, we also perform simulations for the MEE algorithm alone. The time series is denoted as The In the following simulations, the elements of the initial weight matrices The first simulation compares the learning curves of the MEE algorithm and the synergistic algorithm. In synergistic learning, the activation functions of neurons in both the hidden layer and the output layer are adjusted by IP. In this simulation, structural parameters of the FNN are set to In order to analyze the synergies between IP and synaptic plasticity in detail, input, output, and error distributions of neurons for the training set “MG” are presented. All these distributions are obtained by kernel density estimation in a single run, and results of two independent runs are similar. In order to explain the results clearly, we decompose the FNN into two parts, which are shown in Thus, the analysis on the results in The second simulation concerns the IP learning rate. As in the case of the FNN, we discuss how the recurrent neural network handles the problem of single-step prediction using the same data sets. The input of the network consists of The values of elements in the initial weight matrix are also randomly selected as small values uniformly distributed in . Results in this section are also averaged over 10 independent runs. The MEE learning rate is set to The first simulation compares the learning curves of the synergistic algorithm and the MEE algorithm. Structural parameters of the RNN are set to The second simulation concerns the initial IP learning rate. In some of the above simulations, we do not show the results for the data set “SS”, since they are similar to those for the data set “MG”. All activation functions used in the above-mentioned neural networks are tanh functions, thus the intrinsic plasticity rule for the tanh function is applied to the synergistic learning algorithm. In the case of using logistic functions, similar results can be obtained.Combining the MEE algorithm as the synaptic plasticity rule and the information-maximization algorithm as the intrinsic plasticity rule, we proposed a synergistic information-theoretic learning algorithm for training artificial neural networks. Whereas the information-maximization algorithm can increase the mutual information of a single neuron, it can not optimize the cost function such as EEC and MSE. Nevertheless, simulations have shown that this information-maximization-based IP rule benefits the artificial neural networks in both the convergence speed and the final learning result. As the IP rule adjusts the activation function of a single neuron to match its input distribution so that all output levels tend to appear equivalently, the input can be encoded much more efficiently and the discriminative ability of the neuron is enhanced. We believe that the discriminative ability of a neuron plays a nontrivial role in the performance of artificial neural networks. In terms of the FNN, the synergistic learning algorithm with IP only in the hidden layer or only in the output neuron still outperforms the MEE algorithm without IP, but is inferior to the learning algorithm with IP in both layers (we do not present these results in the paper).Compared with the original algorithm, the synergistic learning algorithm can be performed with a relatively small increase in computational cost due to the local nature of the IP mechanism and the simplicity of the information-maximization algorithm. In addition, we have used the efficient batch version of the information-maximization algorithm. In applications, a long training process is unnecessary since the improvement is minor at the end part of the training. For example, with a 300-epoch training, the IP rule is quite effective to improve the performance. In a long run, the synergistic learning maintains good performance.Advanced search methods for nonlinear optimization such as conjugate gradient algorithms and the Levenberg-Marquardt algorithm can be used to further speed up the learning process. In order to focus on the synergies between IP and synaptic plasticity and preclude influences of other advanced search methods on learning, we used the simple gradient descent (GD) method.In biology, Bell and Sejnowski's information-maximization algorithm can match the statistics of naturally occurring visual contrasts to the response amplitudes of the blowfly's large monopolar cell (LMC). The contrast-response function of the LMCs in the blowfly's compound eye approximates to the cumulative probability distribution of contrast levels in natural scenes, thus the inputs are encoded so that all response levels are used with equal frequency, resulting in a uniform output distribution As related work, several studies have combined synaptic learning algorithms with Triesch's IP rule