A new approach to the study of a relatively neglected property of mental architecture—whether and when the already-processed elements are separated from the to-be-processed elements—is proposed. The process of numerical proportion discrimination between two sets of elements defined either by color or by orientation can be described as sampling with or without replacement  depending on the possibility to tag an element once or repeatedly. All empirical psychometric functions were approximated by a theoretical model showing that the ability to keep track of the already tagged elements is not an inflexible part of the mental architecture but rather an individually variable strategy which also depends on conspicuity of perceptual attributes. Strong evidence is provided that in a considerable number of trials, observers tagged the same element repeatedly which can only be done serially at two separate time moments. In a seminal experiment, Sternberg The way mental processes are organized—their architecture—has been one of the main concerns for both psychologists and neuroscientists vs parallel controversy is that it cannot be resolved in isolation from other relevant attributes of the cognitive architecture. For example, it became evident that the questions about stopping rule – the conditions under which the system ceases processing and generates a response – or the questions about capacity limitations, are inevitably linked to the question about serial vs parallel architecture vs parallel processing in the context of enumeration accuracy of independent sets, e.g. vs parallel debate has almost entirely escaped the numerosity discrimination accuracy problem. At least one study has shown similar counting and subitizing processes to those measured in standard enumeration tasks to be involved in the number discrimination task with a single stimulus set One lesson that can be derived from the serial counting as referring to any process aimed at finding the total number of elements in a set. The term is neutral with respect to the temporal properties of the processes involved: counting can be parallel, serial, or mixed.In the following, we use the term It has long been known that it takes at least 5–6 years before children are able to learn all principles that are needed for counting, including assignment of numerals for objects vs parallel processing styles, we attempt to reveal this property of mental architecture on the basis of probability distribution of responses. Our approach stems from an ideal observer analysis which purpose is to establish an absolute scale of performance for an ideal perceptual device that is limited only by stochastic characteristics of the stimulus itself Ars conjectandi (1713/1899) devised an ingenious urn problem as an idealized mental exercise in which some objects or concepts of real interest  are represented as colored balls or pebbles which are drawn, one after another, randomly from the urn and their color is noted. Every probability textbook teaches that balls or pebbles once extracted can or cannot be returned to the urn, which leads to two distinct probability distributions for the number of balls of a given color: the binomial and hypergeometric distributions, respectively. These two different replacement schemes, however, have an important application to the problem of mental architecture. Provided that Bernoulli's urn model describes sufficiently accurately what happens in the perception of numerical differences, the scheme of sampling with replacement  implies that there is no tagging of which elements are already counted and which are not: the same element can, in principle, be inspected more than once. Consequently, if empirically determined psychometric functions for numerical discriminations between two sets of items are better described by binomial than hypergeometric distribution, it would provide evidence that some of these elements are inspected twice or more times which, understandably, can only be done serially at two or more different time moments. On the other hand, the scheme of sampling without replacement (leading to the hypergeometric distribution) implies that there is accurate one-to-one tagging of which elements are already counted and which are not, leading to an element being inspected only once, maximally. The attribution of one-to-one counting tags (corresponding to the sampling scheme without replacement) is by itself neutral to the problem of parallel or serial counting.To the best of our knowledge, there is no generally accepted method for establishing whether or not the tagging process follows exactly the one-to-one principle. Unlike many previous studies which have used analysis of reaction times to differentiate between serial K (the number of elements taken into account in the decision process) being equal to the total number of elements in the stimulus display, N, then he or she would always determine correctly which of the two types of the elements is more numerous. The fact that the real observers in our experiments make errors indicates, within the proposed approach, that either they only take into account proper subsets of the elements (adhering to the hypergeometric model with K<N) or they count some of the elements more than once, adhering, at least partially, to the binomial model. Our analysis below indicates that both these possibilities take place: to account for the data best we need to assume that the observers in some trials use the hypergeometric model and in other the binomial model, with K varying from trial to trial. In relation to the seriality vs parallelity of counting, the conformity of the data with the hypergeometric model  leaves the question of seriality vs parallelity open. But once the data are shown to require the binomial model for at least a fraction of all trials, one has to accept that some elements can sometimes be counted more than once, and this can only be done serially, at two or more separate time moments.If an observer strictly adhered to the hypergeometric model (see equations (3) and (4) in the The overall aim of the experiments was to introduce a new approach for the assessment of mental architecture, namely the property of whether, in the process of proportion discrimination of multiple stimulus sets, certain elements were being counted repeatedly. In our view, the aim was achieved by showing that this is indeed the case at least in some of the trials.The study has been approved by the local Research Ethics Committee.2. To diminish the impact of total red vs green area on the responses, size of the circles was randomly varied in the range of 11 to 22 minutes of arc. In the second series of the experiments a collection of short black line segments of luminance 0.3 cd/m2 and tilt of 20° either to the left or to the right from the vertical direction was presented. The width and length of a line subtended 2′ and 19′ respectively . Both types of stimuli were presented within an elliptical gray background with luminance of 54 cd/m2 and with lengths of horizontal and vertical axes 8.86° and 8.70° respectively. This elliptical background was in the center of a rectangular area of luminance 64 cd/m2 filling the rest of the screen. In order to avoid overlaps between elements, each element was positioned within an invisible inhibitory area which prevented other elements to be closer than 22′. Each stimulus element had a high contrast to guarantee its 100% identification would it have been presented in isolation. The total number of objects N presented on the display was kept constant through each experimental session and was equal either to N = 9 or 13 elements. These two relatively small values were chosen because the difference between the response probabilities from the binomial vs hypergeometric models is greater in case the total number of elements is small. During experimental sessions, the relative proportion of the type A and type B elements was varied. For example, for the total number of N = 9 the relative proportions of A (red or tilted to the left) and B (green or tilted to the right) element categories were the following: 1∶8, 2∶7, 3∶6, 4∶5, 5∶4, 6∶3, 7∶2, and 8∶1. The stimuli were presented at a viewing distance of 170 cm for 200 milliseconds, with 3 seconds for responding.Four 20-year-old female observers with normal or corrected to normal vision were asked to decide which of the two distinctive sets of objects were more numerous by pressing one of two buttons. In two separate series these two sets of objects were distinguished either by color or by orientation. A schematic view of the two types of stimulus configurations is shown in ViSaGe (Cambridge Research Systems Ltd.) stimulus generator. Every stimulus condition was replicated 100 times. Choice probability of the red circles was plotted as a function of the proportion of red elements NR in the total number of elements on the display N = NR+NG. Similarly in the orientation experiment, probability of the choice of the leftward tilted elements was measured as a function of the proportion of leftward tilted elements N(\) in the total number of elements on the display N = N(\)+N(/).All stimuli were generated on the screen of a Mitsubishi Diamond Pro 2070SB 22″ color monitor (frame rate was 140 Hz with the resolution 1024×769 pixels) with the help of a K from the binomial model are given by equations (1) and (2):The probabilities of a certain choice response for odd and even k is any positive natural number;p is the proportion of a certain type of elements to the total number of elements (either NA/(NA+NB) or NB/(NA+NB), depending on the experimental definition;K is the number of elements taken into account in the decision process.K from the hypergeometric model are given by equations (3) and (4):The probabilities of a certain choice response for odd and even k is any positive natural number;NA is the number of type A elements in the stimulus;NB is the number of type B elements in the stimulus;N is the total number of elements in the stimulus (N = NA+NB);K is the number of elements taken into account in the decision process.K because the probabilities given by a pair of equations  are equal, given equal values for k.As stated above, one only needs to consider either odd or even values of The obtained psychometric functions are shown in A and B are based on the inspection of K elements that are randomly selected from all available elements N. If the number of the A-type elements KA in the selection exceeds the number of B-type elements (KA>KB), then the response category “A” is chosen; in the opposite case the response category “B” is chosen. If the numbers of A and B elements happen to be equal (KA = KB) for an even number of selected elements K, then the choice between “A” and “B” response categories is random with probability 0.5. Following this simple decision rule it is easy to compute all theoretical cumulative probability functions for binomial and hypergeometric distributions. K (the sample size). One only needs to consider odd numbers of elements since K = 2k−1 (odd) and K = 2k (even) yield identical predictions. The equivalence of K = 2k−1 and K = 2k is easy to demonstrate numerically for any arbitrary k value or demonstrate their formal equivalence by using, for example, Wolfram's Mathematica. However, an analytic proof seems to go beyond ordinary algebra. The mathematical formulations of response probabilities from both types of models – binomial and hypergeometric – are given in the It is assumed that the observer's decisions between response categories K in each trial through the whole sequence of trials. It is more realistic to assume that the number of selected elements K is a variable and changes from one trial to another. Also, there is no clear reason to hold any one specific combination of theoretical models strictly superior to the others as, within error limits, many mixture models are able to provide a comparable fit. Therefore, the emphasis of the current analysis is to estimate the relative performance of the hypergeometric models to that of a combination of both hypergeometric and binomial models. We are greatly indebted to Ehtibar Dzhafarov for suggesting the described approach. At the heart of the underlying logic lies the assumption that in case any binomial component(s) is/are able to improve the overall fit of the mixture model  then that would be an indication in support of serial processing in at least some of the trials.Only in a few cases were the empirical psychometric functions close enough to one of these model predictions. This outcome is expected since it would be unrealistic to assume that the observer can use a fixed number of elements vs the empirical psychometric functions, the latter were shifted to the left or right to make their mean (μ) equal to 0.5. If the mean of all responses deviates from the expected 0.5 then it characterizes a response bias towards one of the two response alternatives. As expected, the empirical means were close to 0.5, ranging from 0.44 to 0.53.An approximation algorithm based on least squares optimization was written which looked for the weighted combination of all theoretical models which minimizes the sum of squared errors between theoretical predictions and points of empirical functions. Prior to plotting the best mixture of theoretical models K or hypK) indicates the percentage of trials in which each of these models is expected to be used. For example, in the first row in 31•hyp5+26•hyp7+15•hyp9+28•bin3, which means that for the observer S1 the best fit was obtained when the hypergeometric model with the sample size of either K = 5, K = 7 or K = 9 was supposed to be used in 31%, 26% and 15% of all the individual trials, respectively, and the binomial model with the sample size of K = 3 was used in the remaining 28% of the trials. Even a visual inspection can reveal that the fit to all 16 empirical psychometric functions shown in The best predictions of the mixtures of theoretical models are shown in N = 9 (N = 13 (The maximum number of components in the best fitting mixture models is four in case N = 9  and six  (N = 13  in orderIn general, it is known that numerical discrimination based on color is more efficient than one based on geometric attributes, such as orientation In both types of tasks the hypergeometric distribution provided a better fit than the binomial one: in 65.3% of all trials when applied to discrimination on the basis of orientation, and in 88% of trials when applied to discrimination based on color. It was not entirely surprising to discover some small individual differences since it was previously shown that some participants adhered to a serial processing profile in most conditions while other participants could exhibit parallel-like strategy in some conditions at least In order to enumerate objects accurately it is necessary to follow certain rules. One of these basic rules is the maintenance of the one-to-one relationship between objects and tags assigned to these objects: every object needs to be tagged only once. It is generally unknown whether and how well different perceptual processes are able to separate the to-be-counted items from the already-counted ones. In this study we have proposed a new approach to this problem. Although the question of whether and when people can perform perceptual and mental operations in parallel or in series has been dominating debates about mental architectures, it was also made clear that this central question can be answered only when other related questions such as stopping rules, selective influence The obtained evidence does not allow to assert that the adherence to the one-to-one tagging principle is an inflexible part of the mental architecture. Previous studies have shown that depending on the observer and stimulus conditions the parallel processing strategy can be used in some and the serial processing strategy in other situations The observed individual differences are particularly interesting in the light of a recent report showing that the ability to discriminate numbers of elements in two sets was correlated with a psychometrically measured intelligence


The advent of molecular biology techniques and constant increase in availability of genetic material have triggered the development of many phylogenetic tree inference methods. However, several reticulate evolution processes, such as horizontal gene transfer and hybridization, have been shown to blur the species evolutionary history by causing discordance among phylogenies inferred from different genes.To tackle this problem, we hereby describe a new method for inferring and representing alternative (reticulate) evolutionary histories of species as an explicit weighted consensus network which can be constructed from a collection of gene trees with or without prior knowledge of the species phylogeny.We provide a way of building a weighted phylogenetic network for each of the following reticulation mechanisms: diploid hybridization, intragenic recombination and complete or partial horizontal gene transfer. We successfully tested our method on some synthetic and real datasets to infer the above-mentioned evolutionary events which may have influenced the evolution of many species.Our weighted consensus network inference method allows one to infer, visualize and validate statistically major conflicting signals induced by the mechanisms of reticulate evolution. The results provided by the new method can be used to represent the inferred conflicting signals by means of explicit and easy-to-interpret phylogenetic networks. Molecular data have played an instrumental, and usually indispensable, role in many phylogenetic and evolutionary studies in the recent decades. Their increasing availability is due to outstanding advances in the development of fast, efficient and affordable sequencing technologies . AlthougAnother key factor that contributes to the incompatibility among gene trees is stochastic errors resulting from analytical features such as choice of optimality criterion, taxon sampling and sequence evolution model -14. ThesEvidence from many studies conducted on different groups of species, from fruit flies to hominids ,18-25, hconcatenation approach to elucidate the ancestral relationships among certain groups of species [Although there have been successful cases of using the  species -29, mult species ,30-32. T species ,34.consensus tree reconstruction methods is that each of the phylogenetic trees from a given collection of trees should contribute to a consensus tree according to the presence of its clusters. Among the most known and widely used consensus tree reconstruction methods are the majority rule consensus [The main idea behind traditional onsensus  and Nelsonsensus ,37. The onsensus ,37. The onsensus  and thenonsensus . Moreoveonsensus .consensus phylogenetic networks, Holland et al. [Phylogenetic networks should be used instead of consensus species trees or species trees inferred from concatenated sequences whenever reticulate evolutionary processes are studied ,5,11. Hed et al.  developed et al.  to visuad et al.  further SplitsTree which reconstructs an unrooted splits graph from a collection of phylogenetic trees through selecting all the splits that are present in more than a fixed percentage of all the trees [cluster network approach to build a phylogenetic network from a collection of gene trees using a modified tree popping algorithm which they called network popping algorithm. To estimate the support of any reticulation edge, the average support of that edge  is divided by the average support of the alternative reticulation edges located at the same position and weighted by the average support of all other tree edges [Prunier. Prunier needs a species tree and a gene tree as a reference and does not treat multiple gene trees. Prunier relies on a ranking of branches that are common to the species and gene trees based on the amount of conflicts that is reduced when the branch is removed. This amount of conflicts is a function that depends on the statistical support of the internal branches of the gene tree. For a detailed review of the existing phylogenetic network reconstruction methods the reader is referred to [In another attempt, Huson  and thenhe trees . Howeverhe trees  proposedee edges ,44. The ee edges  proposederred to . Mentionexplicit weighted consensus networks from a collection of trees , with or without prior knowledge of the species phylogeny. Such networks are capable of representing the main historical pattern of the species evolution  as well as the alternative evolutionary routes characterizing the species and genes under consideration. The main advantage of our method is that it allows for visualizing the species evolutionary relationships in a very clear and easy-to-interpret manner. Our algorithm takes advantage of the weights  assigned to the gene trees as well as the weights associated with the tree clusters  to infer the species dominant and alternative evolutionary histories. If a species tree is provided in addition to the collection of gene trees, our algorithm considers it as the dominant evolutionary history  and uses the collection of gene trees to infer the most significant reticulation events. If only a collection of gene trees is given, the new algorithm first builds a weighted consensus tree as the main evolutionary pattern and then infers the most significant alternative events.In this study, we present a new algorithm for the inference of The rest of the article is organized as follows. In the Methods section, a description of the basic concepts of phylogenetic networks and a detailed presentation of our new algorithm are given, followed by the description of the simulation protocol and the three considered real datasets. In the Results section, the results and performances of the new algorithm obtained for both simulated and real data are reported. They are then discussed in detail in the final section of the article.G  consists of a collection of vertices (V) which are connected by a collection of edges (E) in a pairwise manner. A path in a graph is a sequence of at least two vertices  such that, for all i ∈ {1, 2, …, k-1}, there exists an edge {vi, vi+1} in E. A cycle in a graph is a path whose first and last vertices are the same, while all other edges and vertices are pairwise distinct.A graph phylogenetic tree (T) is an acyclic connected graph whose leaves  are labelled according to the given set of taxa . Phylogenetic trees can be either bifurcating  or multifurcating . Phylogenetic trees can be rooted or unrooted, where the root is a node representing a common ancestor of all the species involved in the analysis.A phylogenetic network is a connected graph used either to visualize evolutionary relationships between species or to display conflicting evolutionary signals without such limitations as being acyclic or having a fixed indegree or outdegree of its nodes. Phylogenetic networks can be implicit or explicit: implicit networks such as split graphs are used to represent conflicting and ambiguous signals in a dataset using parallel sets of edges, rather than single branches. These networks often contain nodes that are not representing any ancestral species, hence providing only an implicit representation of evolutionary histories [explicit representation of evolutionary history of species . It is easy to prove that all the clusters satisfying such a rule will be pairwise compatible. For this, it will be sufficient to substitute each tree of the original tree collection by the set of its bootstrap replicates  and then apply the traditional strict majority rule method on this extended set of replicated trees. All the clusters appearing in more than 50% of the replicated trees will be mutually compatible.The main idea of our approach is that each phylogenetic tree from a given collection of trees should contribute to a consensus tree not simply by the presence, but also by the quality of its clusters . The quality of a cluster within a given collection of trees can be defined as the sum of bootstrap scores, taken over all the trees in this collection, of the internal branches associated with this cluster. The traditional majority rule consensus tree includes only the clusters that exist in more than 50% of the considered trees . Note thbootstrap-based extended majority rule method is a two-stage procedure. First, any cluster whose average bootstrap score is greater than 50% will be included in the consensus tree. Then, the method will consider the remaining clusters following the order of their total sums of bootstrap scores, computed over all the trees in the collection, and gradually add to the consensus tree those that are compatible with the current consensus tree until the tree is fully resolved or no more compatible clusters remains. For instance, tree Tbem in Figure T1, T2 and T3.Similar to the traditional extended majority rule method, as implemented in the CONSENS program of the PHYLIP package , the boobootstrap-based Nelson consensus tree each clique will be assigned a score equal to the sum of scores of clusters included in it. The score of each cluster is defined as a sum of bootstrap scores associated with this cluster, computed over the given collection of trees. Unlike the method described by Page [Tbem in Figure T1, T2 and T3), but this equivalence does not hold in general.We also consider the following extension of the traditional Nelson method. To build the  by Page , where o by Page ,46,47). T1, T2 and T3), each of them containing two internal branches with the associated bootstrap scores. The right-hand internal branch (connecting leaves “z” and “w” to the rest of the tree) has bootstrap support of 100% in all three trees. Therefore, it should be included in all consensus trees, or networks, regardless of the reconstruction method used. On the other hand, the left-hand internal branch connecting leaves “x” and “y” to the rest of the tree in T1 and T2 has different bootstrap scores in these trees (40 and 45% respectively). In tree T3, the left-hand internal branch connects leaves “x” and “u” to the rest of the tree. Its bootstrap score, 90%, is higher than the sum of bootstrap scores of the corresponding branch in T1 and T2. When using the bootstrap-based majority rule defined above, we obtain a consensus tree  / 3 = 28.3%. Tree Tbem is also the bootstrap-based Nelson consensus tree of T1, T2 and T3. Finally, the construction of the bootstrap-based consensus network  characterized by their bootstrap support. Network N123 is an implicit consensus network. In this article we will show how such an implicit network can be transformed into explicit one depending on the evolutionary mechanism being studied.In Figure weight-based extended majority rule consensus tree inference.The method we present and apply here also takes into consideration the weights associated with the given phylogenetic trees in addition to bootstrap scores of the tree clusters . Using one of the three equations presented in the section “Inferring weights”, the method defines a weight of each cluster based on the weights of the trees containing this cluster and on the cluster’s bootstrap scores in these trees. Then, after ranking all the clusters based on their weights, it regroups the compatible clusters starting from the top of the list, until a fully resolved consensus tree is built. This method is called here consensus network inference method accepts two types of input: 1) a species phylogenetic tree and a set of gene phylogenetic trees defined on the same set of species, or 2) only a set of gene trees defined on the same set of species. In phylogenetic studies, gene trees are usually characterized by their weights that reflect the quality of the reconstruction process. Such weights could be an average of bootstrap scores of the tree’s internal branches, a maximum parsimony or maximum likelihood score or a Bayesian posterior probability estimate. Thus, we assume that all the phylogenies have bootstrap scores or posterior probabilities (or any other measure of support) for their internal branches. Our algorithm first breaks down all the gene phylogenies into their relevant clusters and calculates a weight for each cluster based on Equations 1, 2 or 3 presented in the following section. Next, the algorithm ranks all the clusters based on their weights. For this type of input, our algorithm uses the species tree as the backbone of the network and gradually adds to it the highly ranked clusters  of the gene phylogenies. For the first type of input, the species tree is accepted as the dominant evolutionary history and the clusters of the gene trees are used to infer the reticulate  evolutionary events. For the second type of input, our algorithm reconstructs a consensus phylogenetic tree using the weight-based extended majority rule consensus tree method described above and then adds to it the remaining highly ranked incompatible clusters which are presented as reticulation branches. In the obtained consensus network, the weight-based consensus tree and the reticulation branches can be regarded as the main and alternative evolutionary scenarios, respectively.Our a correspond to processing the first type of input and those depicted by letter b are related to the second type of input. Steps 2 to 4 are common for both types of input.Regardless of the input type, the resulting representation is a weighted consensus phylogenetic network with a backbone tree structure and reticulation branches being chosen based on their weights which reflect their contribution to the clustering process. These two algorithmic facets are schematically presented in Figure O(n × m2 × (n + r)), where n is the number of gene trees in the considered gene tree collection τ, m is the number of leaves in each of these trees and r is the number of reticulation branches  added to the consensus tree. Note that the cluster inference procedure in Algorithm 1  has the time complexity of O(n × m2) as we use an optimal algorithm for the tree cluster inference, originally described by Makarenkov and Leclerc [for in Algorithm 1) has the time complexity of O(n2 × m2). The time complexity of the second loop while in this algorithm, where the reticulation branches are added to the consensus tree, is O(r × n × m2). The function find_direction in the same algorithm has the time complexity of O(n × m2). A group of clusters  is called compatible if altogether these clusters induce a unique phylogenetic tree. A cluster c has the first degree of incompatibility with a phylogenetic tree T if there exists an SPR (Subtree Prune and Regraft) move of the branches of T induced by the cluster c that transforms T into another phylogenetic tree. For instance in Figure xy) has the first degree of incompatibility with tree T3. In the same way, cluster (xyw) has the second degree of incompatibility with tree T3, as it requires two SPR moves  to transform T3 into a tree where cluster (xyw) is present. In the case of a directed phylogenetic network Nh inferred in Algorithm 1, cluster c will have the first degree of incompatibility with Nh if it has the first degree of incompatibility with the tree T obtained from Nh after carrying out all SPR moves corresponding to the reticulation branches included in Nh. Mention that in all the three presented algorithms we only need to know whether a given cluster c has the first degree of incompatibility with a given network Nh or not.We present here three network building algorithms , each of them being optimized for detecting and representing a specific evolutionary phenomenon. The first algorithm (Algorithm 1), which accepts the input of type 2 (a collection of gene trees inferred for various genes), is suitable for inferring either diploid or polyploidy hybridization events occurred among the observed species, or for finding recombination events occurred at the chromosome level. Algorithm 1 first proceeds by building the weight-based extended majority rule consensus tree followed by finding reticulation events and adding them to the consensus tree with proper direction in order to build the explicit weighted consensus network. The time complexity of Algorithm 1 is  Leclerc , in whicO(|SW| × (O(PhylInfMeth) + n × m2 × (n + r))), where |SW| is the cardinality of the set of MSA  fragments examined by the sliding window procedure and O(PhylInfMeth) is the running time of the phylogeny inference method used to infer the tree T from the MSA fragment MSAf.Algorithm 2, on the other hand, is designed to infer intragenic recombination events or partial horizontal gene transfers which lead to the creation of mosaic genes. This algorithm accepts two types of input . In cases where a species tree is provided, Algorithm 2 uses it as a backbone of the network. A sliding window procedure is then carried out for finding the aforementioned reticulation events and adding them to the backbone in order to build an explicit weighted consensus network. Otherwise, if only a multiple sequence alignment is given, a weight-based extended majority rule consensus tree will be built from it and used as the backbone of the network. The time complexity of Algorithm 2 is O(PhylInfMeth) + O(n × m2 × (n + r)).Our third algorithm (Algorithm 3) is intended for finding complete horizontal gene transfer events. It accepts as input a species tree in addition to one or more gene trees . Algorithm 3 uses the species tree as the backbone for the network and adds to it the most significant clusters  obtained after computing the weights of the gene tree clusters in order to build the weighted consensus horizontal gene transfer network. The time complexity of Algorithm 3 is The resulting phylogenetic network, regardless of the algorithm used, will be an explicit (in the sense that it represents exactly the assumed evolutionary mechanism) weighted and directed consensus network as shown in detail in Figure For each cluster from the set of the given gene trees, we have first to compute its overall weight. Every tree cluster can be associated with two types of initial weights, one being its proper bootstrap score or posterior probability in its tree of origin and another characterizing its entire tree of origin. In the case when the input contains only the weights associated with internal tree branches and lacks any measure of support for entire trees, we use the following equation to calculate the overall cluster weights:Wi(C) is the overall weight of cluster i, W(Cij) is the weight of cluster i in tree j and n is the total number of trees. If cluster i is absent in tree j, then σi equals 0, otherwise it equals 1.where Conversely, when the entire tree support is provided for each tree from the given set of trees but the input lacks individual supports for internal branches, we use the following equation to calculate the overall cluster weights:Wi(T) is the overall weight of cluster i calculated from the tree supports only, W(Tj) is the support of tree j and n is the total number of trees.where Wi, for each cluster i:Finally, when both cluster and tree initial supports are provided in the input, we use the following equation to infer the overall cluster weight, W(Cij) is the weight of cluster i in tree j, W(Tj) is the support for tree j and n is the total number of trees. These overall cluster weights will be used to build the consensus tree or network as described above.where cytochrome c oxidase subunit II of six species of honeybees (subfamily Apinae). The second one comprised eight chloroplast 16S rRNAs (920 nucleotides) from plants, algae and cyanobacterium. These two datasets are well-known and distributed with the SplitsTree program [rpL12e for 14 Archaeal species [We examined three evolutionary datasets to test the efficiency of our weighted consensus network inference method. The first dataset consisted of 677 bp nucleotide sequences of mitochondrial  program  among th species .We applied four different tree inference methods on both real and simulated (described in the next section) data to produce collections of gene trees. One representative from each of the four main tree reconstruction approaches . Let T1 be the tree obtained by an SPR move induced by reticulation branch r with direction d1  and T2 be the tree with r added to represent the gene transfer in the opposite direction . Then, the cumulative Robinson and Foulds distance is calculated between T1 and all the original gene trees containing cluster C = C1 ∪ C2, on one hand, and T2 and all the original gene trees containing C, on the other hand. Finally, the obtained cumulative Robinson and Foulds distances are weighted by the support of the original gene trees containing C as it is shown in Algorithm 1 (see the exact formula is in the function find_direction) and the resulting inequality indicates the direction of the horizontal gene transfer r.We applied these tree inference methods on both whole sequences and fragments of sequences (using a sliding window procedure) in order to search for alternative evolutionary events which might have affected either entire gene sequences  or only small sequence fragments . The latter events are usually ignored when analyzing entire genetic sequences during tree or network reconstruction. In the case of horizontal gene transfer events in Archaebacteria, we also computed the directions of complete and partial horizontal gene transfers using a dedicated function based on the Robinson and Foulds topological distance ; see theax, where the variable x was obtained from an exponential distribution (P(x > k) = exp (−k)), and the constant a was a tuning factor accounting for the deviation intensity. The value of a was fixed to 0.8. The random trees generated by this procedure had depth of O(log (n)), where n was the number of species .We generated sets of trees encompassing multiple reticulation features to test the efficiency of the proposed consensus network inference method in the context of recombination. First, random binary phylogenetic trees were generated using the procedure originally described by Kuhner and Felsenstein . The braSecond, we ran the SeqGen program  to generWe applied the BIONJ, DNAPARS, PhyML and MrBayes methods to infer the evolutionary history of the six honeybee species. The inferred trees are shown in Figure After breaking down the phylogenies into their clusters and calculating the cluster weights using Equation 3, we ranked all the clusters according to their weights and put together the compatible clusters to build the backbone of the consensus network based on the clusters ranks. Finally, we added the rest of the highly ranked clusters to the backbone tree to construct a weighted consensus network of the six honeybee species. In this analysis, we found one reticulation branch  in addition to the backbone (consensus tree). The explicit weighted consensus network built using Algorithm 1 is shown in Figure In this example, we used the same four tree inference methods as in the previous section to model evolutionary relationships among the eight plants from the chloroplast dataset. The application of these methods resulted in one maximum likelihood Figure A, one diWe, then, computed the weights of all the clusters presented in at least one of the seven phylogenetic trees using Equation 3. Finally, we built the backbone of the consensus network and added to it the reticulation branches after ranking the clusters as described in Algorithm 1.In this analysis, we found three reticulation branches which represent possible recombination events. The reconstructed weighted consensus network of the plastid 16 s rRNAs is shown in Figure rpl12e for 14 Archaebacteria species originally analyzed by Matte-Tailliez et al. [rpl12e. The species tree was used as the backbone topology to which we added the highly ranked incompatible clusters to build the weighted consensus evolutionary network encompassing a scenario of horizontal transfers of rpl12e. Using the cut-off level of 30%, we obtained five reticulation branches depicting alternative evolutionary histories , we assigned directions to all obtained gene transfer branches. In the case of Transfers 1 and 2  or incorrectly  identified intragenic recombination events.The results provided by Algorithm 2 (inference of recombination events using a sliding window approach) on simulated data are shown in Figures a Figure  or the na Figure . The resThe obtained results suggest that when the number of recombination events is small, they are more likely to be detected correctly. The best results in terms of both true and false positives were found for longer recombination fragments, i.e., 40 and 50% of the total length of the multiple sequence alignment. Another general trend is that the PhyML and MrBayes methods were much more effective in inferring the correct supporting tree and reticulation events than their BIONJ and DNAPARS counterparts. These results also suggest that it is much easier to detect recombination events in larger  phylogenies. Furthermore, the probability of finding the correct reticulation events increases as the width of the sliding window becomes closer to the real length of the simulated recombination fragment.Considering the results obtained for simulated data, we applied Algorithm 2 based on the sliding window approach and the two best tree inference methods (PhyML and MrBayes) to reanalyze the honeybee, chloroplast and Archaebacteria data described above. The purpose of this new analysis was to discover alternative evolutionary events of smaller lengths . Those mellifer and A. serana in Figure For the honeybee example, the PhyML and MrBayes methods allowed us to infer one and two possible recombination events Figures B and 5C,For the smaller-scale recombination events found using Algorithm 2 for the honeybee and chloroplast data, the intervals where they were detected are indicated between brackets in addition to their supporting weights , we can directly define it as the network support structure. Otherwise, averaging the tree clusters present in the given gene trees and using the consensus tree approach as the starting point for building the consensus network is a natural way of computing the support species tree structure in the absence of reliable additional information. The weights are used to take into account the tree cluster support when building an explicit phylogenetic network. The more gene trees we have, even when some of them are affected by different reticulation events, the more reliable the consensus network is. The most difficult practical situation for our method is when we have only a few gene phylogenies, most of which are affected by the same reticulation event. But there is no any network building method that will infer a correct explicit phylogenetic network is such a situation.We use both the discrepancy between the gene tree topologies (i.e. between the gene tree clusters) and statistical support of the gene tree branches in order to indentify the consensus network branches and reticulation events. Bootstrap scores or posterior probabilities of the gene tree clusters are constantly used to compute weights and thus to validate the selected network braches. The acceptance of some of the clusters and rejection of the other is determined by comparing the cluster weights to a pre-defined threshold. Indeed, like any other phylogenetic method, bootstrapping has its own pitfalls . HoweverMany studies supported by simulations advocate the use of probabilistic methods over distance- and parsimony-based approaches for inferring phylogenetic trees ,60,70. OA. cerana could be a closer relative of A. mellifer, compared to the backbone species phylogeny in which the closest relative of A. mellifer is A. dorsata . The opposite arrows entering into the A. cerana branch concern the intervals that have a very short overlap in both cases  we inferred when analyzing the chloroplast dataset was consistent with the findings of Makarenkov and Legendre . The mosrpl12e inferred using the BIONJ, DNAPARS, PhyML and MrBayes methods . The slhttp://www.info2.uqam.ca/~makarenkov_v/ConsensusNetwork.rar.In terms of visualization and results interpretation, our explicit network model is easily explicable, while the interpretation of implicit network models  becomes extremely difficult when dealing with a high number of species or conflicting events (see Figures The authors declare that they have no competing interests.ML and VM designed the methods. ML implemented them and carried out the simulations. PP-N and VM supervised the project and coordinated the development of the methods. All authors read and approved the final manuscript.


The size and composition of the T lymphocyte compartment is subject to strict homeostatic regulation and is remarkably stable throughout life in spite of variable dynamics in cell production and death during T cell development and immune responses. Homeostasis is achieved by careful orchestration of lymphocyte survival and cell division. New T cells are generated from the thymus and the number of peripheral T cells is regulated by controlling survival and proliferation. How these processes combine is however very complex. Thymic output increases in the first year of life and then decreases but is crucial for establishing repertoire diversity. Proliferation of new naive T cells plays a crucial role for maintaining numbers but at a potential cost to TCR repertoire diversity. A mechanistic two-compartment model of T cell homeostasis is described here that includes specific terms for thymic output, cell proliferation, and cell death of both resting and dividing cells. The model successfully predicts the homeostatic set point for T cells in adults and identifies variables that determine the total number of T cells. It also accurately predicts T cell numbers in children in early life despite rapid changes in thymic output and growth over this period. It is estimated to comprise at least 108 different T cell receptor specificities  rates for both the resting and dividing cell compartments. In this sense, the model can be considered as mechanistic in comparison to empirical or descriptive models where the parameters have no direct biological meaning. The results illustrate the importance of T cell dynamics for the maintenance of constant naive CD4 T cell numbers in adults and the growth of the T cell compartment in children.T cell homeostasis can be described using a two-compartment model of resting and dividing cells with input from the thymus into the resting compartment as shown in Figure X is the number of non-dividing (resting) T cells and Y the number of cells undergoing cell division  = X(t) + Y (t), i.e., the total number of T cells at time t.To develop this model, it is important to have biologically appropriate forms for each of these parameters. Thymic output is known to vary with age with a maximum at about 1 year, which then declines rapidly until about 20 years of age and more slowly thereafter , 18. Thels day−1 . This vals day−1 . An apprls day−1 λ=λ0exp−0 represents the intrinsic ability of a T cell to respond under conditions of no competition , ε is proportional to the amount of resource (IL7) available and N is the total number of T cells competing for the resource. The rate of entry into cell cycle therefore deceases exponentially with decreasing resource or increasing cell number. The rate at which dividing cells return to the resting state r is determined by the length of time taken for one division [known to be about 6 h  by Fas–Fas ligand interactions  and ρ is proportional to the amount of available resource providing the survival signal (IL7)  resting cells. As shown in Figure 11 cells, after 200–300 days  and stability did not depend on thymic output (θ), δ0, or ρ.The homeostatic set point for adults was examined by testing the behavior of the model starting with cell numbers well below and above the equilibrium and with an adult thymic output of 3 × 10ls day−1 . Initial11) resting cells, the proportion of proliferating cells (blue curve) increased very rapidly from 0 to 0.013 and then slowly declined over about 200 days to reach an equilibrium at about 0.5%. In contrast, under starting conditions of excessive T cells, the ratio of dividing to non-dividing cells increased rapidly at first from 0 to about 0.2% and then slowly to reach the same equilibrium of about 0.5%. This equilibrium point is consistent with a low level of cell division in the naive compartment of adult humans as reported previously  and cell survival (ρ) Figure . ConsistHaving established the behavior of the two-compartment model for naive CD4 T cell homeostasis in adults, we sought to determine whether it could also be used to explain the changes in T cell numbers that occur during childhood. During the first few years of life as thymic output changes and children grow, the naive CD4 T cell concentration decreases while the total number increases Figure . The que8 to 2 × 109 day−1 depending on age from 0 to 20 years, with a peak of 2 × 109 day−1 at about 1 year of age  of resting and dividing cells. Exponential forms were used for entry into cell division and death of cells in the resting compartment to represent the known competition for resources for cell proliferation and survival. Alternative functional forms for density dependence may be worth exploring in the future. The model is a mechanistic model based on the known biology of naïve T cell homeostasis so that the different parameters all have a biological interpretation as indicated in the methods. An alternative but non-mechanistic mathematical model of T cell homeostasis has been described , which dWhen T cell export from the thymus was kept constant to represent a young adult, simulated T cell numbers converged from either low or high initial levels to a stable homeostatic equilibrium consistent with cell numbers in a normal, healthy adult. This is concurrent with the increase in T cells observed in response to lymphopenia and the decrease following T cell expansion after infection , 17. ConThe results obtained by altering parameter values gave a clear indication of the effect of the different rates of cell death and proliferation. The corresponding expected rise and fall in the set point of the T cell pool was reassuring. This set point appeared to be more sensitive to increments in the death rate of resting cells than to increases of the same order in the activation rate. Importantly, the T cell numbers at equilibrium decreased as the resource term for entry into cell division (ε) or the resource term for rescue from cell death (ρ) decreased although the equilibrium was less sensitive to changes in the resource required for survival Figure . The senThe other question addressed by the model was whether the incorporated biological mechanisms were in themselves sufficient to explain the known decrease in naive CD4 T cell concentration over the first few years of life when T cell export from the thymus increases to a maximum at 1 year of age and then declines, and the child is growing in size with an accompanying increase in blood volume Figure . The modIn conclusion, we have presented a mechanistic two-compartment model of naive T cell homeostasis based on the known biology, which reproduces results obtained by other methods with good accuracy. It is likely to be an appropriate model for investigations of T cell reconstitution and homeostasis in diseases such as HIV, in patients given bone marrow transplantation and even for understanding reconstitution after thymic transplants for athymic patients with DiGeorge syndrome.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


When faced with the decision of whether to rely on ring vaccination, the public health community should be aware that an epidemic might take time to subside even for an eventually successful intervention strategy.We present a stochastic model for the spread of smallpox after a small number of index cases are introduced into a susceptible population. The model describes a branching process for the spread of the infection and the effects of intervention measures. We discuss scenarios in which ring vaccination of direct contacts of infected persons is sufficient to contain an epidemic. Ring vaccination can be successful if infectious cases are rapidly diagnosed. However, because of the inherent stochastic nature of epidemic outbreaks, both the size and duration of contained outbreaks are highly variable. Intervention requirements depend on the basic reproduction number  We investigated how effectiveness of ring vaccination depends on the time until diagnosis of a symptomatic case, the time to identify and vaccinate contacts in the close contact and casual contact ring, and the vaccination coverage required to contain an epidemic.We investigated which conditions are the best for effective use of ring vaccination, a strategy in which direct contacts of diagnosed cases are identified and vaccinated. We also investigated whether monitoring contacts contributes to the success of ring vaccination. We used a stochastic model that distinguished between close and casual contacts to explore the variability in the number of infected persons during an outbreak, and the time until the outbreak is over. We derived expressions for the basic reproduction number  lasts 12–15 days (s period . At the τ(1), and in the casual contact ring this number follows a a negative binomial distribution with mean μτ(2). The values (μτ(1) = 2 and μτ(2) = 14.9) were chosen such that the total number of contacts per day was comparable to numbers observed in empirical studies (Transmission takes place in two rings of contacts: 1) household and other close contacts, and 2) more casual face-to-face contacts. We assumed that in the close contact ring the probability of transmission is five times higher than in the casual contact ring (g = 0.2). The number of contacts on day τ of the infectious period in the close contact ring follows a Poisson distribution with mean μ studies . For eve,Most people can be infected again, and smallpox can develop 10–20 years after vaccination  all contacts of the diagnosed patient. In our baseline scenario, we assumed that vaccinated contacts are not isolated after vaccination and may therefore transmit the infection to others if they become infectious. In addition, we enhance the baseline intervention by including monitoring of identified contacts. The effectiveness of the intervention therefore is determined by the probability of diagnosis per day of the infectious period, the time needed to identify contacts of the close contact and casual contact ring, the vaccination coverage in the close contact and the casual contact ring, and whether monitoring of contacts is performed. Some of those parameters (speed of diagnosis and time to identifying contacts) differ between the first index case in the population and cases occurring later in the epidemic. In τ the probability of diagnosis on day τ of the infectious period for those persons who have not been diagnosed before. From those probabilities, one can derive the probability that an infectious person is not yet diagnosed on day τ of his or her infectious period (τ(i), we denote the probability that a contact in ring i (i = 1 or 2), who was infected on day τ of the index patient’s infectious period, will be vaccinated within 4 days of being infected. In the τ depends on the diagnosis probabilities, the time needed for contact tracing, and the vaccination coverage. Throughout, we assume that the vaccine efficacy is 100%. We can now determine an effective reproduction number υR that describes the number of secondary cases caused by an index patient in a situation with intervention:We denote with δs period . By υτ(i(i)c is set to zero. Equivalently, it describes the situation that no window period exists . Therefore, 0R can be computed with the formula including vaccination, where the window period is now set to w = 15, of the full duration of the infectious period. This assumption means that regardless of when the index patient is diagnosed, contacts can effectively be excluded from further transmission. If monitoring of contacts is not 100% effective, the parameter (i)c for the coverage can be used to express the extent of successful monitoring.If vaccination is ineffective, but there is monitoring of contacts, the monitoring will have the same effect on υR < 1. In the Table, the model parameters and their baseline values are listed. In the The outbreak can be controlled if υR to below 1, the epidemic will continue to grow exponentially, albeit at a lower rate. For example, if diagnosed infectious persons are isolated, but no ring vaccination is performed, the effective reproduction number is υR=1.65 and the epidemic cannot be contained.An epidemic starting with one index case in a completely susceptible population without intervention grows exponentially, if it survives early extinction (minor epidemics). The large range of possible courses of the epidemic reflects the stochastic variability . If the υR to <1, the size of successive generations of infected persons declines. For the parameter values of the baseline scenario given in the Table, we have υR = 0.67 and 94.4% of all transmissions take place in the casual contact ring. The epidemic can then be contained and the virus eradicated. In w = 15), the effective reproduction number can be further reduced to 0.53. The average number of infected persons drops to 6 (range 1–43 persons), excluding the identified infected contacts who are vaccinated or monitored and to 14 (range 1–96 persons), including the vaccinated and monitored infected contacts. The mean time to extinction is now 53 days (range 20–288 days). The fraction of transmissions taking place in the casual contact ring is slightly lower. at 93.7%.If the intervention succeeds in reducing the effective reproduction number 0 = 10.46, i.e., twice the value of baseline scenario. To contain the epidemic, we now assumed that 80% of all contacts in the casual contact ring were vaccinated in time. The effective reproduction number υR was 0.91. A fraction of 91.8% of transmissions took place in the casual contact ring. The mean number of infected persons during the epidemic was 101 (range 2–663), excluding the infected contacts vaccinated in time and 340  including those contacts. The mean time to extinction was 229 days (range 32 to >900 days). The time to extinction can be very long when υR is near 1 because the epidemic can flare up again when a case by chance produces many secondary infections. A similar picture would result if, in addition to vaccination, contacts are monitored. However, if vaccination coverage is 55% in the casual contact ring, the effective reproduction number is 0.96; 93.0% of transmissions are in the casual contact ring.To contrast the baseline scenario, in υR of 0.67  is determined by the number of index patients that start the epidemic outbreak and by the time it takes to diagnose the first case. We varied those two variables separately while assuming that after diagnosis intervention took place within the parameters defined in the Table, i.e., with an  of 0.67 . While t of 0.67 . Diagnos0R determines whether ring vaccination as defined above can contain an epidemic or not. As the value of 0R is uncertain . υ.R The mean time until extinction approximately doubles to almost 200 days, and the range of possible outcomes increases with maximum possible durations of >2 years. The mean number of infected persons increases by a factor of 5, and the range of possible outcomes increases such that epidemics with several hundreds of infected persons are possible. Hence, if υR is slightly <1, the epidemic might take a long time to control, and the number of persons who become infected and die might be high.Finally, we looked at how differences in intervention effectiveness influence the duration of the epidemic and the cumulative number of infected persons. The effective reproduction number Our simulation results show that a smallpox epidemic starting from a small number of index cases can be contained by ring vaccination provided the intervention measures are very effective. The time to diagnosis has proven to be an essential and sensitive parameter in determining the intervention effectiveness. The speed of diagnosis is less essential if identified contacts are isolated to prevent them from transmitting further if their vaccination fails. The time window limiting the success of vaccination then loses its importance for determining the effectiveness of intervention. The time to diagnosis of cases and the fraction of contacts found by contact tracing are then the key parameters. Contact tracing would be even more essential if substantial transmission would take place during the prodromal period of infection as is assumed by some authors (Some limitations of our modeling approach should be kept in mind. First, we only consider epidemics that are started by a small number of index cases. The branching process approach does not allow for overlapping rings of contact, but we implicitly include such an effect by varying the effective transmission probability such that the distribution of transmissions over the infectious period agrees with empirical findings (0R, Meltzer et al. (In the recent literature, other models, both stochastic and deterministic, of smallpox outbreaks have been introduced to analyze the effects of ring and mass vaccination (0R, and thus makes sensitivity analysis of the results based on that quantity much more tedious. One of the main differences, however, is the way that vaccination is incorporated into the model does not allow investigation of the effects of those parameters that largely determine the success of intervention, namely the time to diagnosis of new cases and the time needed to trace contacts. In a small closed community as the one described in the model, tracing of contacts is not that difficult, but in modern society with its increasing mobility the tracing of casual contacts can pose a big problem.In a study by Halloran et al. (0R and υR are set to prescribed values, while in our model those numbers can be derived from measurable quantities inherent to the transmission and intervention process. In comparison to the estimate of υR of 0.53 in our baseline scenario with vaccination and monitoring of contacts, Bozette et al. assume a much lower value of 0.1. So compared to our results, their results are rather optimistic, but they cannot relate the assumed value of υR to 0R or to parameters describing the process of contact tracing and vaccination.The main difference between the modeling approach of Bozette et al. (Finally, Eichner (With respect to preparing for a smallpox outbreak, alertness and ability to diagnose quickly are important. Physicians and nurses need to be educated and the public needs to be more aware. Also, since we know little about the timing and effectiveness of identifying infectious persons and their contacts in case of a bioterror attack, obtaining more empirical information about contact patterns and contact tracing will be helpful. Recently, some useful data about contact patterns have been collected during severe acute respiratory syndrome outbreaks, but a more systematic investigation of contact tracing is advisable. Considering the uncertainties connected to all parameter values, we conclude that any contingency plan for use of ring vaccination must also identify the criteria under which switching to large-scale mass vaccination is justified.Formal Model Definition


The vast majority of published work in the field of associative learning seeks to test the adequacy of various theoretical accounts of the learning process using average data. Of course, averaging hides important information, but individual departures from the average are usually designated “error” and largely ignored. However, from the perspective of an individual differences approach, this error is the data of interest; and when associative models are applied to individual learning curves the error is substantial. To some extent individual differences can be reasonably understood in terms of parametric variations of the underlying model. Unfortunately, in many cases, the data cannot be accomodated in this way and the applicability of the underlying model can be called into question. Indeed several authors have proposed alternatives to associative models because of the poor fits between data and associative model. In the current paper a novel associative approach to the analysis of individual learning curves is presented. The Memory Environment Cue Array Model (MECAM) is described and applied to two human predictive learning datasets. The MECAM is predicated on the assumption that participants do not parse the trial sequences to which they are exposed into independent episodes as is often assumed when learning curves are modeled. Instead, the MECAM assumes that learning and responding on a trial may also be influenced by the events of the previous trial. Incorporating non-local information the MECAM produced better approximations to individual learning curves than did the Rescorla–Wagner Model (RWM) suggesting that further exploration of the approach is warranted. Objectively, associative learning theory is a thriving enterprize with a rich tradition of experimental work interpreted through the lenses of sophisticated mathematical models. However, there remains a fundamental empirical observation that is still not well captured by these models. Despite many attempts to provide an adequate account of the learning curves that are produced, even in a simple conditioning experiments, there is still considerable unexplained variation in these curves. For example, many formal models of learning lead us to expect smooth learning curves but these are seldom observed except at the level of average data. Small departures from a theoretical curve can be tolerated as measurement error but when this error is large the model must be called into question and some authors have concluded that associative models are fundamentally wrong. An alternative position, the one adopted in the current paper, is that the associative framework is essentially correct. However, it is argued that much more accurate modeling of individual learning curves is needed and can be achieved by using a more detailed representation of the stimuli provided by the learning environment. In what follows I will describe the application of a mainstream model of associative learning, the Rescorla–Wagner Model  and the representation of the outcome (such as a shock US) that occurs on a single learning trial. ΔV is a function of two learning rate parameters, α for the CS and β for the US, and the parenthesized error term. In the error term λ is the value of the US on that trial  and ΣV is the summed associative strength of all the predictors that are present on the trial. The RWM is said to be error driven and competitive. It is error drive in the sense that the amount of learning depends on the difference between what occurs, λ, and what was expected, ΣV. It is competitive in the sense that the updates applied to the associative strength of a stimulus depend not just on the strength of that stimulus but also on the strength of all the other stimuli that are present on the trial—ΣV is used in the error term rather than V alone. This competitive error driven formulation is a defining feature of the RWM and has been adopted in many neural network models of learning  is the fundamental RWM learning equation. In the equation ΔP (Correct Response) = 1−αt−β where t is the trial number in the series, α and β are parameters of the curve. An equation of this type generates a curve in which the proportional progress toward asymptote declines with trials. In contrast, an exponential function P (Correct Response) = 1 − αet−α generates a curve in which the proportional progress toward asymptote remains constant with trials. Although there is now doubt about the status of the Power Law  we can see that associative strength increases as a fixed proportion (αβ) of the difference between the current associative strength and the asymptote. From the RWM we therefore expect orderly negatively accelerated learning curves. Because each theory of learning makes characteristic learning curve predictions, in principle, analysis of learning curves should be theoretically decisive. Unfortunately, the utility of this approach has not been realized because of the empirically observed heterogeneity in learning curves. Smooth monotonic, S-shaped, and stepped curves have all been seen at one time or another leading Mazur and Hastie to comment “In fact, learning curves of almost every conceivable shape have been found.” , f(j), … f(n)) ≠ f)  provides the learning equation used in MECAM. There is no difference between the RW and MECA models in the way associative strength updates are made except that in the MECAM the additional parameter ω is combined multiplicatively with the learning rate parameters α and β (compare Equation (1) and Equation (2)). The value of ω is allowed to vary for each cue according to the buffer in which the cue is defined. Primary buffer cues have ω = 1 whereas for secondary and interaction buffers 0 ≤ ω ≤ 1. Further details are provided in the Appendix and below there follows a short outline of MECAM's operation.V. In this way the contents of all three buffers contribute to the outcome expectation for the trial. Second, the associative strengths of the cues present in all three buffers are updated. The cues present in the primary buffer are always just those that occur on the current trial  whereas the secondary buffer contains a copy of all of the stimuli that occurred on the previous trial. These remembered stimuli have their own representations and associative strength. Thus, stimuli A and At − 1 are distinct entities, as are ab and at − 1bt − 1. Because the outcome of the previous trial is just as likely, if not more likely, to be remembered than the cues, the previous trial outcome is also coded as one of the remembered stimuli in the secondary buffer (Ot − 1). The interaction buffer encodes a subset of the configural cues that are processed by MECAM. This subset consists of pairwise configurations of the elements of the current trial and the remembered elements from the previous trial. In the Trial 2 example shown in Table t − 1, Bt − 1, and Ot − 1 from the previous trial. This results in nine configural cues appearing in the interaction buffer. The use of three buffers allows different ω weights to be used for different classes of stimulus entity. The third trial illustrated in Table Table The MECAM is predicated on the assumption that the source of the behavioral complexity in individual learning curves is to be found in the environment to which the participants are actually exposed. A corollary is that even if the RWM is correct in its basic principles then simulations of individual participant behavior using the RWM will be inaccurate unless the input representations for the simulation match those in the individual's learning experience. The MECAM hypothesis is that during learning some of the influences on participant responding will be due to learning of associations between trial outcomes and memories of events occurring on previous trials. If this is correct then MECAM simulations, which incorporate representations of the previous trial events as inputs to the learning and expectations for the current trial, would provide better approximations to individual learning curves than the RWM, which involves learning and expectations only for current trial events. The experiments reported below involved participants making judgements about the likelihood of an outcome in each of a series of trials. Participant responses were in the form of ratings on an 11-point scale, running from 0—event will not occur, through 5—event will/will not occur with equal likelihood, to 10—event will occur. However, these judgements are not represented directly in either the RWM or MECAM. The currency of these models is the unobserved theoretical quantity of “associative strength.” Therefore, to model the changes in these judgements during learning it was necessary to find an appropriate way to map between the theoretical quantity of associative strength and observed judgements.Unfortunately there is little agreement on the specific mapping between association strength and behavioral response  college in Hampshire, UK. Participation was voluntary. One hundred and forty-four participants took part in Experiment 2. Their average age was 22 years and they included 41 males. They were recruited from the student and staff at the University of Wales Swansea campus and were paid £3 for participating.In Experiment 1 participants were tested in groups at three computer workstations housed in a mobile research laboratory set up in the load compartment of a specially equipped Citroen Relay van. To minimize interference between participants auditory stimuli were presented over headphones and seating was arranged so that participants could easily view only their own computer screen. The screens measured 41 cm × 26 cm (W × H) and were run in 32 bit color mode with pixel resolutions of 1440 × 900. The display was controlled by a computer program written in Microsoft Visual Studio 2008 C# language and used XNA Game Studio Version 3.1 for 3D rendering of the experimental scenario. In Experiment 2 participants were tested individually in small experimental cubicles with sounds presented over the computer speakers. The screens measured 28 cm × 21 cm (W × H) and were run in 8 bit color mode with pixel resolutions of 640 × 480. The display was controlled by a computer program written in Borland Turbo Pascal.In all experiments participants were given a brief verbal description of the procedure before reading and signing a consent form. Next, a more detailed description of the procedure was presented on-screen for participants to read. In Experiment 1 the on-screen information was given along with a voiceover of the text, played through the headphones. The text from Experiment 1 is reproduced in full below. The text used in Experiment 2 had minor wording differences but conveyed the same information.Thank you for agreeing to take part in this experiment. During the experiment you will be shown a series of “playing cards” on the computer screen. The cards were played in a game at Poker Faced Joe's Casino. The experiment is divided into a series of trials, each trial representing one card game. On each trial you have to rate the likelihood that the cards on the screen will WIN or LOSE. Make your rating by adjusting the indicator using the UP and DOWN arrow keys. When you have made your rating press RETURN. When you press return the cards will be turned over and you will find out whether they win or lose. Your job is to learn what outcome to expect. At first you will not know what to expect so you will have to guess. However, as you learn, you should aim to make your predictions as accurate as possible, to reflect the true value of the cards that are in play. Review these instructions on the screen. When you are sure that you understand what is required, press the key C to continue. Please note, Poker Faced Joe's is an imaginary casino you will not lose or gain any money by the rating you make. However, please try to make your judgements as quickly and as accurately as you can. Ask the experimenter if you have any questions or press the key C to begin.n = 73 color predictive and n = 71 symbol predictive). In both experiments trials AX+ and AY+ were reinforced trials and were followed by the “win” outcome after participants made their judgements. Trials BX− and BY− were non-reinforced trials, and were followed by the “lose” outcome after participants made there judgements. Outcome feedback was in the form of onscreen text “win” and “lose” accompanied by distinctive auditory signals.After reading the instructions participants initiated the experimental trials with a key press. There then followed a series of trials. Each trial was one of four types; AX+, AY+, BX−, or BY−. In Experiment 1 participants had eight of each trial type presented in a random order, with order randomized for each participant subject to the constraint that no more than two trials of the same type could occur in sequence. The symbols and colors serving the cue functions A, B, X, and Y were selected at random for each participant from a set of 14 symbols and a set of 13 colors . The background colors were allocated to role of informative cues (A and B) and the symbols allocated to the role of redundant cues (X and Y) in an approximately counterbalanced fashion so that 30 participants had colors in the A, B roles and foreground symbols in the X,Y roles; vice-versa for the remaining 31. In Experiment 2 participants had four trials of each type presented in one of five different orders, each order randomized subject to the constraint that no more than three trials of one type could occur in sequence. Four different symbols and three different colors were used. Allocation of colors and symbols to the role of informative (A and B) and redundant (X and Y) cues was approximately counterbalanced  measures of goodness of fit. This was done to provide comparability between Experiment 1 and Experiment 2. This was necessary because Experiment 1 had 32 learning trials whereas there were only 16 trials in Experiment 2. Thus the SSD values for Experiment 1 were larger than those in Experiment 2. Because RMS error is the average error over all data points RMS magnitude is not directly affected by the length of the trial sequence.Figure Figures F = 2.28, p < 0.001] of Block (1–16) × Reinforcement (non-reinforced “v” reinforced) × Group  confirming that the development of the discrimination between non-reinforced and reinforced trials differed according to the model goodness of fit.Figure F = 2.30, p < 0.001]. In contrast to the Fechner based model, the model responses on the non-reinforced trials decline over trial blocks.Turning to Figure t-tests on the RMS error showed that the mean RMS fit was significantly better for the Stevens Response Model than for the Fechner Response Model . The mean RMS error values are given in Table F = 3.93, p < 0.001] and the Stevens response model also produced significantly better fits for the data of Experiment 2 than did the Fechner model .Student's F = 1.24] nor in Experiment 2 [F = 1.50]. In addition to providing better fits overall the MECA Model also produced less stable responses from trial-to-trial and it is in that sense a better approximation to the responses produced by the participants. In many cases the trial-to-trial variation in the model predictions does not covary with the participant responses but in a number of cases there are striking correspondences  = 5.68, p < 0.001 and t(143) = 5.88, p < 0.001, respectively]. The RMS error values are given in Table Although the RWM captures the general trends in the data, particularly when using Stevens response mapping, consideration of the individual data in Figures For Experiment 1 there was an improvement in the RMS error value for the MECA Model over the RW Model in 43 out of 61 cases—70% of participants has better fits using the MECAM, the median improvement value was 0.11. In Experiment 2 the median improvement value of MECAM over the RWM was also 0.11 with the MECAM producing smaller RMS values in 89 out of 144 participants—62% had better MECAM fits than RWM fits. Figure F = 1.61, F(9.195) = 0.84, and F = 1.42, respectively]. MANOVA did show a difference between experiments when the RWM with Fechner response mapping was considered . Follow-up t-tests using Welch's correction produced significant differences only for the response mapping parameter c. Lower values of c were found in Experiment 1 than in Experiment 2 .Multivariate Analysis of Variance (MANOVA) was used to compare Experiment 1 and Experiment 2 to assess whether or not the fitted model parameters differed for the two datasets. The parameter values for Experiment 1 and Experiment 2 did not differ in three out of the four cases. The parameters were the same in both datasets for the MECA Model with Fechner response mapping, and for the MECAM and RW Models with Stevens response mapping [approximate Fs n − 1 memory stimuli serve to add noise to the observed responses. Because these stimuli are unlikely to remain predictive for long sequences of trials they will tend to lose their influence toward the end of the trial sequence.Two principle findings emerged. First, in this model fitting exercise, better results were obtained by using a mapping between associative strength and response strength based on Stevens' Power Law than by using a mapping based on Fechner's Law. The average model predictions using the RWM and Stevens response mapping differed from the participant data by 2.82 (Experiment 1) and 2.60 (Experiment 2) units on an 11 point response scale. In comparison the same figures for the Fechner response mapping were 3.24 and 2.91  has not been fulfilled. Several factors have contributed to the difficulties including using multicomponent tasks and problems with averaging  and the other model representing the current and previous trial (the MECAM). Nevertheless we can assert that configural cues are important by looking at the optimized values of the interaction buffer weight in Table The inclusion of configural cues in the MECAM may seem questionable because there is no requirement that participants use configural cues to respond appropriately in the tasks used. Whilst some studies have shown that the weight attached to configural cues can be increased by experience  and, because these cues are typically not predictive for long trial sequences, the influence of these cues adds noise to the observed learning curves. The improvements made by the MECA Model over the RWM suggests that this assumption is reasonable and the cue-structures defined in the current investigation are offered as an initial approximation subject to further investigation.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Interest in instrumental learning in earthworms dates back to 1912 when Yerkes concluded that they can learn a spatial discrimination in a T-maze. Rosenkoetter and Boice determined in the 1970s that the “learning” that Yerkes observed was probably chemotaxis and not learning at all. We examined a different form of instrumental learning: the ability to learn both to escape and to avoid an aversive stimulus. Freely moving “master” worms could turn off an aversive white light by increasing their movement; the behavior of yoked controls had no effect on the light. We demonstrate that in as few as 12 trials the behavior of the master worms comes under the control of this contingency. Historia Animalium (353 BCE/1910). They have limited sensory capabilities, at least by vertebrate standards, and their motor functions are limited largely to locomotion, sexual coupling, exploratory head and mouth movements, and rapid writhing escape movements. Yet behavioral scientists have been interested in their ability to learn since shortly after the advent of comparative psychology.In many ways earthworms are little more than “the intestines of the earth,” as Aristotle described them in his Allolobophora foetida, a type of worm commonly known as a manure worm and today renamed Eisenia foetida. Remarkably, although Yerkes alluded to the collection of many worms, his 1912 paper is devoted to one worm, “No. 2.” The worm was rewarded with access to a dark, moist tube for making the correct choice in a spatial discrimination task; an incorrect choice resulted in contact with sandpaper and either a strong saline solution or an electric shock. Over the course of many days of training his worm showed evidence of learning, but it was far from compelling.Lumbricus terrestris, a large burrowing worm  We believe that the earthworm might also serve as a viable and inexpensive animal model for studies of the neural bases of learning.Our interest in learning in this organism represents more than idle curiosity. The relatively simple nervous system .We worked with the epigeic worm ad libitum access to food in their home environment. The temperature of the room housing the bin was held near 22°C; temperature within the bin rose as high as 29°C.Worms were maintained in a commercially-available compost bin . They lived in a medium of coconut coir, shredded newspaper, and a bit of soil that was kept moist; vegetable scraps were added to this medium several times a week; worms should be considered to have had Earthworms are non-regulated animals, and therefore this research did not require the approval of our Institutional Animal Care and Use Committee.A Duplo base plate ; Lego Corporation) with a rubber perimeter to create a barrier for the worm served as the behavioral apparatus. The Duplo base plate is smooth red plastic with raised circular pegs 4.5 mm high, 9 mm in diameter and positioned 16 mm from center to center. This provides an environment in which the thigmotaxic worm can feel things touching its body as if it were in soil, and the observer can still see the worm see . A glassWater used to rinse the worm and to moisten the plate was tap water that had been sitting for several days. Experiments were conducted in a room illuminated by dim red light, to which earthworms are not sensitive .Behavioral sessions were conducted between 1400 and 1600 h. Worms were selected from the bin and placed in a group in a small Styrofoam tub. Each session began with the worm being removed from the tub, placed on a paper towel, and sprayed lightly with water to remove any soil. The Duplo plate was sprayed with water, and the worm was transferred to the center of the plate with a thin wooden stick. A 15 min habituation period began when the glass cover was placed over the worm.Experiment 1 involved the measurement of the worm’s responses to the light and the vibration. Beginning 1 min after the habituation period, the worm received 18 trials at 2 min intervals. Three presentations each of six types of trials occurred: 3 Stimuli  for 2 Durations (10 or 30 s). Stimuli were presented pseudo-randomly, with no Stimulus occurring more than twice in a row, and no Duration occurring more than 3 times in a row.η2 (Locomotion was determined by counting the movements of the worm’s anterior end (“head”) past the raised pegs on the Duplo plate. Every time the head crossed a “north-south” or “east-west” imaginary line connecting vertically- or horizontally-adjacent pegs a count was recorded. Movement could be forward or backward, but the line crossed had to differ from the line responsible for the previous count in order to be scored . The movement score recorded for the worm was the mean of the responses to the three presentations of a trial type . Differences between the responses in the 3 conditions  and the 2 durations (10 s and 30 s) were assessed with a repeated measures analysis of variance. Effect sizes were determined by calculating a partial η2 .At the conclusion of the session the worm was retired to a different compost bin and not reused.F = 8.00, p = .005); a partial η2 of .533 indicates that this is a large effect. For both the Light and the Vibration, responses were greater for the 30 s duration than the 10 s duration  = 46.536, p  < .005); the effect size indicated by partial η2 = .869 was very large. There was no Stimulus by Time interaction  = 0.83, p = .457, partial η2 = .106).Worms moved more in response to the Light than they did to the Vibration, which caused no change in locomotion over baseline  group  and 18 in the Yoked (control) group .The same apparatus was used as in Experiment 1. In Experiment 2 the stimuli were controlled by computer interfaces and software .Preparation of the worms and of the Duplo board matched those of Experiment 1. A 15 min habituation period commenced once the worm was placed on the board. Following this habituation the learning session occurred. During the learning session the experimenter observed the worm and pressed a switch that was monitored by the computer whenever the worm crossed a line between pegs, as defined in Experiment 1.Our procedure is modeled after that of Each Control worm was yoked to a Master worm; it received the same number of Vibration and Light presentations, unpaired and in a pseudo-random order . ITSs of 90 s resulted in a session of equal length to that of the Master Group, and assured that the intervals between successive Lights or Vibrations were comparable to those of the Master worms. Light durations were determined by the responses of a worm’s yoked partner .Behavioral sessions for the Master worms began between 1013 and 2048 h . Each Yoked worm by necessity began on average 75 min after its Master worm.η2  and during the Light stimulus (escape response). Differences between the Master and Yoked worms were assessed by paired two-tailed t-tests. Effect sizes were assessed by calculating η2 .t(17) = 2.066, p  = .054, η2 = .201; see The mean probability of an avoidance response (defined as 3 peg crossings during the Vibratory CS) was greater for the Master worms than for the Controls (t(17) = 2.150, p = .046, η2 = .214; see p = .0245, effect size of .692, The probability of an escape response (defined as 3 peg crossings during the Light US) was greater for the Master worms than for the Controls (η2s suggest large effect sizes, indicating that the influence of contingency on behavior is strong. Thus worms are capable of avoidance and escape learning.The results of Experiment 2 demonstrated that the behavior of an earthworm can be affected by its consequences. In Master worms for which motor responses either prevented  or elimiComparisons across species are difficult, but it is worth noting that We have demonstrated the feasibility of using locomotor activity of earthworms in response to Vibratory CSs and Light USs in studies of instrumental learning. In just one session we found that locomotor responses were more probable in worms for which prevention or offset of an aversive light stimulus was contingent on the behavior in comparison to worms for which there was no such contingency.Escape responses occurred with a higher frequency than did avoidance responses, and there was a tendency for escape responses to increase over the course of the session in the Master group. It remains to be seen whether longer sessions or sessions repeated over several days will lead to more robust responding.The use of the yoked control condition allows us to be sure that the differences in escape response probabilities were the result of the contingencies between responses and stimuli rather than to any differences in exposure to the stimuli. The Light stimulus presentation and duration were contingent on behavior only for the Master worms. Because of the Vibration-Light pairing for the Master but not the Control worms, it is possible that the avoidance responses were Pavlovian conditioned responses rather than being instrumental in nature, although they had the effect of avoiding the Light. The present data do not allow an answer to this possibility; perhaps a procedure similar to that devised by The differences that we observed were small; more work is necessary in order to demonstrate the rate at which escape and avoidance is acquired in the worm. We will examine the parameters of this learning in additional studies with repeated sessions, and we will assess the duration of the learned behavior by conducting tests at various times after the acquisition session. Basic learning phenomena such as generalization and discrimination should also be tested. Results of such studies will reveal the extent to which this learning in the earthworm is comparable to escape and avoidance conditioning in rodents and other species. Our ultimate goal is to establish an inexpensive model in which neural or chemical bases of learning can be examined easily.