In this paper we describe the application of a learning classifier system (LCS) variant known as the eXtended classifier system (XCS) to evolve a set of ‘control rules’ for a number of Boolean network instances. We show that (1) it is possible to take the system to an attractor, from any given state, by applying a set of ‘control rules’ consisting of ternary conditions strings  with associated bit-flip actions, and (2) that it is possible to discover such rules using an evolutionary approach via the application of a learning classifier system. The proposed approach builds on learning (reinforcement learning) and discovery  and therefore the series of interventions for controlling the network are determined but are not fixed. System control rules evolve in such a way that they mirror both the structure and dynamics of the system, without having ‘direct’ access to either. Complexity theory has been applied to the study of a number of different fields including biology , economidriver nodes (CCTool (available at: http://cctool.herokuapp.com) has been developed to aid in identifying strategic intervention points   that areRandom boolean networks (RBNs), as developed and communicated by Kauffman , 1993 anRandom Boolean networks are closely associated with specific Boolean networks constructed to model real world systems. Boolean networks that model real-world systems are identified and used within the work of Kim et al.  and inclLCSs  are rulecontrollability problem so that if a response is desirable then rules leading to actions that led to that state are ‘strengthened’. Over time, rules that lead to a basin of attraction, in the particular network, are more likely to be triggered whilst those that have proven unfit are gradually eliminated. It is in this sense that we talk about evolving rule sets and hence the evolution of control in our approach.We apply reinforcement learning to this The work herein illustrates how XCS can be used as a controller for a simple network. However, we believe that the overall approach may be used to control more complex real world networks. Wherever a network has Boolean, integer or real-valued variables that can be manipulated or influenced in some way, directly or indirectly, an eXtended Classifier System variant may be constructed that endeavours to ‘learn’ and control the network.Our approach differs to others in a number of ways. First, an LCS intervention can potentially consist of a number of small ‘nudges’, where appropriate, rather than a single large intervention. Second, LCS rules use #-symbol ‘don’t care’ wildcards in addition to 0 and 1 values for direct matching. These wild cards mean that you need far fewer rules than the number of states in the system to cover the full state space. The resulting ternary (three state) classifier rules mean that the LCS is able to cover large areas of the state space in a compact form. Third, when considering dynamic networks, it is possible that an LCS will be able to continuously adapt to a network subject to external perturbations rather than proving brittle to such changes. Finally, it is worth noting that the state of a Boolean network can be most easily represented as a bit string – this naturally fits with the ternary structure of the LCS rules.from scratch. No human knowledge is required inside the XCS itself (though the detectors must be configured correctly for XCS to function).One other contribution may be clearly stated. The application of XCS in the way we have proposed learns to control a system via the creation of internal knowledge (i.e. the classifier rules) The remainder of this paper is organised as follows. “Random Boolean networks were introduced by Kauffman in , with laK=2 this logic function may well be one of the well known Boolean functions such as AND and OR, though this is not definite. Each node has a binary state (0 or 1) which is randomly selected at the time of model creation and then updated as the model runs.To construct an NK Boolean Network, N nodes are first created. For each node, K nodes are randomly selected and a directed link is created from each of the K nodes to the current node in question. For each of the N nodes a random logic function is created and assigned to the node. When The network dynamics take place in a number of discrete time steps. The precise behaviour depends upon whether the updates are performed asynchronously (one update per time step) or synchronously . The binary state of each node is updated based on the state of the input nodes connected to that node. For an example we shall use an ‘OR node’. For a given set of two inputs, an OR node will output a 1 if either or both input values are 1 (it will otherwise output a 0). So, for instance, if the OR node finds one of its input nodes is set to 1 whilst the other is set to 0, then its state will be switched to 1 in the next time instant . With synchronous updating the results of each update must clearly be ‘cached’ temporarily and then assigned to the nodes once all of the update calculations have been completed.Whilst the construction of an NK boolean network is extremely simple, the behaviour of the network can be varied and complex : the staPrevious work has considered control in the context of the dynamics of the system . In the Learning classifier systems  are ruleA minimal classifier system functions as follows . FirstlyThe subset of population rules that match the current input (RBN state) is termed the ‘match set’. The rules within the match set will suggest a number of actions. The fitness-weighted payoff of each action is used to select a single action to implement in the environment. The match set rules that suggest this single selected action form a subset of the match set called the ‘action set’. The action selected is then fed to an ‘effector’ that performs the desired actions within the environment. Information is then gathered on the payoff associated with the actions taken and the fitness of the rules within the rule collection is updated.One final mechanism of note must be highlighted. There is a genetic algorithm that interacts with the collection of rules, generating new rules via crossover and mutation, whilst a deletion mechanism prunes the rule set if it gets too large. For more details on Learning Classifier Systems please refer to the work of Urbanowicz et al. . For detThe concept of network motifs has been studied within network science . Both thAs mentioned in “random Boolean network these functions are random. In Fig. depends on the precise function at each node. If XOR (eXclusive OR) functions are used for the network in Fig. It should be noted that in Boolean networks all nodes possess a function taking two inputs and outputting a single value. In a Figure ;###:2 or ###:3 . In contrast, to reach the single point attractor from the state cycle attractor we can apply one of a number of rules: 110:3;011:1;101:2;001:3;010:2 or 100:1. It is worth noting that the rules 110:3;011:1 and 101:2 all require that a step be implemented after the intervention, whilst the other rules in the list bring the system to precisely the correct state.It is possible to apply the idea of using the Learning Classifier System ternary rules described within “From the above analysis we can see that the number of required control nodes is one, no matter the desired attractor. The example network is very controllable. The current basin of attraction can be altered by just a single bit flip from any location. This is not the case for more complex Boolean networks. Depending on the nodes that are in fact controllable, the controller may have to wait for a number of time steps before the system can be altered .The above analysis is intended as a demonstration of the application of LCS rules to a Boolean network. However, the rules are intended to work within a full LCS system . In this section we explain the construction of the XCS implementation used in the remainder of the paper. Regrettably, due to space constraints a full description is omitted. Here we aim to describe the critical details of the system, whilst the remaining implementation is covered by the excellent algorithmic description provided by Butz and Wilson .A number of components must be outlined in order to form a complete LCS configuration see Fig. . PreciseGet the situation from the environmentCreate a match set for the situationCreate the prediction array from the match setSelect an action using the prediction arrayExecute the actionObtain the payoff value from the environmentCreate the action set for the selected actionP from: of the previous stepthe payoff the max payoff of the current prediction arrayCalculate previous action set, use P to: update the classifier’s predicted payoffupdate the classifier’s errorupdate the classifier’s action set sizeFor each classifier in the θga to determine whether GA should run on previous action setUse the previous action setthe population in which to insert new rulesof the previous stepthe situation If GA is to be run, run with the following inputs: If not first step: The overall mechanics of the program are as follows. For a given network instantiation, a number of trials of the network are run, with the following steps repeated until the target attractor is reached: current action set using the current situation and the current payoff.When the specified attractor is reached, a final update of the action set and a final GA iteration are performed on the The detector simply reads in the current state of the nodes (0 or 1) in an ordered manner such that this order does not change between time steps. The state of the network thus becomes a bit string, such as 0001001010 .R rules exists. The rule format combines the bit string provided in the previous section with an index of a node to bit-flip. So, for instance, we may have the rule 1010110101 : 8 which simply means that if the current state of the system is equal to the string on the left, flip bit 8. In this case the bit 8 is a 1 and would be flipped to a 0 by this action. The rules thus look similar to the set shown in Table A population of up to When an input is fed in to the LCS a match set is generated. This match set is a list of the rules from the population that match the input string. This is either via a direct match, such as when 0101110101 : 5 directly matches the input 0101110101, or an indirect match, such as when 1111##0001 : 7 matches the input 1111000001.Once XCS has generated the match set, a fitness-weighted predicted payoff for each action is calculated and stored within the prediction array.One action must be selected from those in the match set. A number of possible approaches are available for action selection.Roulette wheel selection is one possibility: buckets proportional to the relative predicted payoffs of each action are created, a bucket is then selected via the drawing of a random number in the range [0,1), the action matching the bucket is then returned.pexplr the system explores, selecting a random action from the action set. With probability 1−pexplr the system instead exploits (or functions in a ‘greedy’ manner), selecting the best action, as indicated by the prediction array.Another approach is mentioned by Butz and Wilson , wherebyHere we use a deterministic method of selecting whether to perform the best action or a random action. If the predicted payoff is greater than 500, we chose the best action (exploit), whilst if the predicted payoff is less than 500 we choose a random action (explore). In our parameter explorations it was found that this tended to out-perform the probabilistic approach on the problem at hand.doActionSetSubsumption is set to true, action set subsumption occurs, whereby more general classifiers ‘subsume’ less general classifiers. When subsumption occurs the less general classifier is deleted and the more general classifier has its numerosity (instance count) incremented by one. See  are updated according to the Butz and Wilson pseudocode . α and νess see “” SectionUnlike in other LCSs, the genetic algorithm within XCS is run on the action set rather than the overall population of rules (though the generated rules are still added to the population). If, for this particular action set, the GA has not run for a specified time interval, the GA is activated. The GA functions as follows.χ, whereby two new rules replace the copies, each formed as a re-combination of the two copies. We then mutate each index position of the new rules with probability μ.Two ‘parent’ classifiers are selected using roulette wheel selection (described above). ‘Child’ copies are made of these two classifiers. Crossover occurs with probability doGaSubsumption to true, a ‘subsumption’ check is performed on the two new strings: if either of the parents of each of the new rules is more general than the new rule, with the same action, then the parent’s numerosity is incremented by one and the new rule is not added to the population. When the two new rules are created their prediction, error and fitness values are the average of the two parents values. If one or both new strings are added to the population  the population size is assessed. If the population size is found to exceed the maximum, roulette wheel selection is used to select which rules survive, based upon a ‘vote’ for each classifier /steps. If one of the desired attractor states is not reached, the reward is zero. The intervention and step counts are reset every time the environment is reset after the target attractor is reached.The reinforcement program (RP) determines the reward associated with a particular action and current environment state. In this system, the reward mechanism functions as follows. Upon initialisation the reinforcement program sets a step count to zero and an intervention count to zero. Each time the RP is required to supply a reward calculation (i.e. once per step) the step count is incremented by one. The intervention count is incremented by one if the action is not 0 (which is the representation of ‘take no action’). When one of the desired attractor states is reached the reward given is simply 1000 must be under this value if it is to be able to subsume other classifiers. If set too high, erroneous classifiers may subsume other classifiers resulting in a loss of good quality classifiers and the strengthening of poor quality classifiers. If set too low, subsumption either never occurs or occurs so rarely that many superfluous rules persist.ε) of less than ε0 are awarded an accuracy of 1, thus increasing the classifier’s fitness and its chance to be selected by the GA. If ε0 is too large here, many classifiers are awarded an accuracy of 1 despite substantial variance in error, thus making the GA less focused (inferior rules are more likely to be selected for reproduction). If set too low, then any payoff variance (resulting in an increase in error) is likely to prevent a rule being reproduced (the GA may effectively become too selective or ‘elitist’) even if the rule has a high payoff.This parameter also affects fitness value updates. During the rule variable update, classifiers with an error value . If set too high, rules that are definitely inferior may linger in the population rather than being aggressively removed. If set too low, newer rules with uncertain performance may be aggressively removed despite this uncertainty. Butz and Wilson . The two-tier update rules enable the initial values of fitness, performance and error to reach accurate values more rapidly . This m. Wilson  uses a sα Affects the calculation of the classifier’s fitness as follows. A classifier’s accuracy is crucial in determining its fitness. Accuracy depends on the classifier’s error, the value of ε0, α, and ν. When a classifier’s error is below ε0 accuracy is set to 1. When a classifier’s error is greater than or equal to ε0, the accuracy is determined by the settings of α and ν. If α is high then there will be a large ‘drop off’ when a classifier’s error exceeds one. In contrast, when α is high (approaching 1), there is very little drop off and the rate of accuracy decrease as error increases is determined almost completely by ν. See Fig. ν Affects fitness value updates. Butz and Wilson (ε0). If high, accuracy falls rapidly. If low, accuracy falls slowly. See Fig. d Wilson  advise aχ The likelihood of crossover being applied when the GA is run. If set high, crossover will always occur. This may be desirable. If set too low, rules are never crossed over or are crossed over infrequently, reducing the likelihood of fitter, novel, recombinations of existing rule conditions. Assuming that the problem is of the type such that recombination of good solutions can produce even better solutions, a low crossover rate could decrease the speed at which the rule population is improved. In practice the authors would suggest 0.70–0.90 for this parameter, in line with the suggested range of 0.5–1.0 by Butz and Wilson . If set.doActionSetSubsumption Perform subsumption in the action sets. If set to true, less general rules are subsumed by more general rules in the action set . Subsumption is beneficial because it compresses the rule set, removing superfluous rules. Subsumption can be detrimental because it can enable overly general rules to take over the population, eliminating new promising rules before they become established. Some authors, such as Lanzi  or state cycle. The experiments where run on a laptop with an i7–7700HQ (2.80 gigahertz) processor with 16 gigabytes of random access memory. The program itself was written in the Java programming language.The parameter settings described here apply to the XCS variant used for the experiments. The initial parameter settings used were aquired from the Butz and Wilson paper . Final pα (0.087), ν (0.01) and ε0 (18.5) and explain why we believe this combination works well on the present problem. As explained previously, our payoff function in the ‘reinforcement program’ is 1000(steps−interventions)/steps. This means that for a given rule, payoff may fluctuate (sometimes the rule is used in an ‘action chain’ that involves many steps and sometimes it is used in a chain that involves just a few steps). This tends to increase the error of the classifier in question. In order to preserve these classifiers, the above parameter combination is more tolerant of error than is usual. Future work should involve an approach that adopts a payoff function that avoids these fluctuations whilst at the same time devising a method to minimise the number of active interventions (which is what the present payoff function is designed to do).Here we must make a note on the unusual combination of N = 5 and K = 2 and synchronous updating. The network is instantiated according to the description within “Since this is, to our knowledge, the first time an LCS has been applied to controlling an RBN here we focus on a simple boolean network with Due to a limitation of the XCS system there is no guarantee that XCS currently evolves rules that completely cover the state space. For this reason it is sometimes necessary to run XCS for greater than 250,000 steps to acquire a solution. With good parameter settings this is a very rare event. Our XCS implementation is designed to evaluate the rule set after 250,000 steps and then every 10,000 steps thereafter. If the solution covers the state space and produces no failures then the system moves on to the post-processing stage. Future work could improve this behaviour.Figure Figure Figure Figure Net indicates the network instance number, Run indicates the run number, CR indicates the compressed number of rules required to minimise the number of average interventions , AS indicates the average number of steps required to reach the attractor with this rule set, AI is the average number of interventions required to reach the attractor with this rule set, and Time is the complete time taken to produce the rule set in seconds.Table Each control step is always followed by at least one ‘natural’ step. This emerges as a result of our restriction requiring that only a single intervention is permitted at once (permitting multiple interventions would reduce the difficulty of the problem).The step count includes any active interventions (the application of one of the actions in the range 1 to 5) and the ‘natural’ steps that occur during the evolution of the system. The action ‘0’ (i.e. no action) is not counted as a step or intervention. We now present a number of the specific rule sets that have been evolved. For network instance 1 rule set presented in Table the LCS and rule-compression combination is able to successfully evolve control rules (sets of classifiers) for the full range of networks, requiring fewer than one rule for each state (with two major exceptions). Performance, in terms of the post-compression rule-set size with minimum number of interventions, ranges from 35 rules to 10 rules . Performance, in terms of average steps ranges from 4.531 when selecting for minimum number of interventions to a relatively small value of just 2.781. Similarly, the number of average interventions range from 1.714 to just a single intervention on average.From Table The above results suggest that (1) it is possible to control random Boolean networks with rules developed via learning classifier systems, specifically the ‘standard’ variant of XCS and (2) the ternary structure of the rules tends to enable control of the RBN in question with far fewer rules than there are states in the system.Figure whilst over-general classifiers exist, this does not prevent XCS from successfully controlling the system.From consideration of Tables Training times range from 22–120 s (using an i7-7700HQ processor). On Network 1 training only takes between 22 and 24 s. On Network 2, training takes between 113 and 120 s. On Network 3 training takes 54–57 s. Finally, on Network 4 training takes 63–67 s. This discrepancy in training time indicates that not all networks are equally easy to learn to control. As was recognised earlier and can be seen in Fig. ε0 parameter is set to 18.5 whilst the θsub parameter is set to 31.579. These high settings both act to prevent inexperienced rules from being subsumed too soon by over-general classifiers. However, it is possible that, as a side effect, this mechanism may be somewhat too effective at preventing subsumption, thus leading to excess rule variety. This is clearly another area where further work is required, to better understand this occurrence.The rule sets of 304 and 168 rules represent a puzzle to solve. Initial consideration of the parameters provides an indication of what may have happened. The the overgeneralised classifier problemavoidance of exhaustive search for state mapping and compressionrelaxation of the single-intervention-at-once restrictionmodification of the LCS to ensure full state space coveragespecific nature of the rule set evolvedThere are a number of limitations to the study that also serve as opportunities for future work. Broadly these can be listed as follows: The overgeneralised classifier problem raised in “Avoidance of exhaustive search is also important for techniques to control larger and more complex systems. In the present system, the original mapping of the state space to locate attractors is done in an exhaustive manner . Exhaustive iteration through start locations is also used in the final stage, both to test that the rule set covers every possible situation.A relaxation of the single-intervention-at-once restriction could be considered. Presently multiple actions in succession, or an action that flips multiple bits is not considered. It may be that multiple simultaneous interventions are required when operating on more complex networks. Multiple actions could be considered either through converting the action to a bit string such as 10100110 : 00010010 (i.e. flip bits 3 and 6) or by permitting multiple rules to fire at each time instance.It could be possible to modify LCS to ensure that the rules developed always have full coverage of the state space. This would ensure that the final set of rules produced would cover all possible states in the state space. In contrast, the present implementation sometimes has to prolong the run if the coverage of the state space is incomplete.A final limitation concerns the specific nature of the rule sets evolved. XCS is able to specifically tailor rules to control a specific network. This may be seen as a strength, because a more general control system may perform less well across all networks. However, this must also be recognised as a drawback of the approach – time must be spent evolving a specific rule set for each target network.new XCS variants that can control more complex RBNsapplication of LCS to control of more realistic or complex network modelsfurther performance refinements to the current approachconsideration of situations with limited control nodesdevelopment of more formal approaches for comparative purposespossibilities for control of dynamic networksimproved integration with existing theory on controllabilitycontrolling networks with incomplete informationThere are also a number of future work opportunities that are not related to limitations of the present work: A major opportunity for future work is the application of LCS to more realistic or complex models. The Boolean networks listed by Kim et al.  and Gatefactors (nodes) come with variables that capture controllability and importance. These are attributed with high, medium and low values. Similarly, Schoenenberger and Tanase  would require knowledge of which node was about to update so that rules could be related to particular nodes updating . This would require an integer-valued detector indicating the next node about to update.  Something akin to the ‘class imbalance problem’ faced by ‘regular’ (i.e. single-step) classifiers  as considered in other approaches . One pieDevelopment of more formal approaches for comparative purposes could prove useful. This may be an approach of the kind used by Schoenenberger and Tanase . Alternaet al2009;Moschoyiannis and Krause Possibilities for control of dynamic networks could also be considered. Savvopoulos et al.  have loofull state of the system. It may be that XCSI is able to ‘bridge’ the unknowns here and continue to control the system. Such an investigation marks an important piece of future work.The XCS system does not require any knowledge of the system under control other than that available via the detectors. Indeed, the condition–action rules evolve from scratch to model the external system. However, there may be circumstances where only a subset of detectors are available and we wish to control one or more of the variables covered by the detectors without being aware of the On a related note, it may be possible that the network model that XCS learns on is incomplete or has inconsistencies or errors in it. This is an especially challenging area for the application of XCS . Missing model components must be completed before XCS is run on the model. For instance, missing Boolean functions on a non-random Boolean network modelling a real-world system would have to be filled in. This can be done in a random manner, producing a range of possible network structures. XCS can then be run on these structures, producing a range of possible controllers. Data on the real world network can then be used to eliminate erroneous network structures and associated controllers until a useful controller remains. This is not dissimilar to the approach within the work of Giacomantonio and Goodhill  but for with respect to a particular set of control nodes and a particular goal state or states if the desired state can be reached from any other state in finite time. Controllability with respect to every attractor could well be achieved by providing a detector and associated classifier condition for desired attractor state. In this way, a subset of rules would evolve for each targeted attractor, though the rule sets would still overlap where possible.A further line of work is achieving greater integration between notions of controllability when considering pure network structures and notions of controllability when considering the dynamics of networks. In the traditional sense a network is controllable when it can travel from one state to any other state in finite time  and require between 10 to 18 rules to achieve this (depending on network complexity). We stress that the LCS learns to control the system instances without human intervention and that both the structure and dynamics of the network are automatically taken in to consideration whilst the rule set evolves.LCS rules are ternary in nature, providing a compact rule structure with fewer rules than there are states in the system. The Immediate future work to overcome the limitations of this work includes the development of pre-processing and post-processing procedures that do not require exhaustive search, modification of XCS to avoid overgeneralised rules and further modifications to ensure full rule coverage of the state space for every complete run. The most promising future work involves investigating the control response in dynamic networks and consideration of constrained control situations where only a subset of nodes are controllable.


The authors are to be congratulated for a very interesting paper. They are also be thanked for recommending reading chapter 8 of our book on dynamic prediction . Given the observation history, inference on T can be made in a very classical way by computing the log-likelihood of the data. In the main paper such a log-likelihood is shown in the left panel of Figure 5 for one specific case. To get more insight a kind of landmark analysis was carried out, in which all 278 patients still at risk at ll(T) of the standard model is computed for survival time T contains the whole interval . A summary of the results is given in Table If we ignore the uncertainty in the regression parameters we have a model mentclass2pt{minimT at Figure To investigate the behavior of the implied prediction model, the landmarking approach is used. That means that for a fixed time-point entclass1pt{minimaFirst a simple calibration of the marginal survival is obtained through the model mentclass2pt{minimNext we consider the cumulative hazard Personally, I am very hesitant about any modeling beyond the observation limit The coefficients in the revival model are shown in Table Table entclass1pt{minimaThe revival approach can be a very useful tool for taking account of the observation time in prediction models. The comparison of different approaches to obtain prediction models using the calibration in the full data set as shown above might be optimistically biased. To avoid this optimism bias some form of cross-validation is needed, but that is beyond the scope of this commentary.


Turbine tip clearance of aero-engine is important to engine performance. Proper tip clearance can reduce the gas leakage over turbine blade tips and improve the engine efficiency of turbo machinery and reduce the fuel consumption. Therefore, accurate tip clearance measurement is essential. The inductive measurement method is one of the non-contact distance measurement methods, which has the characteristics of high sensitivity, fast response speed, and strong anti-interference ability. Based on the principle of inductive sensor measuring tip clearance, the ambient temperature change may cause the material electromagnetic performance change for the conductivity and permeability varies with temperature. In order to verify the temperature effect on the sensor performance, the repeated calibration experiments were carried out to obtain the sensor repeatability error of 5.4%. Then, the sensor was calibrated in the range of 0 mm–4 mm clearance at temperature from 600 °C to 1000 °C and obtained the measurement error of 4.6%. Results indicate when the temperature ranged from 600 °C to 1000 °C, clearance measurement error is smaller than the sensor repeatability error so the temperature effect on the sensor characteristics can be ignored. This conclusion makes the sensor promising for monitoring the blade tip clearances at various temperature environment. The blade tip clearance of gas turbine is significant for its performance and efficiency. Therefore, precise measurement of tip clearance is the premise of accurate design and optimization of the tip clearance ,2,3. TheMark R.W. et al. ,5,6 fromBased on the existing studies, inductive sensors have been developed and have gained considerable success for their simple structure, low cost, and easy installation . It is iHowever, the measurement signal of eddy current sensor has temperature drift problems and it is inevitably affected by temperature variation for the reason of measuring principle. Lyu Y.T. et al.  used temThe current research mainly focuses on the temperature compensation and practical application, so there is little experimental research of temperature effect on the sensor performance especially on the characteristic curve. As the basis of measurement, accurate calibration is beneficial to optimize the measurement uncertainty. Hence, this paper focused on validating the high-resolution inductive sensor performance and aimed at finding the temperature influence on the sensor characteristics and measurement results in the range of 600 °C to 1000 °C. Furthermore, instead of the signal processing circuits in , the simThe working principle of inductive sensor is based on Faraday’s law of electromagnetic induction and Lenz’s law. The main component of the sensor is an inductive coil, which can be a three-dimensional spiral coil or a two-dimensional planar coil. The coil generates a magnetic field when excited by a high frequency AC signal, and then the eddy current is induced in the metallic target when it passes through the magnetic field and thus a reverse magnetic flux caused by eddy current declines the inductance of sensor coil. 1V and 2V can be derived from the Kirchhoff Voltage Law (Equations (1) and (2)).I is the current in the circuit, Vc. Inductance of connecting wires in the circuit were much smaller than The relation between equivalent circuit parameters and output signal CV) is Equation (4):The current in the circuit Data processing process is shown in There is peak detection error caused by the discontinuity of collection signal as B generated by circular coil is calculated by Equation (5). When the material permeability I is determined by exciting circuit, it can be seen that coil turns (N) and coil thickness (h) can both influence the magnetic field intensity.The magnetic field intensity of circular coil is higher than that of square coil. The magnetic field intensity N), radial dimension (r), and inverse correlation with thickness (h).Harold Wheeler  had publN and inversely proportional to h when the sensor coil is connected to a certain circuit. That is to say the coil with more turns and thinner thickness can generate higher magnetic field intensity, thus coil has higher sensitivity and wider measuring range.From the above, the magnetic field intensity and coil inductance are proportional to h equals to the wire diameter and the compact method is adopted to wrap coils to permit more turns for the given coil size and generate higher magnetic flux density.The study result of Du L. et al.  indicateThe sensor probe in this paper was manufactured by the device as Q is the quality factor which indicates the inductive element quality and it is equals to Three coils were fabricated by the method above with the same structure and the parameters were measured by the LCR meter at 4 MHz. For the phase angle that indicates the inductive element quality, which means the higher phase angle, the element is more inductive and more sensitive in the remote inductive measurement. Equation (7) is the calculation formula of the sensor phase angle.In order to determine the signal frequency in the experiment and the quality of the sensor, the phase angle of sensor at different frequency were measured by LCR meter and In order to verify the sensitivity and measuring range of manufactured sensor, the calibration experiment was conducted at room temperature to obtain the sensor characteristic curve. The characteristic curve of the tip clearance sensor refers to the relationship curve of sensor characteristic parameter and the clearance. The relative variation of voltage was used as measuring quantity in the study.The calibration device, as 0 on the sensor coil without target object. Then, we adjusted the distance between sensor surface and target blade surface from 0 mm to 5 mm with 50 μm step, and collected voltage signals on the sensor coil at each position. The voltage relative variation dV/V0 was denoted as sensor characteristic parameter and calculated following the steps shown in The calibration system includes sensor probe, calibration target, position controller, function generator, and DAQ system. After connecting the measuring circuit  and fixing the target position, we collected the voltage V2 is 0.9992. The measurement data near 5 mm is listed in 0 = −2.3419; a1 = 1.7963; a2 = −0.76074; a3 = 0.20458; a4 = −0.030464; a5 = 0.0018549. According to the calibration curve in The calibration result is shown in In order to verify the repeatability of the sensor, three sets of repeated calibration experiments were carried out at room temperature: collected the measured signals of the sensor at different positions with 50 μm step length from 0 mm to 5 mm, and then collected the signals from 5 mm to 0 mm in the same way. Repeated this process three times and recorded the results as The fitting parameters of different calibration curves are listed in The clearance measurement results at different positions are plotted in The purpose of calibration at high temperature is validating the sensor so it can operate well at different environment temperatures over 500 °C. The heat resistance of the sensor was verified in . The higThe heater temperature control precision is 1 °C by PID control method and the position adjuster with ±1 μm accuracy is controlled through the motor driving screw.The calibration experiments at high temperatures were carried out in the range of 600 °C–1000 °C and the calibration range was 0 mm–4 mm with 50 μm step length. The calibration results are plotted in The fitting parameters of different calibration curves are listed in The clearance measurement results of different temperature at different positions are plotted in Compared with the repeated calibration results at room temperature in According to the calibration results at room temperature, the measuring range and resolution of designed sensor were proved to be over 5 mm and 10 μm respectively. The maximum relative error of repeated measurement was 5.4% within the range of 4 mm based on the analysis of results of repeated calibration experiments.The error analysis of calibration results at different high temperatures shows that the maximum relative error of the 600 °C–1000 °C calibration results was 4.6%, which is smaller than that of the repeated measurement. This indicates the difference of measurement results during the range of 600 °C–1000 °C may be caused by the sensor repeatability error, and the temperature effect on the measurement results can be ignored at high temperature. This conclusion is consistent with the conclusion in  that theThe above conclusions apply to the planar platinum wire coil which is fabricated with the same method and has the similar structure with the coil in this paper. While the material or the structure is changed, the coil inductance may change and also the resistance of other materials may change differently with temperature, so the coil performance at the high temperature should be tested additionally.The designed sensor with planar coil made of platinum wire was proved to be a good inductive sensor for its phase angle is up to 80° under 4 MHz excitation frequency and the sensor performance meets the requirements of tip clearance measurement for the measuring range was over 5 mm and the resolution is better than 0.01 mm within 5 mm range according to the static calibration result.The sensor was effective at high temperature like 1000 °C, and maintains good stability within the temperature range from 600 °C to 1000 °C. This suggests the designed sensor is capable to operate over a wide temperature range and that is an important basis for the sensor to be used in turbine tip clearance measurement in the future.Compared with the measurement relative error of repeated calibration at room temperature, that of measurement under different temperatures was smaller, which means the difference of measurement value at different temperature may be caused by repeatability error rather than temperature influence. This meaningful conclusion will greatly simplify the calibration process and data process at high temperature.Based on the principle of inductive clearance measurement sensor and comparative experimental research, the following conclusions can be drawn in this paper:The measurement system is usually cooled by additional cooling system. The temperature range investigated in this paper is the temperature that current sensor can withstand not the real turbine temperature.Furthermore, it is necessary to conduct in-depth research work on reducing the sensor repeatability error in the future. After that, a series of thermal dynamic tests will be conducted on the turbine rotor using the measurement method adopted in this paper.


Recently, minimally invasive surgery (MIS) robotics enters the phase of autonomous operation. However, because of the high variability of the environment, conducting a fully autonomous surgery is still extremely difficult. This paper presents a share control system, the objective of which is to suggest the optimum path of tool guidance through the use of force on the master manipulator (hereinafter as master), meaning the surgeon's hand. Owing to this type of control, the surgeon has full control over the position of the tool the entire time and is supported by the system to better and faster guide the tool during surgery. The force should be felt by the surgeon but, simultaneously, must not hinder or impact the surgical process. Furthermore, the share control system presented in the paper can be turned on or off at any moment during surgery. WatanabeAdmittance control strategy characterized by time‐varying stiffness, damping, and inertiaThe force feedback control is another method that is used in the robotic assist surgery. There are a lot of methodsTo sum up, no cardiac surgery robot is presently available that would significantly facilitate the work performed by surgeons and, simultaneously, would guarantee full safety for patients. Therefore, it is of the most importance to design such control architecture that will benefit from the advantages of autonomous systems  and teleoperation . Such a control system has been designed and is presented in this paper. The control system supports the surgeon during tool guidance. Therefore, the surgeon can operate the tool freely, without any restrictions. When help with tool guidance is required , operation assistance can be turned on. Then, the control system will only suggest the optimum trajectory of tool guidance but will not force the surgeon to rigidly keep to the predefined trajectory. Consequently the forces on the master are always orthogonal to the trajectory and do not initiate the motion along the path. This kind of control is named as the passive guiding tool.Few examples of share control systems can be found in the literature. The most similar one to described in this paper are presented by Shahin et al.In the system used by Marco de Baar,2SettingDefining the space where the robot will follow the standard trajectory.Determining the robot's position in relation to the implemented standard trajectory.Determining the force value and direction on the master based on: the velocity on the standard trajectory and the tool's distance from the standard trajectory.Setting force interactions on the master.Determining the position of the master, i.e. determining how the surgeon moves the master (with set forces).Transferring the set positions to the angular position in the slave manipulator's joints.Movement of the robot (tool).Repeating the activities from 3 to 8 until a specific procedure associated with the operation has been completed.The general idea of the designed system is to treat the master as an object with shared force control—through drives (the autonomous part) and by the surgeon .It is important that calculations at subsequent stages of the iteration are made in set positions of the slave manipulator's tool. Based on experiments conducted at the initial stage of the studies, such an approach to robot share control improves the smooth movement of the tool.The points 1 and 2 of the action plan are described in others research works.2.1i = 1, …, n is the ith position of the tool, and the i index increases from 1 to n in the direction of planned movement, while Xi=xiyizix˙iy˙iz˙i=pip˙i, where p˙i velocity of tool in ith point of the path. The entire path τ¯.It is assumed that the tool is located outside the standard trajectory when share control is turned on (at the beginning of the task). Therefore, it is appropriate to find such a point on the SP that will be closest to the current position of the tool. In paper,Control system studies were carried out on the Robin Heart surgery robot. This robot was developed in cooperation of Foundation of Cardiac Surgery Development (FCSD) and Lodz University of Technology.p˙a. Then, it is assumed that one of the points belonging to the SP is the set position Xd=pdp˙d , it is assumed the index d’ corresponds to the geometric center of a figure created under the function (𝑗). It is determined based on the following dependence:The point of which the similarity parameter will have the highest value will be the Xd equals d', rounded to the nearest integer value.The index of the new set position pd corrects the quality of designation. However, several tests showed, when share control system is switched on and the master is not moving along the paths, velocity of master can disturb best point search process, therefore velocity is not included in the calculations at this point.Sometimes, the trajectory passes the same points several times, so adding velocity as an additional parameter to be compared in order to determine 2.2From the perspective of cardiac surgery robot's operation, turning on/off forces interactions along the set trajectory is crucial. It is unacceptable that the robot “pushes” the surgeon's hand, which would lead to excessive autonomy. It could result in improper tool guidance and damage of the patient's tissue. The cooperation between the robot and the surgeon should be in compliance with the following assumption: if the surgeon does not move the master at any time during operation, the robot must not make any movements that would continue the surgery. Otherwise, the robot would perform individual activities autonomously. Therefore, the control system's role involves mainly suggesting the optimum path for the tool by impacting the master with force in the direction perpendicular to the planned movement while the surgeon is still in charge of the pace of movement and guidance of the tool. This is possible by means of setting a temporary selection matrix  without an orientation analysis .στ¯ ‐ the standard deviation (SD) between registered paths.τ.¯without share control and τ̿ ‐ the average path calculated based on the registered paths with share control enabled. This path is used only for calculation the Standard Deviation of the paths with share control enabled.στ̿ ‐ the SD between registered paths with share control enabled and τ̿στ¯τ̿ – the SD between τ¯ and τ̿t¯ ‐ the average time of tool movement along the SPW – the set of paths registered in the case of operation with share control disabledSC – the set of paths registered in the case of operation with share control enabledDesignation example: W(L)—set of L‐shaped paths, SC(S) —set of Small Loops paths with share control enabled.The results of the calculations are presented in Table k is the number of the path, the points of which are currently compared with the average path; m is the number of paths, nk is the last point of the k‐th path; and δxikτ¯ is the distance between the ith point on the kth path (xik) and the average path τ¯.where δxikτ¯, the points ware matched using the dynamic time warping (DTW) methodxik lay on one of the simple paths and (k‐th) and xjτ lay on the average path),In order to calculate the distance T is the value according to which the velocities are calculated with reference to position values (in the experiment T = 0.07).στ¯ has a higher value for the paths registered with disabled share control for both shapes). It is significant that tool guidance with enabled share control was approximately 30% faster than without share control.Based on the results presented in Table 3.2In this section, two experiments showing the functionality and stability of the system are presented.Turning force control on.Moving the tool away from the SP—this movement took place while the tool was moving along the path in order to visualize the increment of force and tool's distance from the path.Moving tools as close as possible to the SP.Change in movement direction.Pull tools away from the SPThe first study was intended to verify whether the algorithm designed actually limited the force interactions on the surgeon's hand. For this purpose, an experiment was planned, which involved moving the tool away from the SP in two different directions for the standard L path. The entire experiment was composed of the following stages:This experiment was conducted with enabled assistance, at the same time, the trajectory and the values of the forces generated by the control system on the master were registered. The results of the experiment are presented in Figure The forces presented in Figure The figure also presents the SP used in share control algorithm and the real path of the tool motion (T1P) Figure A. As is Another experiment involved a verification whether force assistance could be enabled and disabled at any moment during operation. In order to do that, an experiment was planned, which involved guiding the tool along an L‐shaped vertical line. During guidance, the tool was moved away from the SP (with motor assistance enabled) and then tool guidance assistance was interchangeably disabled (in about 38 and 25 mm Z‐axis position) and enabled (in about 33 Z‐axis position) during the movement. The result of the experiment is presented in Figure 4removing the possibility of autonomous work—the robot does not move its tool by the path on its own, which is being achieved by implementing the selection matrix S;adding forces restricting surgeon's hand movement to a level that can be easily overcome so he can guide the tool the way he wants to;adding of dumping and stiffness coefficients that stabilize the motion.An additional safety procedure will be implemented in final application. It will be based on the verification if the surgeons is holding master and is observing operation field.One of the medicine aspects is not to harm the patient. The same principle is actual in case of medical robots. And it was also the case while working on this surgeon assistance algorithm. It's done by:The safety level using share control increases as the system prevents uncontrollable and pattern inconsistent motions to appear. The plan is that in final application for repeatable motions,Tool guiding system can be used for any shapes and lengths of SP, which makes this method universal. It needs to be pointed that enabling recording of SP directly from the robot enables exact guidance of tool during operation. Nevertheless, during operation, tissues can be slightly displaced, which makes it necessary to adjust the tool. Therefore, for tool slightly displaced from SP, the guiding forces will be small Figure  so the sp and damping Kd coefficients were selected. The correctness of this selection has also been confirmed during tests presented in previous sections. Figure The stability of the system cannot be proved theoretically as one of the system's elements is a human being. The damping and compliance of human hand differs for each individual and is being determined by muscles tension. This problem is widely discussed in Szaniewski, Podsędkowski's study.5The method of force interactions presented in this article is destined to support the surgeon while performing repetitive movements or when the movements are difficult to perform. Several aspects are of significance in the presented method. One of them is that the surgeon can define, independently, the trajectories along which the surgery will be conducted. This way, the surgeon is given an opportunity to implement any path desired to the control system, ie, during suturing, tying knots, or in the case of difficult movements that need to be made with extreme precision during surgery. Additionally, the surgeon may, at any time during the surgery, enable or disable tool guidance assistance without the necessity of discontinuing operation. It is sometimes decisive when external conditions change—ie, tissue rupture, sudden bleeding, etc. The very manner of force interactions implementation is an important aspect. Force interactions are proportional to the tool's distance from the nearest point on the SP. Obviously, maximum force is limited to such an extent that it is perceptible on the master's handle but still not too high to hinder the operation of the cardiac surgery robot. Such an approach allows the surgeon—with share control enabled—to still be able to correct the path of the tool without using excessive forces.To sum up, implementation of such a control system gives the surgeon full control over the position of the cardiac surgery robot with simultaneous movement assistance ensuring that the trajectory of the tool is in compliance with the previously defined model. The results of the studies presented herein can also be applied to train surgeons using a training stimulator with implemented share control, which would teach surgeons' correct guidance of the tool also in the case of surgery without a cardiac surgery robot.


The synoptic‐scale meteorological conditions leading up to the 30 most extreme subdaily summer rain events for two regions of the United Kingdom (northwest and southeast) were examined for the period 1979–2013. Using a recently available, quality controlled, national hourly rain gauge data set, we were able to identify extreme 3‐hr rainfall accumulations that may be indicative of flash flooding. Composites of the state of the atmosphere leading up to these dates were produced to investigate synoptic‐scale processes, thus potentially allowing for them to be identified in coarse resolution reanalyses and in climate models. The results show that the two regions have different dominant synoptic‐scale conditions leading to extreme 3‐hr rainfall, which is thought to be related to the type of rainfall typically experienced in each region. In particular, positive anomalies in mean sea level pressure and the geopotential height at 200 hPa over the United Kingdom are associated with extreme rainfall in the northwest, where the position of the westerly jet is also important. For the southeast, no clear anomalous synoptic‐scale conditions could be identified; however, localized moisture sources and unstable air masses were observed in association with extremes. These results indicate the importance of better understanding of both synoptic‐scale and thermodynamic drivers of short‐duration extreme rainfall, with potential implications in forecasting and flood warning, as well as for understanding the representation of key processes by regional climate models. Different synoptic‐scale meteorological processes dominate on days with JJA subdaily extreme rainfall depending on the region of interestA positive anomaly in the geopotential height is associated with extreme rain events in the northwest of the United KingdomLocalized moisture sources and unstable air masses observed with extreme rain events in the southeast of the United Kingdom This is also true for their identification in most climate model simulations due to their coarse resolution and the use of parameterized convection schemes, although this is being addressed by the emerging generation of convection permitting regional climate models , where here n = 30. In contrast with the annual maxima approach, this potentially allows for multiple events to be identified in a wet year with no events consequently being identified in relatively dry years. The use of this data set allows for the identification of subdaily rainfall extremes at a national scale with a relatively high network density which has previously not been possible. This increases the likelihood of capturing those events that lead to flash flooding whereas daily rainfall extremes, for which data has long existed, would more typically lead to fluvial flooding.We use a recently available, quality controlled U.K. hourly rainfall data set .The investigation looks at composites of the 30 highest 3‐hr accumulations for each region. The composites shown in this section are the anomalies of the daily means of each field to the monthly mean, with the composites of the absolute daily means shown in the The results from the investigation into the synoptic situation, the mean sea level pressure (MSLP), and relative humidity at 850hPa are looked at first section . The sta3.1Composites of the daily anomalies , with respect to the monthly mean, are shown in the top rows of Figures As would be expected, the patterns in the anomaly plots are relatively noisy  surrogate was then calculated using the equations from Davies‐Jones (λ≡1/κd ( = 3.504) as κd = Rd/cpd ( = 2.2854) is the Poisson constant for dry air, and C is a scaling constant =273.15 K.Next we examined the stability of the atmosphere. Typically, these are small‐scale fields or are not widely available in reanalyses therefore surrogates for atmospheric stability/CAPE  were used. Although CAPE is available in the reanalysis used in this study, a surrogate was used so that this work could be applied to other data sets. The surrogate chosen in this study was the wet‐bulb potential temperature, based on Davies‐Jones  and N. Res‐Jones : (2)θw=θw surrogate values between two levels in the atmosphere. If the stability surrogates are less negative at higher levels  that is suggestive that the atmosphere will be unstable—it is representative of an air parcel rising from a warm and moist lower atmosphere to a colder and drier upper atmosphere.The stability of the atmosphere can be inferred by taking the difference in the θw, is used in this paper, results (not shown) did not show much difference between the θw and θe surrogates. The choice of levels used to calculate the differences in θw was also investigated but again found similar values for each pair of levels ; the 850‐ to 500‐hPa difference was chosen as it showed the largest differences. A comparison between the θw (850–500hPa) field and CAPE (not presented) showed that a similar pattern, both spatially and the evolution of that pattern over time, is seen in the two fields for both regions. The same conclusions were drawn in regard to the stability/instability of the atmosphere for both fields and whether they were a precursor for the extreme rain accumulations.Whilst the moist surrogate, θw (850–500 hPa) can be seen in the colored contours on the second row of Figures The composites of A different pattern (and evolution of that pattern) in the anomalies for NW events Figure  is seen 3.3An issue with the stability criteria is that they tend to be relatively localized, resulting in noisy spatial patterns, as observed in our results. Another variable that is often used to investigate atmospheric instability is the geopotential height. If a low in the geopotential height at a given pressure level is observed, it suggests a height cap, limiting the vertical movement of an air parcel. If this air parcel is moist, this height cap increases the chances of convection being triggered. This is a simplified approach of looking for cutoff lows, which were determined to be present during the U.K. summer floods of 2007 by Blackburn et al.  and are Results (not shown) from an investigation into the sensitivity to the choice of level showed that all four levels identified lows in the same location and at the same time. The 200‐hPa level was therefore chosen and is presented in Figures θw, both fields become less coherent and smaller in magnitude as the day of the rain event approaches.The geopotential height at 200‐hPa (z@200hPa) anomaly to the monthly mean is shown as contours on the second row of Figures θw field. The geopotential height is shown to be a large positive anomaly, compared to the monthly mean, over western Europe. While the magnitude of this anomaly decreases toward the day of the extreme rainfall, the extent of the anomaly also decreases but moves to and remains over the United Kingdom, with a positive anomaly being seen over the United Kingdom on the day of the rainfall. This corresponds to the ridge in absolute geopotential height seen in Figure A much clearer pattern is seen for NW Figure , which aθw anomaly for the two regions is not clear.These anomalies in the geopotential height are likely related to the location of the storm track; a positive anomaly in the geopotential height field suggests strong storm track activity. However, while the same general conclusion may be inferred as before, that the type of processes dominating the extreme rain event in the two regions differs, the reason for the opposing evolution in both geopotential height anomaly and the 3.4The previous sections have highlighted the importance of looking at moisture availability and moisture transport. In this study two variables were investigated—IVT and TCWV. Potentially, by looking at these variables in combination with one of the stability variables, this will provide information as to when the atmosphere is both moist and unstable thus increasing the chances of convection being triggered. IVT is used in the identification of atmospheric rivers (ARs), which have also been shown to be associated with winter flooding , relative to the monthly mean, in the MSLP is seen centered over the United Kingdom; no clear anomaly is seen for SE. The relative humidity field shows a small region of high relative humidity in the days preceding the rain event, for both SE and NW, but only NW sees a negative anomaly on the day of the extreme rainfall, suggesting that the local moisture supply has been used by the process that led to the rain event for the NW.θw field indicates an unstable atmosphere over the United Kingdom and its surrounding region on the day of the extreme rainfall and for the preceding 5 days. However, compared to the monthly mean, this instability was not anomalous for SE. A positive anomaly was observed for NW 5 days prior to the extreme rainfall, but this was not anomalous by the day of the rainfall. The pattern and evolution of the geopotential height at 200 hPa was similar to those for θw for both regions. No clear signal in the geopotential height could be seen for SE. However, the geopotential height had a positive anomaly over western Europe 5 days prior to extreme rainfall for NW, which decreased in magnitude and centered over the United Kingdom by the day of the rainfall. The signal in the geopotential height was also thought to be related to storm track activity, with the positive anomaly suggesting high storm track activity.The θw, geopotential height, moisture availability, and convergence variables (IVT and TCWV) are all dynamically linked, and thus, it is not surprising that they show similar trends and patterns. A positive anomaly in the geopotential height indicates a ridge; a high TCWV indicates a moist atmosphere. This suggests there is upward motion of moist air within the atmosphere, which will result in high θw. Further, the IVT field indicates areas of moisture convergence, which explains the colocation with areas of high θw. The benefit of using geopoential height and TCWV is that they are larger‐scale fields, compared to θw, which makes them easier to identify in coarse resolution models.The The strong, moist westerly jet associated with the Azores high, combined with the high topography in the NW region has also been shown to be associated with the occurrence of extreme rainfall. While the SE region has less topography, the effect of the westerly jet when over the southern United Kingdom is to create a cyclonic circulation resulting in strong upward motions over the region, a key condition for the development of convective events.θw field, a localized field, which is an indicator of convection, can be inferred from larger‐scale fields—positive anomalies in both the geopotential height and TCWV. We also show that the processes leading to extreme subdaily rainfall are dependent on the region, using two contrasting regions for the United Kingdom. The NW region, due to the presence of orography, is strongly affected by larger‐scale orographically enhanced processes that are less reliant on localized convection. The SE region has more localized sources of moisture and has relatively unstable air masses leading up to extreme rainfall. While we believe that SE extremes are driven more by convective processes than large‐scale processes (as shown for NW), this cannot be proven here and requires further investigation, outside of the scope of this paper.A number of conclusions can be drawn from these results. We show that the These results are consistent with those derived from simulations of hourly rainfall from models, including those at a 1.5‐km convection permitting scale, indicating that larger‐scale conditions are important for the generation of subdaily extreme rainfall. Chan et al.  found thThis study confirms that the availability of quality controlled rainfall observations is essential to improve our understanding of subdaily extremes. Further, the influence of the large‐scale atmospheric circulation needs to be examined in conjunction with local thermodynamics to more fully understand the drivers of these extremes (Lenderink & Fowler, Supporting Information S1Click here for additional data file.Figure S1Click here for additional data file.Figure S2Click here for additional data file.


What is intuitive: pro-social or anti-social behaviour? To answer this fundamental question, recent studies analyse decision times in game theory experiments under the assumption that intuitive decisions are fast and that deliberation is slow. These analyses keep track of the average time taken to make decisions under different conditions. Lacking any knowledge of the underlying dynamics, such simplistic approach might however lead to erroneous interpretations. Here we model the cognitive basis of strategic cooperative decision making using the Drift Diffusion Model to discern between deliberation and intuition and describe the evolution of the decision making in iterated Prisoner’s Dilemma experiments. We find that, although initially people’s intuitive decision is to cooperate, rational deliberation quickly becomes dominant over an initial intuitive bias towards cooperation, which is fostered by positive interactions as much as frustrated by a negative one. However, this initial pro-social tendency is resilient, as after a pause it resets to the same initial value. These results illustrate the new insight that can be achieved thanks to a quantitative modelling of human behavior. Some studies, based on the premise that decisions which take less time are more intuitive, suggest that making unselfish, cooperative decisions is a human instinct that is then undermined by rational deliberation8. The claim that showed that people under pressure make more cooperative decisions in Public Goods games4 failed a recent Registered Replication Report9 including 21 independent experiments. Other criticisms include that, after controlling for the strength-of-preference between the two options (as the players could have strong preference of one choice over the other), there is no significant difference between the pro-social and the selfish behavior10. A more detailed meta-analysis of a large number of experiments suggests that things are more complicated, claiming that deliberation inhibits what they called ‘pure cooperation’, however it does not appear to inhibit the strategic cooperation11. Studies using fMRI scanners to monitor brain activity of human subjects participating in game theory experiments show that selfish participants will cooperate when they have incentive to cooperate: consequently cooperation can also be a result of longer deliberation, not just intuitive acts of pro-social individuals13.Decision times have emerged as a new important aspect in experimental game theory. They have been measured in a wide range of games like: Ultimatum games14  for pro-social and selfish decisions. It was assumed that players think less to make intuitive decisions and longer to make deliberate decisions, and therefore concluded that the type of decisions with smaller RTs are intuitive and those with the longer RTs are deliberate. This simplified approach might be the reason why, thus far, a consensus is far from being reached. There are a number of reasons why tracking just the mean RT values is not a good idea. First, the experimental RT distributions have a number of properties which prevent it from being properly described by only mean value and standard deviation 17. It thus represents a view complementary to other models describing the subject’s actions. It focus on the shorter timescale18 of neurobiology rather than attempting at finding the cause of observed behavior as a consequence of natural selection (e.g. evolutionary dynamics), learning process (e.g. operant conditioning), or of the individuals’ strategies . It has in particular the unique feature of precisely outlining the statistics of decision times, while it is intrinsically limited in its ability of identifying the causes of single actions taken.DDM connects neuroscience to economic behavior by approximating the dynamics of the neurobiological process in act while the decision is made22. The process is starting from a possibly biased initial condition, and the two options are associated to two absorbing barriers  needs to determine as quickly as possible the direction of a cloud of dots24. Comparing empirical RTs with the model allows us to evaluate its free parameters  the drift rate v is a measure of subjects’ ability to gather evidence, also dependent on task difficulty; (iii) the bias z represents the a-priori inclination for one of the alternatives . We also introduce a fourth parameter, the non-decision time t0, that accounts here for the perceptual and motor processes associated with the task, processes which play a very minor role here given the longer characteristic timescale of the RTs in our experiments.More in detail, in the DDM at each moment subjects randomly collect evidence in favour of one of two alternative choices, which are in our case cooperation and defection. The continuous integration of evidence in time is described by the evolution of an one-dimensional brownian motion (see Methods), whose stochastic character is a consequence of the noisy nature of the evidenceCteo = 0.5. Using the DDM, the deviation of the experimental cooperation rate C from random behavior can be decomposed into a contribution due to v  and a contribution due to z (intuition). If only the bias is absent, one would expect a cooperation rate Cteo. Therefore, we identify as contribution of the rational deliberation the difference ΔCrat = Cteo − 0.5, and as contribution of the intuitive bias the difference between the empirical value and what expected without bias ΔCint = C − Cteo.In absence of bias or drift, decisions are completely random and the cooperation rate is expected to be For quantifying how much a decision is influenced by the a-priori bias, in this paper we introduce the ‘rationality ratio’14, and the model is still largely unknown in the experimental game theory community. In the last year, some experiments have showed that DDM can be applied to economic experiments, however only in situations when the player has the full information. For example when the outcome does not depend on the action of the opponents such as in the Dictators game27, or when the action of the opponent is already known, such as on the receivers side of the Ultimatum game25. The decision making in the aforementioned works is value-based, but not strategic. The players know exactly how much will be their payoff for each decision, and this payoff does not depend at all on the actions of other players.The application of DDM to more deliberate economic decision making has been uncertain until recently26 takes into account how much the players care about themselves and about other players. A similar approach to decisions in complex strategy games would require the development of a non-trivial addition to the model trying to predict the decision of the other player, but there is no obvious way to include this in the model.Here we extend the use of the DDM to outline the cognitive basis of cooperative decision making and characterise the evolution of a subject’s behavior when facing strategic choices in game theory experiments, where the decision is dependent on the unknown action of the opponent. The application of DDM to Prisoner’s Dilemma games is not obvious given the current state of the art, since in the deliberation process includes forming an opinion on what the other player would do. For example, the model proposed in29: (i) a pairwise game with 16 players associated in fixed couples, iterated 100 times; (ii) a multiplayer game on 13 × 13 lattice with 8 neighbours, where in every round each player makes a single action (cooperate or defect) which applies to a game against each neighbour. All participants of the multiplayer experiment played a total of 165 rounds in 3 separate phases: fix1, rand, fix2 (in that order). In the phases fix1 and fix2 the network is fixed and all players play with the same neighbours. In the rand phase the network is randomly shuffled after every round, so the players always play with different neighbours. . The alternation between a static and a changing network was originally intended as a sort of “control” in the experimental setup. Serendipitously, it allowed us to compare a more complex task  with an easier one , while in the pairwise game an almost full cooperation is attained after about 70 rounds see Fig.  right. TIt might be surprising that we chose two experiments with such different setups. However, we have a good reason for that. We originally only analysed the large network experiment, but eventually we realized that we need to test if our conclusions would still hold in cooperative environment. Achieving cooperation in the large experiment by tweaking small number of parameters was very uncertain. Furthermore, this kind of experiments where almost 200 people play the game at the same time are logistically very difficult and quite expensive. Finally, it was unnecessary to have a similar experiment, when the only thing we wanted to know is how the cooperative environment influence the bias and the drift. Therefore we opted to perform the simplest experiment where we knew the cooperation will be established. Evidently, since the details of the two experiment are quite different, in order to avoid drawing any conclusion that could be influenced by these differences here we only discuss the different cooperation level finally attained.t where we separated RTs for defections and cooperations by assigning negative RTs values to the defections. The curve is normalized to one considering both positive and negative values, therefore the larger area under the negative curve, as compared to the positive curve, corresponds to a larger number of defections in the experiment. The comparison between the empirical scattered data and the theoretical curves (solid lines) shows that DDM successfully fits the empirical RTs of the different phases of our experiment. In the following, we show what one can observe by tracking the evolution of the DDM parameters in our experiments. This allows us in general to describe the learning process of the subjects of game theory experiments from a novel perspective aiming at distinguishing between rational deliberation (described by the drift v) and intuition (associated to the a-priori bias z). The fits have been performed using the HDDM python tool30, which fits simultaneously the distribution of decision times for cooperative choices, defections, and the fraction of cooperation using Hierarchical Bayesian Estimation. In Fig. a ≈ 5 secCorresponds to the perceived difficulty of the task and therefore drops over time in all experiments. In both experiments, the threshold starts from the same value of see Fig.  Then, tha is the main factor determining the average decision time. In Fig. z = 0.5, v = 0, see Methods). The value of the average decision time in our experiment is thus dominated by the caution with which the players consider their response. The decisions become thus progressively quicker mostly because less information is demanded for the final deliberation. We emphasise that the fit is done simultaneously for both decisions, therefore the observed evolution of the threshold parameter equally influences RTs for both decisions.The value of v is zero )−1, as expected for no bias (z = 0.5) and with the threshold observed at the first round , as we have in the first turn. In Fig. −5 and 3 · 10−4 respectively. The trend is instead not perfectly monotonic for the pairwise experiment, and if we consider the last part of fix2. The sawtooth shape of the evolution of z is not reflected in the cooperation rate, because the drift quickly becomes, and remains, the dominant factor in the decision process . In Fig. R we introduced in Eq. R = 0 while in case of totally rational decision R = 1. In our experiments, the numerical result for the first rounds in both experiments suggests a compromise between rational and intuitive behavior (R ≈ 0.5). It is however important to remark here that for both experiments the value v = 0 lies within the margin of errors for the first rounds, and consequently our results are not inconsistent with the hypothesis that the first round are dominated by intuition (R = 0). If for the first rounds we have a balance between intuition and deliberation, it then appears clear that as the players learn how to play the game, their behavior becomes ultimately rational as R grows reaching values close to 1.In Fig. 8 where the faster decision is the cooperative one. Consistently, in interative experiments, which are averaging the first round with a relatively large number of the later rounds3 this trend has not been observed.These results show how rational deliberation, based on the gathered information, quickly becomes dominant over an initial intuitive bias towards cooperation. The role of intuition is still decisive when any information on the expected behavior of other players is lacking, as in the first rounds or in one-shot experiments32, after which it defection becomes the quicker answer.Intuition is not however strictly suggesting cooperating, as the bias can turn towards defection after playing multiple rounds in an un-cooperative environment. Nevertheless, the resilience of the bias we observe between the phases of the multiplayer experiment confirms that we might have a ‘natural’ intuitive optimism concerning the social behavior of the others. This is because, although the initial attitude leans toward faster decisions associated with cooperation, learning the game implies becoming a rational player. If the rational choice is to defect, as in our multiplayer experiment, spontaneous altruism is just a transient effect possibly due to optimistic initial expectations17 perspective. This kind of statistical approach is possible when the datasets are large enough to make it possible to statistically test the exact shape of the empirical decision times distribution. In our case, the multiplayer game with 169 players allowed us to study every round separately to confirm the remarkable ability of such a simple model to summarise very rich experimental results with up to 8000–10000 decisions. Tracking the evolution of the drift speed v, the accumulation threshold a, and the initial bias z during the experiment offers a new perspective into this learning process. This would not be possible by only studying the average decision times and permits a new interpretation of differences between experiments, contexts, experience, inclinations and strategies.Our results above are not only proof that the use of the Drift Diffusion Model can be extended to the description of the complex strategic actions taken during game theory experiments, but illustrate how an accurate modelling of decision times allows us to get new detailed insight on human decision process from a neuro-economical10, as it can be seen as representing how the different utility between the chosen and unchosen options is estimated during deliberation21. When the context of the game suggests the players to defect, cooperative decisions are faster only in the first round, or equivalently in one shot games. In this case, the decision is indeed dictated by intuition, since there is no game experience to base a rational decision on, and the cooperation level equals the value of the bias z. But if then the decision process is repeated, rational deliberation becomes dominant over an initial intuitive bias towards cooperation. As we learn the game, the value of |v| grows as the process of integration of evidence becomes more efficient, allowing us to make faster but more rational decisions. This process is coupled with a progressive drop in the threshold a, as players also become less cautious and demand less information for the final deliberation, and with a rapid accommodation of our perceptual and motor systems to the task at hand embodied by the non-decision time t0. The transition between naive and informed decisions demands only a few rounds  and intuition (associated to the bias) than simply comparing the average response times. We see here that it is not in general true that faster decisions are dictated by intuition, nor that the intuition necessarily suggests cooperating. The drift in the DDM embodies the effects of strength-of-preferenceSee Fig. , after w15.More in general, the possibility of monitoring in real time the learning process by studying a single and easy to measure quantity, response times, allows for the development of new practices to test whether the players  have mastered the task and to eliminate any initial transients by verifying if they have reached a steady state. Future studies can take advantage of the apparent universal validity of DDM, the range of use of which spans from memory retrieving, to perceptual, value based, and strategic decision making. Indeed, this suggests that the neurophysiology behind all these types of decision is similar, and that a set of decision models with a more elaborate architecture, which can be re-conducted to DDM under some parametric choices, represent natural candidates for a more detailed description of strategic decision makingThe game theory experiment was performed in April 2009, at the Universidad Carlos III de Madrid in Spain. The participants were 169 students from the Engineering Campus of Leganes. The subject’s age ranges between 18 and 26 years old. They played a Multiplayer Iterated Prisoner’s Dilemma game on a square lattice, simultaneously with eight neighbours . The lattice had periodic boundary conditions as if the players were placed on a torus. They had to choose one action: Cooperate (C) or Defect (D), which would be then applied to the game with each of their eight neighbours. For each neighbour, they would earn the payoff which is presented in the table:R = 7 cents, if they defect and the other person cooperates they earn a “Temptation to defect” T = 10 cents, and if the other player defects they earn nothing. Therefore “Punishment for mutual defection” and the “Suckers payoff” are both zero P = S = 0. Notice that this is a so-called weak Prisoner’s Dilemma, meaning that “Punishment” and “Sucker’s payoff” are here equal. If the other player defects, the focal player earns nothing in any case and there is no cost in switching to cooperating. The reason for choosing a weak Prisoner’s Dilemma setup is to make the game more cooperative than a strict Prisoner’s Dilemma. In the original experiment, which was designed to test if the cooperation could be established, it was important to set the conditions as favorable to cooperation as possible. Here, this particular choice is however not relevant. For each round, the total payoff for each player is then calculated as a sum of the outcomes from all eight games. At the end of the experiment, the participants are rewarded proportionally to the payoff accumulated in Euros.If they both cooperate, they each earn a “Reward for mutual cooperation” 33. Understandably the particular choice of the examples could frame the participants, however the risk of them not understanding the instructions was far greater and the examples were chosen to be balanced and the least likely to frame the participant in one way or the other. It should be also emphasised that all students participated in all three phases of the experiment. Therefore the different phases are not independent, since in each phase they have the experience from the previous one. Consequently, the phase we call rand is not a proper control of the experiment, in which the player without the previous experience would play the game. In order to avoid having players defect in the last round, knowing that there is no more game to play and therefore no reason to cooperate anymore, the end of the game was decided randomly by computer and it was at round 47, 60 and 58 respectively for fix1, rand and fix2. The data set is very large, making in total more than 27000 decisions. The software used in the experiment was developed in PHP, javascript with python controlling the background processes. The participants had 30 seconds to take an action afterwards the computer would make a decision for them, but there is no countdown to the deadline shown. The automatic decisions were excluded from the analysis. As the primary purpose of the experiment was not a precise measurement of the decision times, they were recorded on the server, not on the client. Therefore there was a certain delay in their measurement. We estimate these delays by analyzing the times recorded for the automatic decisions, which are made by the client in exactly 30 seconds. This recording goes under the same procedure as the other times. On average the delay in the system was less than 0.03 seconds, which makes it only ≈0.5% of the average decision time. More importantly, the standard deviation of the delays is even smaller of around only 0.01% (4 · 10−4 seconds). Thus, their influence on the shape of the decision times distribution is only of a negligible shift that gets included in the non-decision time t0. Three outliers, with delays of more than 30 seconds, were excluded from the analysis.All participants experience the following steps: (i) login; (ii) read the instructions; (iii) test if they understood the instructions; (iv) the experiment itself. The experiment has three phases: fix1 , rand (network reshuffled after every round), fix2 . In the original paper, these phases were called exp1, control and exp2. Here, we are renaming them in order to provide more intuitive names for this specific context. The size of the network was a very important factor for the original experiment. In particular, the objective was to have a number of players an order of magnitude larger than any previous experiment. Because of the number of participants needed, it was impossible to have multiple sessions. However, we believe that the system is large enough to self-average, as a single realisation of a large system can be equivalent to averaging over a whole ensemble. In any case, the lack of multiple session does not have any influence on the particular analysis in this paper. At all stages, the information provided to them on the screen . However after 30 seconds were up, they could still play, but would be given a warning to take action immediately. In most rounds all players took their actions in a much shorter time than the given 30 seconds limit. The experiment lasted less than 45 minutes from the moment participants entered the room until the moment they left. We adapted the software from the previous experiment to be used for this one.20. For the pairwise experiment, ethical approval by reference number ECHW2015_3 was obtained from the Ethical Commission for Human Sciences at the Vrije Universiteit Brussel to perform this experiment.Both experiments were performed according to the standards of socio-economics experiments. The participants’ anonymity was always preserved and they were never deceived during the experiments. All participants signed an informed consent. For the multiplayer experiment, please take a look at the Ethics statement in the original paperx(t) as a one-dimensional brownian motion with diffusion coefficient v:For the description of the response times distributions for binary decisions, we use a statistical decision model, the Drift Diffusion Model (DDM), which has the advantage of being defined by a simple linear, first-order, stochastic differential equation. In the DDM, at each moment subjects randomly collect evidence in favour of one of two alternative hypotheses. The continuous integration of evidence in time is described by the evolution of dt the quantity x(t) is increased by vdt (drift term) plus a noise v and ξ(t) is a white noise. In absence of boundary effects, the probability density P of the solutions of Eq. (μ =  = x(0) + vt and variance σ2 = Dt.For each entclass1pt{minimax = 0 and x = a and an initial condition x(0) = z · a with z ∈ , it is well defined the commonly called “gambler’s ruin problem”35, where x(0) represents the initial bankroll of the gambler, the absorption at x = a represents the gambler leaving a possibly unfair game (if v ≠ 0) after collecting her target winnings a, and the absorption at x = 0 represents the gambler’s ruin. The probability distribution of the times at which the process reaches the origin x = 0 before reaching the exit value x = a is known as Fürth formula for first passagesGiven two barriers at t and is characterised by an exponential tail.which represents the probability distribution that a gambler will be ruined at time v, D and z being interdependent, it is common practice to set D = 1 and use only the other three parameters to fit to the data with the curve P = P. This simplification is equivalent to re-defining the process in Eq. .The parameters s in Eq.  as \docux = 0 to defection and the barrier x = a to cooperation. When v is positive, the gathered information for the deliberation is tendentially in favour of cooperation, while when it is negative the evidence gathered is mostly supporting defection. The module of the drift speed |v| is the signal-to-noise ratio of the drift representing the unbalance in the amount of evidence supporting the two alternative options. |v| depends upon the difficulty of the decision. The lower |v| the more difficult the task36. The initial condition x(0) here describes the biases prior to deliberation. An unbiased initial condition would be z = 0.5. For values above 0.5 the decision maker has a bias toward cooperation and below 0.5 the initial bias is toward defection. The probability distribution P thus describes the decision times of rounds where the ultimate decision is defection. Conversely, the probability distribution to associate with cooperative rounds is P. The area under the curve 30 fits at the same time the non-normalised distributions P and P, thus evaluating at the same time the decision times for cooperation and defection, and the fraction of cooperation.For describing decisions in a Prisoner’s Dilemma experiment, we associate here the barrier at x(t) is associated to the difference in the amount of evidence supporting the two alternatives. To better illustrate this point of view, it is convenient to shift the process so that an unbiased initial condition, which lies exactly between the two barriers x(0) = a/2 (z = 0.5), is mapped to x = 0 and the barriers become symmetrical x = ±a/215. The new quantity x′ = x − a/2 can then be associated to (i) the difference in activity (firing rate) in two neuronal populations, each associated to one of the two alternative options (neuroscience perspective16); (ii) the difference between the utility expected for the two options (economic perspective21). In both cases, |x′| represents the expected benefit obtained from taking the decision which is better supported by the evidence. Again from the neuroscience perspective, the final decision is then made by a second brain circuit that acts downstream detecting the event of the threshold crossing37.In the modelling of decision times, the quantity a is the threshold parameter of the DDM, representing the amount of information needed for the final decision. Its value can be naturally associated to the average first passage time of the diffusion process without drift (v = 0) in the unbiased case (z = 0.5).The distance between the barriers t〉 for this unbiased case has the form15Indeed, the average value of the decision times 〈v → 0, we have a pure diffusion process and Eq. , or aggregated over five consecutive rounds in the pairwise experiment (90 response times). We are therefore fitting the model to the group as if the are all copies of an ‘average subject’. This assumption is of course limiting, but our choice is motivated by the need of proposing a method that can be scaled to what is commonly done in game theory experiments, where the task is repeated only few times, and often only once.The data were fit using an open-source python library, the HDDMSupplementary Information