Research indicates transgender individuals have a higher incidence of depression, anxiety, victimization, and discrimination. SE™ tools may support transgender/gender non-conforming individuals to increase resilience in the face of discrimination and social injustice.Somatic ExperiencingN = 7) pilot study assessing the impact of a 10 session SE™ based group treatment on depression (PHQ-9), anxiety (GAD-7), somatic symptoms (PHQ-15), quality of life (QoL) (WHOQoL-BREF), and coping with discrimination (CDS) for a cohort of seven individuals identifying as transgender/gender non-conforming. Materials were created in collaboration with members of the LGBTQIA community. Care was taken to be inclusive of gender non-conforming identities and culturally responsive in design.This study is a pretest posttest within group  p = 0.097, SD = 3.31 and a modest effect size of d = 0.68; somatic symptoms (PhQ-15) p = 0.093, SD = 3.52 and a modest effect size of d = 0.72.Participants described their gender identities as: non-binary, female to male, male to female, and gender fluid. Participants had significant increase in psychological QoL  (WHOQoL-BREF) These data indicate that a brief 10 session intervention of SE™ could have a meaningful impact on symptoms of depression, somatization, and QoL for gender non-conforming individuals. Further research is warranted.First, this study has a small sample size limiting statistical power and generalizability. Second is a history effect. Less than 1 week prior to final data collection, there was a significant hate-motivated act in Florida targeting the LGBTQIA community. SE™ focuses on re-establishing an individual’s innate capacity for ANS, physical, and emotional regulation. According to SE™, trauma resides in the nervous system, not in the traumatic event itself, and is a relatively short-term, somatically based approach to resolving trauma  r05-03-15 FINAL.”Significant studies indicate that the Transgender population reports a higher incidence of depression, anxiety, mental health concerns, suicide, lack of access to health care, unemployment, violence, victimization, and discrimination. Additionally, Transgender persons must live their daily lives in a world constantly challenged by social expectations regarding their gender expression and presentation, causing undue stress and anxiety.The Patient Health Questionnaire-SADS (PHQ-SADS)  is a brit-Tests for each variable and significance will be reported at a p-value of 0.10. All comparisons groups that do not conform to assumptions of within groups t-Test, i.e., does not meet criteria for normal distribution will use Wilcoxon single rank t-Test. Pearson’s r will be used to assess the correlation between CDS scores and outcomes on PHQ-SADS and WHOQoL–BREF. Normality of distributions will be assessed using visual analysis on Q–Q plots as well as Sharpiro–Wilk test. Data that do not conform to the assumption of Person’s R will be assessed using Spearman’s Rank. Data will be assessed for outliers using visual assessment via scatter plot, SDs from the mean, and the Dixon outlier test.This is a pretest, posttest within groups design. Main effects will be assessed using paired one-way A convenience sample of adult group participants who may be experiencing and/or have been formally diagnosed with Gender Dysphoria was solicited from within the South Portland, ME, USA Transgender community. Participants needed to self-identify as “Transgender/gender non-conforming/gender variant,” which describes people whose gender identity or expression is different, at least part of the time, from the sex assigned to them at birth. They could also be in a process of transitioning from their physiological gender to their self-identified gender in a binary context. Individuals who passed pre-screening were given the opportunity to participate in a 10-week SE™ informed group therapy intervention model.Appropriate candidates were provided with additional instruction at an informational meeting. During this event, candidates had an opportunity for discussion of the procedures, the risks, and benefits of participation in the study, their rights as a research participant, confidentiality procedures for their personal information, time commitment, group schedule, and duration. Participants were informed that treatment referrals were available should they have an adverse reaction to the testing or else discover a need for care. All went through a three-level informed consent process and completed a written informed consent acknowledgment. They were apprised that any data collection is private, the group experience is confidential, and no personal data would be retained that could connect an individual with their measures. The signed informed consent forms and all subsequent collected data were stored separately, maintaining appropriate HIPAA compliance at all times. Participant flow is represented graphically in Figure The original design of this study was to consider the results of a comparison between two independent group experiences with groups running in two different geographical locations. Group parameters and curriculum would be held to consistency, and each group would be facilitated by a different author. Two well-established community locations equally offered a safe, consistent, familiar meeting space, ancillary participant support, as well as a source for soliciting participants. One location was to have been at Sunshine Social Services (SunServe), a local Gay, Lesbian, Bisexual, and Transgender + Social Service Agency located in Wilton Manors, FL, USA. The second location considered was Embodied Liberation in South Portland, ME, USA.Transgender support group promotion began in South Florida. Palm cards were printed, and distribution and community visibility occurred over a 6-month period at several major Transgender events significant to the local Transgender community. The hosting agency, SunServe, provided additional promotion through word of mouth and social media information distribution on their agency website and Facebook page. At the end of the promotional period, only 10 prospective participants expressing interest. Despite these group promotion efforts, we were only able to finally contact, screen, and identify 5 individuals for the group, from a total of 10 inquiries. Five of those who had expressed interest early on were no longer available once the group was ready to begin.The five identified individuals were contacted by telephone, and through e-mail and were provided with the exact dates and times of the introductory meeting and the 10 subsequent support group dates. On the evening of the introductory meeting, none of the five individuals attended, nor did they make contact with the facilitator to advise that they could not attend.–during the day prior to and the day of the initial group evening, weather in South Florida was rainy, damp, and there had been an unusual threat of tornado touchdowns in the area.–participants in this group may need additional reminders, i.e., phone call/e-mail just prior to the date of the group.–the group facilitator identified as a gay, cisgender male.–although the facilitator has been involved in the South Florida LGBTQ Community for over 18 years, he is not well known in the Transgender Community.–the group facilitator had only been able to make telephone and e-mail contact with participants, there has not yet been an opportunity to meet face to face.We, along with SunServe staff, met and discussed possible reasons for a lack of turnout on the first evening of the group. Possible South Florida site factors included:–the group facilitator identified as a Transgender person.–the facilitator is a member of the South Portland Transgender Community.–the facilitator is a practicing massage therapist in the South Portland Community for many years.–the facilitator has an established relationship and is well known and embedded in the community.Our study was finally conducted using only one group, without an opportunity for comparison, with participants selected from the South Portland, Maine community. Group members were solicited from the community in a similar fashion, through palm card distribution, social media, and word of mouth. Eight participants were identified and engaged in the study. Positive site factors considered included:Selected candidates were given a packet of measures consisting of a demographic survey, the PHQ-SADS, the World Health Organization QoL–BREF (WHOQoL-BREF) scale, and the Coping with Discrimination Scale (CDS). The same set of measures, with the exception of the demographic survey, was administered once again at the conclusion of the 10 weeks of sessions.The intention of the group is to assist participants who may be experiencing Gender Dysphoria to develop: a general increase in awareness of resources, skills, and abilities, increased skills for affect management, increased skills for nervous system regulation, increased capacity for tolerance of “being in their bodies” and managing “real life” situations relating to confrontations and microaggressions, increased resiliency, a decrease of negative symptoms such as depression, anxiety, and feelings of social isolation. Subjects who participated in the study may have experienced discomfort while answering some pre- and posttest questions, and during participation in a therapeutic group process. But there is no clear data indicating iatrogenic effects of answering the mentioned measures. Further, subjects may have experienced discomfort relative to the emergence of material relative to their therapeutic process during their group experience. The group setting is intrinsically designed to be a safe, supportive environment to nurture growth and change. Heck  indicateMaintaining the ongoing safety and support for participants at all times was an utmost priority in this study. A set of screening questions was developed and utilized to identify those individuals whose capacity for managing these possible periods of discomfort and shifts in cognitive awareness is compromised by a history of high risk for self-harm, severe mental illness, or active use of mind-altering substances. For these individuals, participation in the study was not recommended. Group members who were selected to participate were asked to not be under the influence of alcohol or other mind-altering substances during participation in groups.Several additional factors were considered for the design, creation, and implementation of a therapeutic group model that could be both extremely effective for as well as hold the interest of participants. Such factors included the safety of the setting, both actual and perceived, group facilitation, program consistency, a high degree of being SE™ informed, access to ancillary supports outside of the group experience, and ease of replication.The group met weekly, in 10 consecutive 90-min sessions, was free of charge to participants, and was offered to appropriately screened adult individuals aged 18 and over who identified as transgender, gender non-conforming, or gender variant. Participants could also have a formal diagnosis of Gender Dysphoria, which was not a requirement for inclusion. We chose to impose an age restriction upon our participant sample, solely to avoid the need to obtain parental consent for participants under 18 years of age. We firmly believe that the design and benefits of an SE™ informed Transgender Support and Healing Group experience would be equally appropriate and effective for young adults and adolescents, and with some modifications could be effective for working with children as well.It was important to provide group participants with a consistent, appropriate, safe, and familiar space to meet each week. Having the perception of a predictable and safe environment is crucial for participating individuals, as this has been shown to aid in adjustment in a variety of traumatic situations. It is important to have knowledge of the many ways intersecting oppressions can inform participants’ perceptions of a predictable and safe environment –42. The Participants were solicited from and the group was purposely held within the same South Portland, ME, USA Transgender community. Group facilitators were clinicians or practitioners with a professional level of training, experience, and expertise regarding group work process and SE™ work, in our case one of us. The role of the primary group facilitator was to guide the group through the SE™ informed curriculum and associated exercises, utilizing their skills as a practitioner. As well, the primary group facilitator works at tracking the nervous system at multiple levels, that of individuals, and that of the collective nervous system of the group. The facilitator provides support for the group and individuals, as the capacity to tolerate more intense experiences increases, within the range of resilience . An optiSubjects who participated in this study may also have had the added benefit of already being engaged in other pre-existing relationships of supportive psychotherapy and/or case management, by virtue of their possible affiliation with community providers or agencies. Otherwise, they did not receive any unique or additional treatment, procedures or information other than the participation in the group experience and data collection.Clinicians and staff of affiliated community agencies can play an integral part in providing participant support outside of the group setting, as participants work on change in the group process. Considering this, prior to the start of the group series, local community agency providers were offered an orientation regarding: concept and principles of SE™ work, what will be done in the group setting, predictable outcomes and responses of participants making change during their process, ways that providers can be supportive allies during this work. The option was given for participants to grant written permission for individual providers and group facilitators to be able to communicate regarding any emerging concerns relating to group members’ participation in both individual and group therapy sessions.®, a well-recognized treatment model for working with dysregulation syndromes such as PTSD, anxiety, and depression; focusing on re-establishing an individual’s innate capacity for ANS, physical and emotional regulation. SE™ informed work involves the application and integration of SE™ theory, concepts, and practice principles into a treatment model; in this case a Transgender Support and Healing Group experience. The goal being to address the care and treatment of and give attention to biopsychosocial affects upon a gender diverse identity which can contribute to some form of Gender Dysphoria. This study targets how SE™ informed work, and more specifically working with the “felt sense,” can assist Transgender, Gender-Non-Conforming, and Gender-Variant persons with alternative ways to help manage, re-establish, and build capacity for ANS, physical, and emotional regulation. There is a body of literature that indicates increased skills in self-regulation leads to increased resiliency and coping. In the interpersonal process of an SE™ informed group setting, the survival physiology and activation patterns related to fight, flight, or freeze responses can be re-worked in vivo, allowing for the alteration of interpersonal symptoms often created from these responses —55 min, and wrap up/closing—15 min. More detail is provided in Table Integral and vital to SE™ informed work is the concept of resourcing. Initial treatment must involve reinstating lost resources, learning new ones, and strengthening those that exist. This concept is fundamental in many body-based approaches and is essential when working with traumatized clients. Ogden et al.  explain We believe that the group setting itself can provide a source of resourcing for participating members. Much like the herds of animals that Peter Levine studied in the wild, a group can operate with a “collective nervous system,” sensing, identifying and responding to common threats that may evoke in the group process and dialog . ConversWe believe that transgender history can be a powerful resource, as many transgender individuals lack a reference to positive role models, pioneers, “heroes,” and events in history, instead of being constantly reminded of a sense of insignificance and the negative challenges they must confront. We integrated a “trans history” exercise into our curriculum so that group participants could take positive and affirming ownership of some of the many notable origins of “being transgender.”Re-orienting in the here and now is an integral part of an SE™ informed process and being in the group presence can serve as part of this important function. The capacity for embodied social engagement is intrinsically self-calming and appears to make a powerful contribution to health and happiness, according to Levine . This caAt times, it would be necessary to assist the group with ways to collectively re-regulate participants’ ANS, especially after highly charged material emerged from within the group process. We developed a tool and coined it “Movement Interlude,” that could be used when group facilitators recognized high levels of group energy that may be too collectively overwhelming or too difficult for the group or a majority of individuals to manage. This tool is a physical movement intervention when the need for activation energy discharge is necessary. The possibility of use of this interlude and its purpose was previously discussed with group members, and a brief practice run occurred. One group member was invited to lead the group in a brief, creative random movement exercise  that allowed the opportunity for collective and individuals’ energy to be discharged. The exercise would take approximately 2 min, with an additional 2 min for grounding, settling, centering, and re-focusing. During this time, group members were encouraged to share about what they noticed for themselves and for the group.N = 7 participants had complete data for the WHOQoL-BREF. As shown in Table p = 0.003) was found to display both a statistically significant and a clinically meaningful improvement. This change would also represent a meaningful increase a participant’s ability to psychologically respond to stressful life events. The effect size for this change was moderate with a Cohen’s d = 0.71. All other domains were non-significant. The below data sets met criteria for normality using visual assessment of line on Q–Q plot as well as Sharpiro–Wilk test.p = 0.003) on the WHOQoL-BREF, effect size d = 0.71. The increase in psychological QoL would be clinically meaningful and likely would represent an improved overall psychological sense of wellbeing.Figure N = 7 participants had complete data for three clinical measures. These measures assessed depression (PHQ-9), anxiety (GAD-7) and somatic symptoms (PHQ-15). None of these measures were significantly reduced. The data did indicate a reduction in clinical symptoms between pretest and posttest in all three measures. These changes could have a modest impact in an individual’s overall functioning. Data sets shown in Table These data indicated a modest reduction in clinical symptoms between the first and second measurement point Figure . Changesp-value of with 0.024, 0.028, and 0.006 compared to each of the above measures posttests. The data associated with this participant also ranged from 2 to 3 SDs higher the mean. There were several notable factors when comparing this individual case to others that may indicate why this individual does not fit with the group mean. These are higher level of reported illness including chronic pain, insomnia, and other chronic health conditions. This individual also had a higher level of psychosocial stressors than others reporting a history of homelessness and higher levels of exposure to violence. It may be that this individual case would represent an interaction between the impacts of gender identity and chronic illness. In further studies adapting the group materials to address chronic illness could impact this outcome.One member of this group was found to likely be an outlier in using the Dixon outlier test with p = 0.093 and 0.097, respectively. These data had a modest effect size using the Cohen’s d test. The findings indicated effects size of d = 0.68 for depression symptoms and d = 0.72 on somatic symptoms. Anxiety symptoms were not found to be reduced significantly. Data sets shown in Table Due to the above factors, it appeared worth considering the data set without this case included. The data without this case included trended in both a more clinically meaningful and statistically significant direction. With this outlier excluded data trended toward significant reduction in symptoms of depression and somatic symptoms with p = 0.093 and 0.097, respectively, with a moderate effect size of d = 0.68; and d = 0.72, respectively. Symptoms of anxiety did not show significant reduction between measures. Depression symptoms were reduced by 3.6 points between measures, somatic symptoms were reduced by a mean of 2.17 points, and anxiety symptoms were reduced by a mean of 1.7 points. These changes would indicate a reduction in clinical symptoms that likely could be observed by both a clinician and would be experienced as meaningful by the participant in the study.Reduction of clinical symptoms between pretest and posttest indicates an improvement in mental health. Figure Two participants who completed the PHQ-9 were in the clinically significant range. Over the course of the treatment these individuals moved below the clinically significant range moving from 10 to 8 (moderate depression symptom score to mild/no depression score) and 12 to 2, respectively (moderate depression symptom score to mild/no depression score). Two individuals had a modest increase in symptoms, with one individual having a 5 point increase shifting into the severe range and another an increase of 3 points remaining in the minimal to no symptom range. Figure On the measure of anxiety symptoms (GAD-7), there were two individuals that scored in a range that is considered likely to indicate the presence a clinical diagnosis of anxiety. Their scores were 11 and 15, respectively. These individuals’ range had what would have been a meaningful reduction in their clinical symptoms moving from 11 to 2 (moderate symptom score range to mild/none score range) and 15 to 5 (severe symptom score range to mild/no symptom score range) on the GAD-7.Those in the subclinical range had no significant reduction, either staying the same or displaying a modest increase staying in the mild to no symptom range. Three individuals had an increase in anxiety symptoms, these ranged from a 4 to 6 point increase. Two of these individuals had near 0 initial symptoms and moved to a score of 5 . One individual had an increase in anxiety from a score of 10 (mild anxiety) to a score of 16 (moderate anxiety range).It is possible that increased somatic awareness increased their awareness of the physiological stress that they were experiencing. The individual who showed the highest level of increase also was in the group with the highest rate of health and mental health conditions reported, the highest exposure to gender-based violence, and had experienced significant life stressors. This might indicate a need for increased focus on chronic conditions and self-care in future adaptations of the curriculum. Figure Three participants had a reduction in symptoms. Two of these individuals moved from moderate symptoms to mild to no symptom range. These individuals had a 7-point and a 6-point reduction, respectively. And the other individual who showed symptom reduction went from high mild symptoms to low mild symptoms (2-point symptom reduction). Three individuals had an increase in scores. Two displayed a modest increase of 1 point between each measure and one participant had a 5-point increase in score. The individuals who had increased scores all had higher levels of overall health concerns, two reported three incidents of exposure to violence related to their gender identity and all three had experienced homelessness. Figure There were four individuals who completed the CDS and all other measures. Two of these individuals had an increase in sub-scale on CDS that measures coping through education (a positive empowered form of coping). For these participants, there were larger reduction in symptoms on all levels of SADS-PHQ than those with reduced education score on CDS.A total number of eight individuals were screened as appropriate candidates to participate in the SE™ informed group study. One individual later dropped out from the study, and we will report responses of all eight individuals. All eight identified as “white.” One participant also identified as “American Indian/Alaska Native,” while another also identified as “Hispanic/Latino/Latina.”Of the eight participating individuals, all considered themselves to be transgender/gender non-conforming/gender variant/gender diverse in some way. Five were born biologically female and three were born biologically male, as indicated on their original birth certificate. In response to “what is your primary gender identity today,” one biologically born female identified as “male/man,” two biologically born males identified as “female/woman.” Other self-identified gender descriptions included: “non-binary demi-boy” , “no gender trans-female” , “non-binary, gender queer” , “no gender, some male/some female” , “unknown” .Participants were given the opportunity to respond to questions regarding whether they were in transition from a M → F or female to male (F → M) binary gender identity, and whether any were receiving medically monitored HRT. Seven respondents indicated that they were in transition. One (the group’s drop-out) was unclear in their gender transition response, had completed desired transitioning, and was not receiving HRT. A F → M indicated that they had not completed desired transitioning, and they were receiving HRT. Another F → M indicated that they had not completed desired transitioning, and they were not receiving HRT. A M → F indicated that they had completed desired transitioning, and they were receiving HRT. Another, who considered themselves F → M → FUnsure, had completed desired transitioning and was receiving HRT. Another who considered themselves F →?, had not completed desired transitioning, and was not receiving HRT. Another participant, who considered themselves non-binary, had completed desired transitioning and was not receiving HRT. One last respondent, who was ambiguous (answered yes and no) regarding whether they were in the process of transitioning, considered themselves “res binary,” had completed desired transitioning, had received top surgery and oophorectomy, and was not receiving HRT. Of the three participants indicating that they were receiving HRT, none were obtaining hormones from the internet.Participants were asked to respond with “not at all,” “somewhat,” or “strongly” to a listing of self-describing terms used in the Transgender community. Responses are presented in Table Determination of support systems, relationships, living arrangements, employment, school, and homelessness were explored. All eight respondents reported having a circle of friends or family that they could count on. Five participants identified as “single.” Of these five, one respondent was employed, a student, had never experienced homelessness, and lived with their three children in a home that they owned. All of the remaining participants reported having no children. Another single respondent was not a student, not employed, had previously experienced homelessness, currently lived with a roommate/friend, and paid rent. A third single respondent was not a student, was employed, lived alone, had experienced homelessness in the past, and did not answer as to whether they owned or rented. The fourth single participant had responded as both single and partnered, they were not a student, not employed, they had experienced homelessness in the past, they currently lived with a roommate/friend, and they paid rent. The final single respondent was not a student, was not employed, had never experienced homelessness, they currently lived with a roommate/friend, and they paid rent.The remaining three of eight respondents identified as “partnered.” One of them was not a student, was not employed, had previously experienced homelessness, lived with a spouse/partner/significant other, and paid rent. Another partnered respondent was a student, was employed, had never experienced homelessness, lived alone, and owned their home. The third partnered respondent also identified as having a non-monogamous relationship, was a student, was employed, had never experienced homelessness, lived with a roommate/friend, and paid rent.Participants were polled for having some common medical conditions. None reported having diabetes, heart problems, nor kidney disease. One reported having high blood pressure that was controlled with prescribed medication.Three study participants reported having a formal diagnosis of Gender Dysphoria. All three also saw a mental health provider for psychotherapy support, two of the three took prescribed medication for a mental health condition, and one of these two also saw a psychiatrist for medication management. A fourth participant was not sure if they had a formal Gender Dysphoria diagnosis but took prescribed medication for a mental health condition. The remaining four participants responded as not having a diagnosis of Gender Dysphoria. Only one of these four saw a mental health provider, and another one of the four took herbal supplements for mental health. Degrees of experiences of depression, anxiety, anger/irritability, difficulty sleeping, and pain were also solicited  from health-care providers because of their gender identity, while two acknowledged some form of mistreatment, and one did not provide an answer. Four participants denied experiencing any form of partner or domestic related abuse or violence related to their gender identity. Four acknowledged some form of partner, domestic related, or other violence in the past relative to their gender identity. Five of eight respondents acknowledged current verbal abuse or harassment in their day to day activities, while three did not. Five of eight respondents acknowledged other current or past types of harassment or abuse in their day to day activities, one did not, one did not because they were not “out,” while one did not answer.The exposure to experiences of discrimination can have a highly negative effect on health and mental health. Some previous studies have identified ANS functioning and long-term exposure to stress hormones as the mediating factor between stress, health and discrimination. Dysregulation in the ANS has been found to have multiple long-term health and mental health impacts. ANS dysregulation may be one of the moderating if not mediating factors that lead to increased rates of depression, suicide and mortality in individuals who identify as transgender or gender fluid. Increased, resilience and coping could improve ANS functioning. Developing skills that support improved resilience, increased ANS regulation and develop capacity to cope with experiences of discrimination could improve health, mental health, and QoL/well-being.p < 0.004). Increased psychological wellbeing would have a meaningful impact on the ability to be resilient after a stressor and to psychological adapt to challenges.These data indicated a modest clinically impactful and statistically significant improvement in one marker of resilience for individuals who identify as gender fluid and gender non-conforming, e.g., psychological QoL. Data showed significant increases in psychological QoL   and somatic symptoms (p < 0.097). These symptom reductions could indicate a reduction from moderate symptoms to minimal symptoms. This would translate to increased ability to effectively engage with work, relationships, and enjoyment of life.It also appeared that exploring these data with this data point removed would offer some meaningful insights. When this outlier case was removed the data showed an increase in the amount of clinical change into a more meaningful clinically significant range. The data also indicated significant decreases in symptoms of depression  who derives income from his practice. SH is an SE™ practitioner (SEP) who derives income from their practice and is a training assistant with the Somatic Experiencing Trauma Institute™. MC is a member of the board of directors for Somatic Experiencing Trauma Institute™, who receives a small yearly stipend for that work, and chair of SE™ research coalition, an unpaid position.


People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These metacognitive assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic. However, a computational framework that accounts for both confidence and error detection is lacking. In addition, accounts of dissociations between performance and metacognition have often relied on ad hoc assumptions, precluding a unified account of intact and impaired self-evaluation. Here we present a general Bayesian framework in which self-evaluation is cast as a “second-order” inference on a coupled but distinct decision system, computationally equivalent to inferring the performance of another actor. Second-order computation may ensue whenever there is a separation between internal states supporting decisions and confidence estimates over space and/or time. We contrast second-order computation against simpler first-order models in which the same internal state supports both decisions and confidence estimates. Through simulations we show that second-order computation provides a unified account of different types of self-evaluation often considered in separate literatures, such as confidence and error detection, and generates novel predictions about the contribution of one’s own actions to metacognitive judgments. In addition, the model provides insight into why subjects’ metacognition may sometimes be better or worse than task performance. We suggest that second-order computation may underpin self-evaluative judgments across a range of domains. People are often aware of their mistakes, and report levels of confidence in their choices that correlate with objective performance. These assessments of decision quality are important for the guidance of behavior, particularly when external feedback is absent or sporadic, and such metacognitive abilities are particularly well-developed in humans . UnderstIn the laboratory, the mechanisms underpinning self-evaluation of performance have been investigated by asking subjects to judge their confidence in simple decisions. As we will outline in further detail below, decision confidence can be defined as a subjective probability of a decision being correct , and is Formal models of decision confidence have focused on the role played by the decision variable—an internal subjective state that is influenced by incoming sensory evidence . For insHowever, a close coupling between decision variables and confidence is potentially in tension with a burgeoning literature identifying dissociations between performance and metacognition. There are systematic differences between factors affecting task performance and confidence in perceptual decisions, including attentional or stimulus manipulations , individSuch dissociations may arise for a number of reasons. First, the evidence contributing to decisions may be subject to further processing that introduces additional variability into confidence reports. This further processing may occur over space and/or time. For instance, metacognitive reports may require a neural “read-out” of confidence from decision circuitry . AlternaThis variety of performance-confidence dissociations has hitherto precluded a unified account of metacognition in decision-making. Here we set out to account for such dissociations in a general framework in which confidence operates as a second-order computation about one’s own performance. Our core proposal is that within a single individual, samples of sensory evidence underpinning decisions and confidence judgments are distinct but coupled. Such a distinction between decision and confidence variables arises necessarily in many of the situations considered above, and once this is formally recognized, sound statistical inference differs in key ways from that prescribed by first-order signal detection theory . In our It will turn out that this framework, inspired by the dissociations reviewed above, holds key implications for metacognitive computation in general. First, second-order computation naturally accommodates different behavioral manifestations of metacognition such as confidence and error detection within a common framework. The intuition, which will be formalized below, is that a secondary view on the decision problem is required for a system to view itself in error . Error mIn the sections that follow we compare the qualitative predictions of second-order computation to those made by first-order accounts with and without postdecisional processing, and evaluate these predictions against the empirical literature on decision confidence and error monitoring. We will show that first-order models are special cases of second-order computation that arise under particular noise conditions see . Our anad, such as whether a stimulus is moving left (d = −1) or right (d = 1). Second, the subject makes a response a to indicate their perceived state of the world . On each “trial” internal states X = [actXconfX] denote the decision and confidence variables. To make a decision, the subject chooses “right” if actX > 0, and left otherwise:We consider three classes of model of how a subject generates a report of confidence in his or her decision. All models have the same basic ingredients. First, we define a categorical world state, z as a degree of belief that a particular choice was correct , given a particular set of internal states X, model m and model parameters ϑ:We define the subject’s confidence X are generated from world states d. However, the models differ in how these states are coupled, and how confidence is computed, as described in the following sections.In all model simulations we assume Gaussian noise for how internal states actX is obtained from a Gaussian distribution conditional on the world state:In the simplest “first-order” model we assume that the decision and confidence variables are identical, such that the same internal state supports both choices and confidence. First, the decision variable confX = actX. Confidence is then a transformation of the posterior belief in d conditional on the action taken :d):The confidence variable confX is derived from actX plus additional information about the world state, newX:In the postdecisional model, the confidence variable newX as an additional sample of evidencenewX ∼ N. One can imagine different generative models—the key property here is that the true world state d is conditionally independent from the action a (and its decision variable actX), given the confidence decision variable confX. Informally, confX should provide all the information contained in actX. This will be satisfied, for instance, if actX and confX are both states of a perfect accumulator  and Confidence-rater (conf). The actor is carrying out a two-choice discrimination task as described above. Both receive internal samples actX and confX generated from binary world state d . We model these samples as draws from a bivariate Gaussian with covariance matrix ∑:We first describe a second-order model of confidence in another individual’s performance to provide the intuition for the within-subject case, and to demonstrate the symmetry between evaluating one’s own actions and those of another actor. Consider two individuals, an Actor  and partly distinct . The Confidence-rater’s job is to say how confident she is in the Actor responding correctly, or the posterior probability that the Actor’s action a was appropriate for the inferred state of the world d, conditional on beliefs about different sources of variability. To do this, the observer infers  the state of the decision variable driving choice (actX) from the confidence variable (confX):P ∝ PPThe covariance matrix has 3 parameters: σconfX but also one’s own action a and beliefs about the fidelity of the decision and confidence variables, captured by ∑. In other words, second-order inference reflects an active process of inferring the state of the decider, rather than a passive sensitivity to the difficulty of the decision. In The core of our proposal is that individuals generate confidence in their own performance by applying an analogous computation to their own actions . ImportaactX and confX arises: it may be that they remain segregated in the brain ; confX may depend on the same neural activity as actX at a later time point, or confX may reflect a noisy read-out of actX. The many possible relationships between actX and confX are flexibly accommodated via the parameters of the covariance matrix ∑. In the special case in which ρ = 1 and σact = σconf, the second-order model reduces to the first-order case, as on any given trial the same evidence supports both actions and confidence . Indeed, variants of the first-order or postdecisional models outlined above are optimal under limited cases in which the confidence computation has direct access to the actor’s decision variable. However, the computational considerations we highlight here apply to all but the simplest cases in which internal states underpinning performance are transparently accessible to those underpinning confidence.We note that these model variants are naturally nested, with each representing an extension of the previous case. The first-order model is a special case of the postdecisional model in which the decision and confidence variables are identical, and the postdecisional model is a special case of the second-order model in which In this section we describe qualitative features of first- and second-order computation, and relate these to key findings in the empirical literature.We begin with internal representations supporting decision confidence. Decision confidence typically increases with stimulus evidence for correct judgments, but decreases with stimulus evidence for errors  following an incorrect response therefore decrease in magnitude. To take a concrete example, suppose we have a leftward trial (d = −1). If the subject’s sample actX is + 0.05, she will erroneously respond “right” and derive confidence from a monotonic transformation of |confX|. But this subjective sample may have arisen from many different objective stimulus strengths θ, including both correct and error trials, and occur more often with some than others. When the experimenter then plots the subject’s confidence as a function of the externally manipulated variable θ, a divergent pattern of confidence emerges for correct and error trials. In other words, the X-pattern is due to the necessity of relating observed confidence to θ (which is unknown to the subject) rather than to confX (which is unknown to the experimenter).To simulate confidence as a function of stimulus strength we modified all models such that the sample mean depends on stimulus strength θ  that the action taken was inappropriate. As in the postdecisional model, an interaction with choice accuracy is also observed in the model’s internal state , the model’s confidence is not necessarily at chance (0.5). Instead, for the parameters used in this simulation, confidence when confX = 0 is around 0.7, due to the confidence computation also incorporating knowledge about the average reliability of actions, that is, σact (P(correct) < 0.5), but the simplest first-order model cannot.It is notable that such representations are precluded in the simplest first-order model because the same evidence drives both choices and confidence, resulting in a lower bound on confidence of 0.5 . In other models , C, confis, σact . In summactX and confX from world state d = 1 . By applying a neutral decision criterion, the observer erroneously responds “left” whenever actX is less than zero. However, whether this error will be detected depends on whether confX provides enough (positive) evidence in support of the alternative, correct response . Error detection is highest when σconf is low, because the confidence variable provides accurate information about the true world state. Notably error detection also depends on the correlation between the samples—as ρ approaches 1 (lower right quadrant of the heatmap) the model reduces to the first-order case and error detection is again precluded.The internal representations of the second-order model that support error detection are illustrated in These simulations of error detection are of course an oversimplification—the criterion for whether to report an error is itself under subject control, and may be adjusted above or below 0.5 in the face of changing incentives . The aimactX and confX are only weakly coupled. This influence arises because actions carry information about the subject’s internal states, leading a rational observer to incorporate her own actions as additional data when computing confidence. Consider y axis is the posterior probability that the current world state is rightward (d = 1) as a function of confidence variable confX. Intuitively, as confX becomes more positive, the model gains greater evidence that d = 1. However, having taken an action a, this inference is modulated, such that a leftward action reduces the belief in rightward world states, whereas a rightward action boosts it.A counterintuitive but important feature of second-order computation is that one’s own actions may causally affect subsequent confidence ratings, particularly if confX at 0. In the first-order case (gray line in d. The extent to which this modulation occurs is dependent on (beliefs about) the covariance of actX and confX. As the confidence variable becomes more noisy (σconf increases), the information provided by confX is less reliable and actions are given more weight (actX and confX increases (ρ increases), actions provide less new information about the possible values of d, and the modulation of confidence by action decreases , then they may compute their confidence without conditioning on self-action , respectively, and handles cases in which metacognitive sensitivity is either better or worse than performance.conf is increased  sets of parameter changes may lead to reductions in the second-order model’s metacognitive sensitivity. In the first, metacognitive sensitivity is impoverished (Type II ROC area is reduced) as the noise in the confidence variable σncreased . In the constant . In otheconstant .d′, which they label meta-d′  may be attributable to artifacts of estimation error or criterion variability. But in our experience, such values are routinely observed in empirical studies , the model reliably detects its own errors (d′ > d′. To demonstrate this we randomly sampled simulated data sets generated from a particular combination of σact (d′) and σconf (holding ρ constant at 0.5), and fitted meta-d′ to each dataset. d′ against meta-d′, color-coded according to the ratio of model parameters σconf/σact. It can be seen that when this ratio is small, values of meta-d′ > d′ are routinely obtained. Furthermore, when we interrogate the relationship between the proportion of detected errors , hyper-metacognitive sensitivity is associated with the emergence of error detection in the model  about these parameters may be malleable, leading to systematic over- or underconfidence , and potact, σconf and ρ. Importantly, for all simulations the actual parameters used to generate internal samples and decisions were fixed at σact = 1.5, σconf = 1, ρ = 0.6. The deviation of the curves from the identity line show that subtly different beliefs about the true underlying parameters are sufficient to produce a range of patterns of systematic over- or underconfidence, typical of the probability distortions observed in the experimental literature  originating in the anterior cingulate cortex is observed time-locked to the onset of the erroneous response . In contd′, that quantifies the Type II ROC area in units of first-order d′  of confidence in memory , which aOur model accommodates dissociations between decision-making and metacognition through alterations in the precision and coupling of internal states, such as the decision and confidence variables. However it is also possible that decision-making and metacognition have different inferential goals, and may be differentially sensitive to different types of information. Introducing these normative constraints into models of metacognition is an important goal for future work. For instance, it would be of interest to explore whether differential sensitivity to evidence for or against a choice , and difA counterintuitive feature of second-order computation is that actions influence subsequent confidence ratings, all else being equal. This influence arises because actions contribute information about possible world states, leading a rational observer to incorporate his own actions as additional data when computing confidence cf. . This feA second-order framework makes concrete predictions about the effect of choices on confidence ratings—namely a decrease in overall confidence and an increase in sensitivity. In addition to the results of The role of action in a second-order framework also reveals subtleties in the relationship between confidence and visibility judgments. In consciousness studies, confidence ratings are often considered proxies for perceptual awareness . For insnot expect an influence of action on metacognitive judgments. For instance, if the confidence variable has access to the same information as the decision variable, then there is nothing more to learn from the identity of the action. This is the case in the postdecisional model shown in d is conditionally independent of a given confX). However, even in these cases of sequential evidence accumulation, effects of action may be obtained in practice. For instance, if the influence of predecision evidence decays over time, this would weaken the cross-talk between the decision and confidence variables, and actions would again carry weight when inferring the world state. In other words, if I make a perceptual decision based on some sensory evidence, but then go on to forget this evidence at a later point in time, I am left with only my decision when inferring what the world state might have been. Interestingly empirical data are potentially consistent with this prediction. We note that there are certain cases in which one would More broadly, the influence of one’s own actions on self-evaluation dovetails with the proposal that preferences and beliefs are constructed rather than revealed by judgments and decisions . PostchoThe models considered here suggest an organizing framework for nascent findings on the neural basis of confidence and self-evaluation. In particular, correlates of confidence should be found across multiple putative internal states, including both those directly supporting actions and those supporting confidence ratings. Empirical studies in humans and nonhuman primates show that neural precursors of a decision are modulated by the eventual degree of confidence of the subject , and micA related line of work has identified a central role for the human prefrontal cortex (PFC) in metacognition see . CrucialconfX) with knowledge about the selected action. Importantly this convergence should be flexible and domain-general.Second-order computation requires integration of state information ) should influence the computation of confidence , actions are binary, such as a button press or eye movement. In practice, however, even simple actions are constructed by specifying the kinematics and forces needed to produce a particular motor output. Indeed, individuals have been shown to take action kinematics into consideration when judging the confidence of another individual , and theFinally, as touched upon above, our model is situated at the computational level  and remaThe model outlined in this paper casts self-evaluation as a second-order inference on the efficacy of one’s own behavior. Such a model has the potential to provide common ground for comparing data from different paradigms such as confidence and error detection, and provides a normative framework for understanding a range of dissociations between metacognition and performance. In addition, it predicts a novel role for actions in contributing to estimates of decision confidence. We have outlined the implications of second-order computation for behavioral control and for candidate neurobiological implementations of metacognition. We hope this will provide a conceptual and theoretical framework for studies of metacognitive computation, and motivate a number of empirical hypotheses to be tested in future research.


Relenza represents the first neuraminidase inhibitor (NI), a class of drugs that also includes the drug Tamiflu. Although heralded as breakthrough treatments in influenza, NI efficacy has remained highly controversial. A key unsettled question is why the United States Food and Drug Administration (FDA) has approved more cautious efficacy statements in labelling than European regulators for both drugs.We conducted a qualitative analysis of United States and European Union regulatory appraisals for Relenza to investigate the reasons for divergent regulatory interpretations, pertaining to Relenza’s capacity to alleviate symptoms and reduce frequency of complications of influenza.In Europe, Relenza was evaluated via the so-called national procedure with Sweden as the reference country. We show that FDA reviewers, unlike their European (i.e. Swedish) counterpart, (1) rejected the manufacturer’s insistence on pooling efficacy data, (2) remained wary of subgroup analyses, and (3) insisted on stringent statistical analyses. These differences meant that the FDA was less likely to depart from prevailing regulatory and scientific standards in interpreting trial results. We argue that the differences are explained largely by divergent institutionalised review methodologies, i.e. the European regulator’s reliance on manufacturer-compiled summaries compared to the FDA’s examination of original data and documentation from trials.The FDA’s more probing and meticulous evaluative methodology allowed its reviewers to develop ‘deep’ knowledge concerning the clinical and statistical facets of trials, and more informed opinions regarding suitable methods for analysing trial results. These findings challenge the current emphasis on evaluating regulatory performance mainly in terms of speed of review. We propose that persistent uncertainty and knowledge deficits regarding NIs could have been ameliorated had regulators engaged in the public debates over the drugs’ efficacy and explained their contrasting methodologies and judgments. Regulators use major resources to evaluate drugs, but if regulators’ assessments are not effectively disseminated and used, resources are used inefficiently. The present study aims to investigate and explain conflicting assessments of the anti-influenza drug Relenza (zanamivir) made by United States and European regulators. Marketed since 1999 by the British drug firm Glaxo Wellcome (GW) (subsequently GlaxoSmithKline), Relenza represents the first so-called neuraminidase inhibitor (NI), a class of drugs that also includes the drug Tamiflu (oseltamivir). Although heralded as breakthrough treatments , 3. In pWhy aren’t the United States Centers for Disease Control and Food and Drug Administration speaking with one voice on flu?”  results, and to the proper modelling of the various sources of variation” .the public health need” for a new anti-influenza drug in light of the risk of viral resistance developing against available products (rimantidine and amantadine)  analysed the results of study NAIA3002 [North American study] and the potential determining factors for the decreased treatment response observed” . Once the issue of the inconclusive North American study was judged to be ‘resolved’, the MPA went on to formally conclude that, overall, the evidence showed that Relenza alleviated symptoms of influenza 1–2.5 days quicker than placebo, which justified marketing authorisation . This imthere were some differences between trials in inclusion criteria with respect to age, duration of influenza symptoms at entry and also to the definitions of absence of fever”  – fever” , p. 33).3.there wshould be avoided” , but regulatory prerequisites included, among other things, “no statistically significant heterogeneity” between trials and “inconclusive studies showing positive trends in the primary variable”  bio-statistical guidelines from 2001 . Accordivoided” . riable” , At the same time, it should be emphasised that the MPA’s decision to approve Relenza was not based on this pooled analysis. Rather, pooling had implications for how results were conveyed in the labelling. Nevertheless, together with the evidence presented below, the tolerance towards retrospective pooling and the apparent disregard for inter-trial heterogeneity suggests a tendency on the part of the MPA to depart from prevailing regulatory standards when interpreting the drug’s beneficial effects.The DAVP were unimpressed by the drug’s efficacy in HR-IP patients (e.g. the elderly and patients with certain chronic disorders) and reviewers highlighted numerous problematic issues related to the studies’ design, conduct and results. As with efficacy in the IP group as a whole, statistical heterogeneity between trials was considered a problem for interpreting results in the HR-IP group . A partihigh-risk definitions might in some instances be applied differently by different investigators, so that uniformity in the average severity of underlying disease of these subgroups in different studies could not be assumed with confidence” .idence” .P value of 0.128 as a demonstration that “there is no specific contraindication to pooled…analysis, across protocols” udies…” In sum, DAVP concluded that there was no compelling evidence of efficacy in the HR-IP population due to the small, inconclusive differences , and the substantial statistical and clinical heterogeneity. It is important to note how this view followed from DAVP’s meticulous and detailed engagement with the quantitative and qualitative facets of trials.Like the DAVP, the MPA assessor, Dr Uhnoo, lamented the small number of HR-IP patients recruited, noted the statistically inconclusive results in this group across trials, and expressed concerns that, in the North American study, which had recruited the largest number of high-risk subjects, Relenza patients fared worse than patients on placebo . She didis considered very limited, but do indicate treatment benefit” ” , but disregarded the negative point estimate in the North American study. Thus, the MPA seemed willing to give the drug (and the manufacturer) significant benefit of the doubt with respect to proving efficacy in the HR-IP group, despite recognising the paucity of data and the statistically inconclusive results – some of which actually favoured placebo.Moreover – and again unlike the DAVP team – she seemed inclined to view the evidence as supporting efficacy in HR-IP patients, noting that the evidence “enefit” .  issue” . days)” . In some studies, a more pronounced treatment effect has been seen in patients that belong to the risk group of elderly patients (≤65 years) and patients with certain chronic disorders in heart and lungs, although only a limited number of patients in these risk groups have been evaluated.”“The MPA took a similarly permissive approach when it accepted GW’s retrospective analyses of HR-IP patient subgroups (e.g. chronic respiratory disease), despite this involving multiple analyses, each with a small number of patients, where apparent efficacy differences might easily arise due to chance. Thus, the original Swedish Relenza label from March 1999, written by GW and approved by the MPA, contained the following statement:a more pronounced treatment effect” on the elderly appears to refer to the positive, but statistically inconclusive, point estimate in elderly subjects in each of the pivotal trials  [it is not practicable to perform clinical studies to demonstrate efficacy  in all conceivable patient sub-groups”  demonstrating an overall reduction in duration of influenza symptoms in HR-IP patients  . As the = 0.005) . This cogroups” .groups” .e made” .Unfortunately for GW, and further contradicting of MPA’s initial statement that Relenza would work better on elderly patients, study NAI30012 failed to demonstrate a beneficial treatment effect. Consequently, since 2001, labelling in the European Union has cited the positive effects asthma/chronic obstructive pulmonary disease patients derived from study NAI30008, but simultaneously noted that efficacy has not been established in elderly patients.To this day, FDA maintains that the evidence submitted to them does not prove a beneficial effect of Relenza on the incidence of complications. In reaching this judgment, the DAVP team pointed to the same kind of technical and methodological issues that had troubled reviewers with regard to effects in the HR-IP group, i.e. the small, inconclusive results  alongside important heterogeneity, which contraindicated pooling .P = 0.049; Table In addition, DAVP argued the trials were not individually powered to demonstrate differences, as the percentage of subjects categorised as experiencing complications was relatively low Table . The onedisparate components of the ‘complication’ definition” ([some complications of influenza (such as pneumonia or respiratory failure) would usually be regarded as more serious than others (such as sinusitis or pharyngitis)” ;gitis)” .gnment” .Like DAVP reviewers, the MPA assessor acknowledged that studies were not individually powered to demonstrate conclusive results on complications. Nonetheless, as with effects in HR-IP patients, she seemed inclined to view the totality of results as still providing adequate evidence of effect, albeit without exploring to any great extent the heterogeneity in trials, for example, by looking at instructions for how complications were recorded , 33.The most common complications were bronchitis, sinusitis and otitis media, which in one study were significantly less frequent with zanamivir treatment. Overall in the Phase III trials, the complication rate was lower and there was less use of antibiotics in patients treated with zanamivir compared with placebo” [.“placebo” .While the alleged evidence of a beneficial effect was not considered sufficiently strong or important to merit mentioning in the original Swedish label, the MPA’s lengthier product monograph  included the following description of effects on complications:complications…were significantly less frequent” [Notably, the one study in which “requent”  with RelP = 0.004) and use of antibiotics , compared to placebo .The immediate explanation for these divergent regulatory interpretations of Relenza’s efficacy was that DAVP, but not MPA, rejected the pooling of efficacy data, remained wary of subgroup analyses and insisted on stringent statistical analyses. Notably, in comparison to the DAVP, the MPA repeatedly took a more permissive approach, and was more likely to depart from prevailing regulatory and scientific standards with respect to the evaluation of the design, conduct, analysis and interpretation of the data. For example, the MPA displayed a greater willingness to depart from established scientific interpretive norms by accepting subgroup analyses within the HR-IP group and a ‘significant’ effect on complications in the North American study despite the well-known risks of spurious associations with multiple hypothesis testing . The MPAHowever, why did regulators’ methods and criteria differ in this way? Our analysis suggests links between regulatory outcomes (in the form of divergent product labelling) on the one hand and, on the other, regulatory resources, disciplinary expertise and institutionalised review methodologies and practices. Thus, DAVP benefited from having greater resources, which included a review team made-up of statistical and clinical reviewers who had sufficient time and the multi-disciplinary expertise required to conduct critical, in-depth assessments. This allowed its reviewers to collectively develop ‘deep’ knowledge  concerniBy contrast, the single, clinical assessor for the MPA did not perform independent statistical analyses of raw data or cross check various definitions and their application within and between trials. Instead, she relied largely on summary tables of data and reports compiled by the drug’s sponsor, resulting in a more superficial understanding of the evidence pertaining, for example, to differences in how study definitions of ‘risk’ or ‘complications’ were applied. Arguably, this superficial understanding weakened the MPA assessor’s ability to challenge the manufacturer’s proposed methods for analysing results  or problematic interpretations of the data . It may also have encouraged a more permissive regulatory approach – a permissiveness that is perhaps most evident in the MPA’s acceptance of GW’s claim that patients with cardiovascular disease benefited from Relenza, when results in two out of three studies actually favoured placebo.Alongside these important differences in regulators’ methodologies and conclusions, our analysis also points to similarities. The elephant in the room was whether judgments were to be derived from results in the ITT or IP populations and, related to this, how to deal with the abnormally high number of IP patients in trials (over 70%). Regulators on both sides of the Atlantic concurred with the company that the IP population was most relevant for efficacy evaluation, since this tested the drug’s effects on the disease. By contrast, some researchers  and HealYet, we argue that, even from the regulators’ perspective, to rely on results from the IP population was questionable since the drug was intended for patients with influenza-like symptoms, rather than only those infected by influenza virus. Furthermore, since the label constitutes the factual basis for drug marketing, emphasising effects in the IP population in the label meant regulators authorised the use of these estimates in marketing. How this might be a problem became evident after the MPA’s unit for drug information, tasked with overseeing drug marketing, complained to the Swedish pharmaceutical industry’s self-regulatory body governing drug promotion  about a Our observation that the MPA relied largely on summaries written by the sponsor whilst the DAVP conducted a more thorough assessment, including independent examination of trial documentation and statistical analysis of raw data, is not unique to this case but has been noted in previous social scientific studies of the FDA and national and supranational drug regulatory agencies in the European Union , 42. OneIt is important to acknowledge that European regulators’ less probing evaluative methodology does not always translate into permissive regulation and, conversely, that FDA’s ‘deep’ review does not always ensure a precautionary approach . Neverthby the completeness and scientific soundness of its work” .We have shown that the FDA’s probing and meticulous review practices in the case of Relenza resulted in ‘deep’ knowledge of the clinical and statistical aspects of the existing evidence base for the drug and, arguably, a more accurate picture of the nature and magnitude of its health benefits. Such knowledge is a valuable asset, but this asset can only be realised if the information is widely communicated to and understood by the public health community . Yet, a To illustrate this, one can point to the fact that, after researchers ‘discovered’ regulatory assessments differed between countries yet were unable to explain the reason for these divergences , neitherNew England Journal of Medicine in 1997 of two phase II studies for Relenza (NAIA2005 and NAIB2005), which states that the definition of illness included “the presence of fever” in both trials [Doshi  has receh trials . Howeverh trials , dismissNew England Journal of Medicine that establishing the FDA as a public health agency required the organisation to communicate “frequently and clearly about risks and benefits”, “provide the data on which it bases its regulatory decisions… and explain its decision-making process to the public” and foster “a culture that encourages scientific exchange” [British Medical Journal described the maze of contradictory statements about NI’s efficacy made by researchers, industry, regulators and other public bodies [This discussion, then, raises important policy questions, not only about the most effective practices and methods of regulatory science, but also about the public health role of regulators and what that role should entail. In 2009, the new Commissioner of the FDA wrote in the xchange” . Just a c bodies .We would argue that persistent uncertainties and knowledge deficits regarding NIs could have been ameliorated had regulators explained their contrasting methodologies and judgments . Had reThe FDA’s more comprehensive evaluative methodology resulted in a ‘deeper’ understanding of the Relenza clinical evidence. Our findings challenge the current emphasis on evaluating regulatory performance mainly in terms of speed of review. Furthermore, had regulators explained their contrasting assessments, this could have improved the broader medical community’s critical appraisal of the evidence base for anti-influenza drugs. Regulators use major resources to evaluate new medicines, but if regulators’ assessments are not effectively disseminated and used, resources are wasted.