As with the eventual demise of a similar attempt to maintain the status quo of professional beliefs within forensic clinical psychology and psychiatry during the last century, those following certain EFPA guidelines might now find themselves required to justify their professional beliefs in legal rather than academic environments.The European Federation of Psychologists’ Associations (EFPA) has issued sets of test standards and guidelines for psychometric test reviews without any attempt to address the critical content of many substantive publications by measurement experts such as Joel Michell. For example, he has argued that the psychometric test-theory which underpins classical and modern IRT psychometrics is “pathological”, with the entire profession of psychometricians suffering from a methodological thought disorder. With the advent of new kinds of assessment now being created by the “Next Generation” of psychologists which no longer conform to the item-based, statistical test theory generated last century, a new framework is set out for constructing evidence-bases suitable for these “Next Generation” of assessments, which avoids the illusory beliefs of equal-interval or quantitatively structured psychological attributes. Finally, with no systematic or  For example, take IQ; to present a mean IQ score to a court in 2-decimal place precision requires empirical evidence that the attribute in question (intelligence), for which the IQ score is the numerical estimate, does indeed vary according to the numerical real-valued, continuous numbers used to represent the mean “magnitude” of intelligence for an individual. The fact that there is no evidence that any psychological attribute varies as a quantity, or with equal intervals between magnitudes, and that no-one can explain the observable “intelligence” difference in an individual possessing say a score of 69 or 71, or indeed a score of 110 and 114, is an admission to the court that the number presented is based upon a belief (an assumption) about the precision of IQ/Intelligence “measurement”, not fact.This is not to deny that we can make pragmatically useful judgments about the meaningfulness of differing magnitudes of psychological attributes, but these judgments/interpretations are confined to statements about “orders” of magnitude whose precision is relatively coarse. Furthermore, given the “common-or-garden”  or “everIn addition, while many point to IRT models as psychology’s answer to physical measurement, Wood  demonstr“It follows that for attributes investigated in science, there are three structural possibilities: (1) classificatory attributes (with heterogeneous differences between categories); (2) heterogeneous orders (with heterogeneous differences between degrees); and (3) quantitative attributes (with thoroughly homogeneous differences between magnitudes). Measurement is possible only with attributes of kind (3) and, as far as we know, psychological attributes are exclusively of kinds (1) or (2). However, contrary to the known facts, psychometricians, for their own special reasons insist that test scores provide measurements.”Many who support the EFPA guidelines (and the psychometric methodologies which they specify as “required”) will suggest that ordered-class data are simply equal-interval magnitudes “measured” poorly; approximations if you like of underlying continuous, real-valued, attribute magnitudes. Once again, Michell ,12 has s“We need more, not less, by way of modern causal mechanisms”. Pennington (2014) asked and answered the question of what is required to provide a scientific explanation in the following way: “What does it mean to explain something? Basically, it means that we identify the cause of that thing in terms of relevant mechanisms”  Tryon  in a resguild-beliefs about measurement rather than evidence-based recommendations.A generative causal mechanism is what enables the technical specification of the rules for the measurement of an attribute. Without any agreed-upon definitions of the precise meaning of any psychological attribute (which may be impossible), and with no obvious understanding of the constituent properties of quantitative measurement by those setting standards for “tests” and reviews, we are left with sections of clerical guidelines which represent the shared The fundamental issue at stake here is about how we seek to acquire evidence that a proposed assessment of a psychological attribute is reliable and valid. That does not require us to commit to a variety of statistical or test-theory assumptions as recommended by the EFPA guidelines , but thaWe may even ask whether an attribute’s assessment in one culture or ethnic group is possible in another. However, this is not achieved using the conventional Structural Equation Modeling invariance analysis methods ,20. ThesAnother problem that arises when considering the EFPA guidelines on reliability and validity is the reliance of many of the psychometric indices and statistical test-theory methodologies on the concept of a “true score”. As Borsboom and Mellenbergh  explain,Then there is the matter of construct validity. Surely what appears in the EFPA guidelines is no longer tenable given the publication of several major critiques of the entire concept ,25,26,27That is, for socioeconomic, political, and academic-status reasons, psychologists have chosen to adopt their own individual/characteristic definitions of measurement in order to create the impression among the wider academic and scientific community that psychology can make measurement in the manner and rigor in which it is made within the physical sciences ,31,32. AWhen psychologists and publishers of tests found there was a profitable market for assessment products, the motivation to create financial rewards overcame any concerns about measurement. Instead, psychometric test theory and its methods were heavily promoted as the benchmark of any credible assessment evidence-base, which empowered professional psychological societies to impose standards for the training of test users. Such training proved to be highly profitable for test publishers while enabling them to promote themselves as “standards-bearers” for technical sophistication and rigor. In fact, psychometric complexity/expertise became a point-of-difference value proposition for many competitive sales approaches within the Human Resource field. The cold reality is that there is little obvious outcome-differentiation between the deployment of a DiSC, MBTI, or Brand X assessment which fails all psychometric evidence-base hurdles when compared with a CEB or Hogan psychometric masterpiece. This is to be expected given that all we can confidently claim is that psychometric test scores represent rank-orders of magnitudes, not quantities. The accuracy/precision accorded to test scores is illusory, which explains the lack of any tangible real-world organizational consequences when using typology assessments compared to “equal-interval” score-based assessments. The sad reality is that test-theory psychometric methodology has been reduced to the status of a marketing ploy; brought out to impress and amaze in competitive sales presentations, but with no-one evaluating the validity of the claims because few understand or agree upon what is actually being presented as “evidence”, given the level of abstraction of the various indices. Which in part explains Briner’s and Rousseau’s  conclusi“The phrase to turn a blind eye is attributed to an incident in the life of Admiral Horatio Nelson. Nelson was blinded in one eye early in his Royal Navy career. During the Battle of Copenhagen in 1801 the cautious Admiral Sir Hyde Parker, in overall command of the British forces, sent a signal to Nelson’s forces ordering them to discontinue the action. Naval orders were transmitted via a system of signal flags at that time. When this order was brought to the more aggressive Nelson’s attention, he lifted his telescope up to his blind eye, saying, “I really do not see the signal,” and most of his forces continued to press home the attack.”From Wikipedia :“The phUsually paraphrased as “I see no ships”, this quotation sums up the studious ignoring of the basic facts, logic, and axioms which define the measurement of a quantity. Those who formed the EFPA and the UK’s BPS and US’s APA committees chose to ignore clear facts concerning the constituent properties of quantitative measurement, and the consequences such facts might have on any guidelines they chose to issue. Instead they issued their guidelines and recommendations to other psychologists and organizations without any caveats.I’m following best practice guidelines” no longer suffices as a judge may realize that equating the measurement of a psychological attribute with that of a physics SI base or derived-unit measure [This denial of reality serves no purpose other than to maintain a flawed status-quo which is increasingly unlikely to survive appropriate adversarial challenge in a court of law. There is now too much published counter-information on these issues, with no substantive published rejoinders. That intellectual silence speaks volumes, especially when a psychologist presenting test scores to a court is now required to justify the quantitative assumption upon which they rely for the validity of any test scores and quantitative statistical procedures offered to the court. The response “ measure  is not jIt might be argued that any set of systematic methodological procedures, such as those based upon an assumption that an attribute varies as a quantity, that “true-scores” can be invoked, that IRT can produce linear scales for measuring the magnitude of psychological attributes, is better than any possible alternative?The problem is threefold:Definitional: We cannot define any psychological attribute with sufficient precision such that psychologists en masse agree on that single, precise technical definition, which in turn permits generation of the rules for measuring magnitudes of the attribute. Hence scales of varying numbers of “items” are generated which accord with a researcher’s personal definition of a “construct”, which entails the production of hundreds of scales all apparently measuring the same-named attributes but which only rarely show score-magnitude equivalence. Psychologists wander between single scales of items from the BFI-10 2-item Extraversion scale, the EPQR 21-item scale, through to the NEO-PIR 48-item scale (with six 8-item facets). All scales are referred to as “Extraversion”, but which one is the definitive measure of the “construct”? What about the MBTI assessment of Extraversion, or even the DiSC assessment, or CEB-OPQ’s “Extraversion” scale? The incredibly simple fact is we do not have a precise definition of the construct, one that determines a standard for the measurement of extraversion. Construct Validity? We might as well ask for Basil Fawlty’s sliced hippopotamus in suitcase sauce [❶ se sauce ! You migCausal: We have no causal theory for any attribute which causally explains, mechanistically, how variations in that attribute occur. As noted above in that quote from the article by Tryon [“On the other hand, in much, perhaps even most, research in psychology, there are no practically relevant units anyway. For example, in social psychology, what is an attitude unit, a prejudice unit, or a self-affirmation unit? In clinical psychology, what is a depression, anxiety, or psychopathy unit? In education, what is a knowledge unit? Similar questions can be asked with respect to measurement in many areas.”❷ by Tryon . None ofby Tryon  (p. 849)Representational: Psychologists persist in equating the results of arithmetic operations on the real or integer number system as though this numerical relational system is actually equivalent with the empirical attribute relational system. What if it isn’t?❸ Back in 1998, I set up a simple simulation in response to William Fisher’s and Ben Wright’s enthusiastic statements at the time, claiming the Rasch model could construct quantitative measurement from “poor” observations of an underlying quantity .Here I comprehensively extended an example briefly presented by Fisher —where I Each measurement is in the form of a dichotomy—a 1 is assigned to a bad measure unit if my cm “measure” extends beyond this unit. Where my cm measure is smaller than the remaining bad-ruler units, I assign a 0 to these units. The “responses” were “jittered”, adding some error (for the Rasch is a probabilistic model); the model dataset consisted of 40 “persons” and 16-item response vectors.The rather simple-minded test here is whether the Rasch model will recover the equal-interval cm scale from the “ordinal” measures made by me. The example details and results begin at slide/page #26 of the 1998 presentation.The results indicated that the Rasch difficulty parameters were “mirroring” my bad ruler integer units. If we did not know that my 16 numerically equal-interval integer “units” were in reality unequally spaced, we would refer to them as “equal-interval”, and plot them accordingly, noting that the Rasch difficulty/location parameters are now also equally spaced, mirroring the integer “units”.e (cm). Then I mapped equal-interval integers onto my units (ignoring the underlying non-linearity) to make my measurement observations as before. The results were the same; the Rasch modeling produced an equal-interval scaling of the additive-metric integer numerals which were indexing non-linear true cm lengths. The real unit of measurement (cm) was never “discovered” or revealed by the model.I conducted a second simulation, to confirm my suspicion that the Rasch model is NOT able to reconstruct the underlying true linear equal-interval measurement from poorly observed data. Here, I mapped my “bad-ruler” units onto log“Now, if a person’s correct response to an item depended solely on ability, with no random ‘error’ component involved, one would only learn the ordinal fact that that person’s ability at least matches the difficulty level of the item. Item response modellers derive all quantitative information  from the distributional properties of the random ‘error’ component. If the model is true, the shape of the ‘error’ distribution reflects the quantitative structure of the attribute, but if the attribute is not quantitative, the supposed shape of ‘error’ only projects the image of a fictitious quantitivity. Here, as elsewhere, psychometricians derive what they want most (measures) from what they know least (the shape of ‘error’) by presuming to already know it.”Following this presentation in November 1998, Ben Wright from the MESA group at Chicago re-analyzed my data—and concluded that there was insufficient stochasticity (random error) in my observations. In short, my data may have been artificially “too clean” for the Rasch model to work as designed  That response always struck me as somewhat bizarre, as Michell [“Item response modellers derive all quantitative information  from the distributional properties of the random ‘error’ component.”It was Ben Wright’s “you need more error in your data” response, coupled with Wood’s  coin-tosThe more random error in a dataset, the more likely a model will fit that dataset.Constructing measurement with particular properties such as those required by a quantity requires detailed empirical experimentation comprising initial phenomenon detection, theory construction, and more experimentation What are the “Next Generation” of assessments? Well, these are, as you might expect, very different in design, look, feel, scoring, and administration, from the generation of assessments that began their life back in the early 1900s. We are all familiar with the current self-report questionnaires, ability-tests and their “puzzle-format” items, assessment center tasks, simple “psychometric” situational judgement tests and their variants. Indeed, many of these assessments and design concepts were originally created during the last century, and their characteristics and features perfected accordingly in line with associated developments in test theory psychometrics  and computing/display/internet technology.However, an OPQ is an OPQ is an OPQ, however much you tinker with the items, display format, and scoring. Likewise, a WAVE, NEO, HPI, MBTI, 16PF, or 15FQ. Same as the WAIS, Stanford-Binet, Raven, MAB, Wonderlic, or other GMA assessments, and the various forms of clinical self-report/diagnostic questionnaires. In their own ways, these represent the “best of class” assessments in their chosen domain. They have been developed and honed over the years into their final forms, short or otherwise. However, their evolution was and is ultimately limited by a design that was conventional many decades ago. It was fashionable then but limited by the requirements and design-rules of test-theory psychometrics and the available technologies for delivering/deploying assessments; from paper-and-pencil supervised administration, hand-scoring, and expert-psychologist report writing through to the present-day mode of delivery of an assessment on a mobile/tablet with completely autonomous scoring and report generation. When you look back at the evolution of assessments, you cannot but be impressed at the sheer ingenuity, drive, and entrepreneurial skills and abilities of those who created the assessments and the current market for them. In essence, these academic and commercial innovators have collectively taken a world market amounting to a few thousands of dollars a year to many billions.“No technology of which we are aware—computers, telecommunications, televisions, and so on—has shown the kind of ideational stagnation that has characterized the testing industry. Why? Because in other industries, those who do not innovate do not survive. In the testing industry, the opposite appears to be the case. Like Rocky I, Rocky II, Rocky III, and so on, the testing industry provides minor cosmetic successive variants of the same product where only the numbers after the names substantially change. These variants survive because psychologists buy the tests and then loyally defend them .The existing tests and use of tests have value, but they are not the best they can be. When a commentator says that it will never be possible to improve much on the current admissions policies of Yale and its direct competitors  , were asuch as facial-feature analysis).However, now, things are very, very different. A clutch of new organizations such as Pymetrics, Knack.it, Revelian, Arctic Shores, Journey, and even the US Army, are now creating games played on mobiles and tablets to assess candidates for job-roles. EI Design, Merck, JobVille, Bluewolf, Cognadev, and many other smaller organizations/consultancies are providing gamified e-learning assessments. More established corporates such as L’Oreal , KPMG , Deloittes Leadership Academy , have built their own custom game-based assessments for internal assessment, leadership and talent management. Then there are the test publishers such as Cut-e, and start-ups such as Cliquidity  and Cohired.com , and more established organizations such as StaffCV and Findly, who provide biodata screens and psychometric assessments to individuals free of charge, so that the individuals can build their own portfolio of biodata and psychological assessments and be matched autonomously by these systems against paying-client organization job-vacancies. Finally, we have organizations such as Cambridge Analytica and the Psychometrics Centre (UK) selling psychographics; the provision of personality psychological attribute magnitudes acquired through behavioral-linguistic analysis of social networking internet activity of an individual. Hogan Assessments, through their innovation division, Hogan-X, are now advertising free online personality assessment using a variety of novel assessments tasks and stimuli ; just information acquired from Assessments sometimes are invariably comprised of thousands of very specific observations, clustered by expert-system rule or empirically into broader categories.Scoring is via expert-system rules and theory-relevant algorithms and/or varieties of machine-learning optimized prediction models.There are no psychometric “scales” as such, just attribute “type”, ordered-class, or quantitative  magnitude assessment which might be constituted from many diverse but theoretically/experimentally related behavioral information sources.Scoring information for any commercial assessment constitutes the Intellectual Property (IP) of an assessment, and is not made public.Reliability of any assessment outcome, type, ordered-class, or quantitative magnitude is assessed using appropriate retest methodologies only.Structural “validation” psychometrics  cannot be effectively used due to the mix of assessment-variable properties and expert-system rule-based scoring procedures which produce the final attribute magnitudes, orders, and classes. Basically, concepts from psychometric test theory simply do not apply to the kinds of attribute constituents, classifications, or “score” magnitudes.The characteristic features of these new kinds of assessments are:distinct goals for its design and deployment: the Scientific and the Pragmatic. These two goals are completely discrete from one another.Section 10 (Reliability), and especially Section 11  of the EFPA guidelines appear somewhat irrelevant when looking at the list of the defining features of Next Generation assessments. This is not surprising because the guidelines were created for the previous generation’s assessments, whose design and construction were based upon a particular kind of “true-score” test theory and questionnaire-item-based measurement model (IRT). Given the untenable quantity assumption that pervades these guidelines, it is surely suggestive that now is the time to return to more straightforward and robust methods of evaluating the reliability and validity of a psychological attribute assessment; methods that are concordant with two the fundamental constituent of the phenomenon. For example, if I am to propose a measure of a psychological attribute such as “grit’, the initial scientific task is to empirically identify the phenomenon of interest i.e., grit. This is phenomenon detection [grit. From this initial set of observations and meaning-assignment, causal explanatory theory construction can now be undertaken along with the construction of a prototype measure of grit, according to the rules now set down by an investigator for the measurement of grit. The evaluation of the measure requires a technical definition of what constitutes grit and what is causal for variations in the proposed measure of the construct. Then, it is a matter of empirical experimentation to demonstrate that what an investigator has proposed as causal for magnitudes of grit results in the numerical or otherwise defined magnitudes indicated by the assessment.From this perspective, an assessment is designed as part of a scientific investigation into a phenomenon, where the primary goal is to understand the cause/s of variation in some attribute; where that attribute is etection , and is grit proposes that magnitudes of grit must vary with equal intervals between its magnitudes, then a strong theory-claim is that grit varies as a continuous variable for which no normative upper or lower range-limiting bounds exist. Its magnitudes are additively structured but those magnitudes are assumed to be continuously varying over some arbitrary range (no agreed-upon minimum or maximum). Therefore, for a proposed measure of grit, these magnitudes are represented using the numerical real-number system but with the proviso there is no standard unit of measurement; hence if ratios of grit are constructed, these ratios are relative to the particular scaling of grit employed by an investigator.❶ If the causal theory for variation in grit is hypothesized as varying additively but not continuously, then a measure of grit would be represented using an equal-interval integer magnitude scale. Ratios of grit are no longer possible to compute except within the framework of integer arithmetic.❷ If grit, by dropping the additive-metric assumption, then we represent orders of grit via ascending-order symbols, which may be numbers or other ordered text-sequences such as letters of the alphabet.❸ If we propose that the best we can achieve is the identification/assessment of “orders” of grit varies additively with continuous or integer equal-intervals is going to be extremely difficult. To do so requires a technical/explicit definition of the construct; as within physics, a precise specification of what it is and how it should be measured, and what is causal or could conceivably cause such precise additive-unit variations in its magnitudes. Then this attribute is somehow manipulated experimentally, confirming that the variations so produced can be accurately represented using the real number or integer numerical relational system. The same “manipulation” issue applies if testing a proposed order-relation representation.Clearly, the experimental task of demonstrating that the psychological attribute Indeed, this “manipulation requirement” is the very issue that Trendler ,51,52 haAlthough some, such as Carpenter, Just, and Shell  have devinformation-bits-per-second processing. In addition, visual stimulus choice reaction time is a very specific form of sensory information processing. We might even be tempted to utilize Lehrl and Fischer’s [However, some investigators do set out to create an assessment which is based upon a technically-specified causal model for variations in a psychological attribute. For example, assume we have created an assessment of information processing speed (scored 0–120 s) based upon an assessment involving the manipulation of text-information by an individual where time taken to correctly manipulate the information is “the measure”. From a scientific perspective, to evaluate the claim that the assessment is measuring “information processing speed’, we might employ an experiment task where we directly control stimulus information in terms of formal “bits” of information required to be processed to complete the task, and note the time taken to correctly complete it expressed as bits-per-second processing speed. A simple criterion task might be a visual-stimulus choice reaction time. However, we are already assuming that what is meant by “information processing” is the same for both tasks. What is missing is that information processing speed as a psychological attribute, measured in seconds, may not be related in any straightforward way to sensory information processing speed measured in ischer’s ,56 Basicischer’s , which iBasically, there are no specific procedures for evaluating the reliability or validity of an assessment that is constructed within a scientific framework. Rather, the very principles and aims of conducting scientific research in an area impose a set of constraints on evaluation that are self-obvious from the very nature and purpose of scientific enquiry.With a typical pragmatic assessment such as the actuarial estimation of the recidivist risk of an offender, the constituent “items” in a scale are “instances” or “exemplars” of behaviors and events considered indicative of some important outcome. The “magnitude” scale is formed from the sum of binary or pragmatically weighted responses to these items rather than assuming that an underlying “latent” variable is causal for variations in the responses. Likewise, employee safety risk, where various indicators of risk are combined algorithmically; the goal being to form an accurate probability estimate of an employee’s safety risk optimized against observed workplace incident data . From thmodus-operandi builds attributes from descriptive aggregate “structural” statistical analyses of item covariance data , via phenomenon detection, or indeed any synthesis of ideas/experiment observations/broad theoretical viewpoints which enable the creation of what is usually considered a “latent variable” attribute magnitude scale. In this assessment construction process, the rules for the constituent properties of measurement of the attribute are always left unstated; the generic assumption is that the magnitudes of any psychological attribute assessment must be “equal interval”—as was stated succinctly by Frank Schmidt in a comment on ResearchGate, dated 26 July 2017 [“The argument about ordering vs. quantification has been made about all social science research. It does not hold water. There is plenty of evidence that many psychological scales are essentially interval scales or close to it”.However, within the psychometric assessment domain, the current uly 2017 .“The arWhen requested to provide the empirical evidence substantiating such a claim, no response was forthcoming. A revised version of the article is now published which drops the phrase “Intellectual Honesty” .In essence, this is the default view for the vast majority of psychological test constructors. It reflects the mindset of the previous generations of 20th century psychometricians, where classical and modern test theory psychometrics was predicated on the assumption that all psychological attribute magnitudes vary as continuous or integer equal-interval entities; the very basis in fact of all metric psychometric test-theory. Put simply, the view is one stop short of claiming that these attributes vary as SI-base and derived unit quantities. In the 21st century, we now know better; and importantly we know this assumption will not survive scrutiny in legal settings where evidence, not assertion, is required to support specific measurement claims.Thus, we now find ourselves in what is essentially undiscovered country; pioneers if you like. We know that any assessment must be evaluated for its reliability but it must also possess validation evidence which justifies its use (and utility). We also know that the next-generation assessments which do not conform to clerical checklist guidelines are here with us right now. In addition, we also now know the old EFPA guidelines concerning reliability and validity assessment are no longer fit for purpose in assessing the modern clutch of instruments which are rapidly becoming the user-norm within the world of personal and organizational development.“Baseball and horse racing provide two popular analogues. Baseball batters with the greatest number of home runs, runs batted in (RBIs) and highest batting average in a year are triple crown winners. In U.S. horse racing, the triple crown goes to the 3-year-old thoroughbred that wins the Kentucky Derby, the Preakness Stakes and the Belmont Stakes in the same year.”How do we then pragmatically evaluate the next generation of assessments? Let me set out the framework—because that is all it is. A framework of requirements. There are no psychometric or formulaic calculations/steps to follow. Assessment evaluation is designed around how an assessment is constructed and delivered, what it reports, and the claims made for what it reports. I’m following Deloittes’ Persistence Project Triple Crown analogy  (p. 28) Clarity of visionDisciplined resource allocationExcellence in execution” p. 39“In the end, triple crown companies share the following attributes:And how the authors summarize their analyses of data revealing these “Crowns” for organizations seeking to succeed:No formula, no specific rules, no specific steps to follow—just three overriding principles shared by organizations possessing enduring rather than short-term success.Therefore, with respect to assessment standards/evaluation guidelines, I’m proposing three key requirements that need to be addressed by any assessment developer, test publisher, or evaluator/assessor/user:R1: The Measurement Assumption: As there is no evidence to suggest any psychological attribute varies as a quantity, or even equal-interval of any description, we should not make that assumption. The most reasonable assumption is that we can assess partial orders or classes with some degree of “fuzziness” between boundaries. Without any clear methodology for determining precisely how an attribute varies and what is causal for those variations, we are relying upon “common-sense” judgement of magnitude, allied to some evidence in R3 below. By all means we may use numerical sum scores, algorithmically-defined magnitude scales, whatever is suitable and convenient, but always with the proviso that these are for sheer convenience rather than an indication of precision. It is important to avoid such assumption-laden work by substituting orders, class-based, and actuarial analyses wherever it makes good sense to do so.R2: Evaluate Reliability: There is only one measure—repeatability/reproducibility. Otherwise known as retest. Alpha and Omega indices are no longer remotely relevant. Retest reliability requires an answer to the question: “will an individual obtain the same score if retested today, tomorrow, next week, next month, next year, over whatever duration is considered relevant?” Ordinarily, retest reliability and a discrepancy-score/class analysis would answer this question nicely, except that retest reliability estimation now includes three potential threats to the accurate estimation of a reliability parameter:Non-systematic random error associated with the internal integrity of the test itself.Systematic and meaningful attribute variation of the attribute over periods of time within and extending across the retest duration.Memory of previous responses artificially causing consistency in 2nd occasion response patterns.The end result of a retest analysis would nevertheless be an indication of reliability, though the causes of any substantive unreliability are not able to be disentangled from each other except by further careful empirical investigation.When working with numerical approaches to reliability estimation, magnitude trumps monotonicity. That is, Pearson or Pearson-based correlation assessments are only used as indicators of monotonic relationship. They are not estimates of reliability because magnitude information is lost as part of the calculation process. Reliability is about score-magnitude equivalence, not indexing an order-relation (monotonicity).will an individual obtain the same type-assignment if retested today, tomorrow, next week, next month, next year, over whatever duration is considered relevant?” The evidence base for reliability is constructed via suitable initial investigations and a body of empirical replications of reliable results.When working with orders or types, existing or new methods directly suited to demonstrating repeatability of order or type are used. The logic of such methods is always clear-cut to explain and justify because no attempt is ever made to extrapolate to hypothetical sampling distributions, item universes, populations of individuals, or true scores. If an assessment provides its results in terms of what type a person is, then that is sole focus of the reliability assessment, answering the question: “R3: Validation: No more or less than providing empirical evidence in support of any claim that is made regarding the results reported by an assessment. If an assessment assigns a type-classification for an individual, then evidence is required to justify the accuracy of that assignment, answering the question: “what evidence is there that the type assigned by the test for an individual is an accurate assessment of that individual?” External data are required which corroborate the type-description based upon the responses or behaviors within an assessment that have been used to assign a type. If the assessment provides an ordered-class “score” or magnitude, then “external-to-the-assessment” evidence is required to substantiate the claim that those ordered magnitudes are indicative of the meaning associated with the assigned attribute magnitudes. This translates to detailing evidence that assessment magnitudes are aligned with externally observed phenomena which are indicative and/or aligned meaningfully with the assessment magnitudes. For example, if an assessment reports ordered-magnitudes of what it refers to as grit, then what is required is evidence indicating that individuals possessing each level of magnitude can be differentiated from others by the qualities or frequencies of behaviors they exhibit which are deemed indicative of a particular magnitude of grit. If an assessment constitutes a game wherein game-playing/interactive behaviors are deemed indicative of particular characteristics of an individual, then external-to-the-game empirical evidence is required to demonstrate the validity of claims phrased as “this measures/estimates/indicates X”. Clearly, if an assessment provider claims that the attribute class, order, or numerical score is predictive of some outcome, then relevant empirical evidence quantifying that predictive accuracy has to be forthcoming in order to justify such a claim.R1, we cannot assess “invariance” or other qualities that might be applicable to quantitative variable assessment. Thus, we have to build a “case” for our assessment. This case would demonstrate the validity of a claim that an assessment which was, for example, validated in the US, possesses similar response behaviors and the same or similar expected phenomenal outcomes in say Spain. Admittedly, this is a complex area because while an assessment may indicate the same magnitude of an attribute in two cultures, how that magnitude is reflected in say frequencies of relevant behaviors may be different, conditioned by particular cultural influences.To justify a claim that an assessment constructed for use in a particular culture is applicable to another, then it is a matter of evaluating the evidence that supported the use of the assessment in the originating culture alongside the comparable evidence which is proposed as supporting its use in another culture. Given A pragmatic evaluation requirement is concerned with constructing an evidence-base for an assessment’s claims which would stand scrutiny under adversarial examination in a court of law. Because no evidence exists that any psychological attribute varies as a quantity, use cannot be made of methods of evidence-base construction which rely upon this untenable and untested assumption. Which is why a “case-building” approach to validation is essential. If on the basis of our assessment, we claim a person will be the type of person who will show “good judgement”, then clear empirical evidence is required to show that indeed such people can be differentiated by the defining phenomenon of interest (good judgements) from those assigned a lower magnitude or class of judgement. In short, as David Freedman  argued, “Validity is not complex, faceted, or dependent on nomological networks and social consequences of testing. It is a very basic concept and was correctly formulated, for instance, by Kelley  (p. 14) A test is valid for measuring an attribute if and only if (a) the attribute exists and (b) variations in the attribute causally produce variations in the outcomes of the measurement procedure.”For many reading the above, the terms: construct, content, face, predictive, concurrent, and ecological validity are missing from the validation process. At best, the validation seems to be all about concurrent/predictive validity. This is because these terms are now entirely obsolete and irrelevant, along with “nomological nets”. Borsboom, Mellenbergh, and van Heerden  “Why did you proceed with guidelines post-1997 which included methods of assessment evaluation predicated upon an untenable and untested assumption?”For those in the International Test Commission, those who constructed the relevant EFPA guidelines in question here, and indeed any professional society which issued guidelines for evaluating assessments based upon that “equal-interval” assumption which pervades psychometrics, the key question to be answered is:in part upon such illusory precision. The average IQ presented to the court was 70.66, with various “standard error and confidence intervals also being presented for each individual cognitive ability assessment. All these statistical and test-theory “adjustments” are themselves predicated upon IQ varying as an equal-interval attribute [The end result is courts being presented with latent variable or average IQ scores expressed to two-decimal place precision, as was the case recently in the US Supreme court where the decision relating to the death penalty for an offender rested ttribute . Given t“Computational psychometrics (CP) is defined as a blend of data-driven computer science methods , stochastic theory, and theory-driven psychometrics in order to measure latent abilities in real-time”.However, it might be argued that the new methodology termed “computational psychometrics” is the evolution of test-theory psychometrics for the 21st century and newer Next-Generation assessments ,67. Lookcomputational psychometrics, we see latent variables being created whose magnitudes vary as real-valued continuous quantities, not necessarily because there is empirical evidence that they do so, but because the authors simply decide that they will do so. It may be others will forgo the illusory precision of creating such quantitative latent variables, as that assumption-laden move is entirely unnecessary.The utilization of computer science/machine learning methods to form probabilities of “beliefs” (as in the Bayesian Belief Networks used by the authors) mimics an actuarial approach to forming probabilities of outcomes as is seen within the entirely non-metric Violence Risk Assessment Guide  and othe“In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller -and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated… In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.”Now it behooves me, of course, to tell you what they’re missing… It is a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of  , but by the courts in which they routinely presented their clinical judgements as “evidence”. Jay Ziskin  releasedWith the advent of new assessment and interview technologies, new highly automated recruitment and selection strategies, and the use of Big Data and Machine-Learning algorithms to generate employee assessment information, the EFPA guidelines on reliability and validity are no longer of any relevance. Now is the time to adopt more realistic and more intellectually honest procedures to evaluate the reliability and validity of any assessment; perhaps embracing the three broad requirements/principles outlined in this article. In reality, little of any importance is lost by discarding test-theory psychometrics, as the precision claimed by so many psychometricians using them is now known to be illusory. In real-world practice, few experienced assessment practitioners pretend their psychological assessments convey more than “ordered-class” information about an individual. When practitioners speak of reliability, it is that concerned with the direct observation of occasion-to-occasion variability, not abstract internal consistency estimates or factor-loading-based estimates. With game-based assessments consisting of hundreds and sometimes thousands of discrete estimation points, where outcome “indexes’, indications, orders, or classes are constructed using machine-learning and/or algorithmic expert-rules, evaluators need to see evidence that the outcome “indicators” from such complex and dynamic assessments do indeed relate to and/or predict that which they are claimed to relate/predict. Such evidence bases need to be constructed using methods appropriate to the specific claims made, avoiding obsolete abstract test theory and analysis methodologies which assume psychological attributes are being assessed as quantities. The EFPA guidelines served their purpose for the previous generation of self-report and ability questionnaire assessments. However, “next generation” assessments demand different kinds of evidence-bases suited to the assessment of attributes in what is essentially a non-quantitative science of psychology.


Chagas disease (CD) profoundly affects the social and emotional dimensions of patients’ lives, and disproportionately impacts poor, marginalized populations in Latin America. Biomedical treatment for CD fails to reach up to 99% of the people affected, and in any case seldom addresses the emotional health or socioeconomic conditions of patients. This study examines patient strategies for coping with CD in the department of Santa Cruz, Bolivia.In this ethnographic study, semistructured interviews took place from March-June 2013 with 63 patients who had previously tested positive for CD. During the fieldwork period, participant observation was conducted and patient family members, providers, community members, and public health officials were consulted.tranquilidad, which partially empowered them to return to a normal existence.Patients often experienced emotional distress when diagnosed with CD, yet were generally unable to find biomedical treatment. Respondents stressed the need to avoid powerful emotions which would worsen the impact of CD symptoms. To manage CD, patients embraced a calm state of mind, described in Spanish as In the perceived absence of biomedical treatment options, patients seek their own means of coping with CD diagnosis. Rather than fatalism or resignation, patients’ emphasis on maintaining calm and not worrying about CD represents a pragmatic strategy for restoring a sense of normalcy and control to their lives. Programs focused on treatment of CD should remain mindful of the emotional and social impact of the disease on patients. An estimated 99% of people with Chagas disease do not receive treatment. I interviewed 63 patients with a diagnosis of Chagas disease at a rural clinic in an endemic area of Bolivia. Only 4 had obtained biomedical treatment. Although the departmental government provided free medication for Chagas disease, this was not typically available for adults in patients’ communities, and additional costs such as consultations and cardiologic examinations were prohibitively expensive for this marginalized population. Travel distances and contradictory messages from providers also made it difficult for patients to access treatment. Furthermore, patients believed strong emotions would worsen the symptoms of their Chagas disease, yet often felt anguished or worried when diagnosed. To deal with the emotional impact of the disease and restore normalcy to their lives, patients adopted a strategy of remaining calm and minimizing worries about Chagas disease. This is not fatalism or resignation, but a pragmatic strategy patients utilize to feel more in control of the disease. Treatment programs need to consider the emotional and social implications of Chagas disease in addition to addressing structural barriers which impede treatment. The consequences of disease transcend the realm of the biological, impacting not only the physical functioning of the human body, but the social life and emotional wellbeing of the affected person. While providers focus on treating the clinical aspects of disease, patients deal with multiple levels of impacts, particularly emotional and socioeconomic ramifications. This study investigates how people with Chagas disease (CD) in an endemic area of Bolivia cope with their diagnosis in order to continue functioning in their daily lives.Trypanosoma cruzi, the protozoan which causes Chagas disease (CD), no? Being tranquila. Not thinking that you are sick. Getting your mind focused on some other thing, right? Because if you constantly worry about the disease, it becomes worse. Foremost is being I have been told that I cannot be troubled. I cannot be worried. And if someone is arguing or there is some kind of problem, you have to get away from it, they say. If not, you are going to get palpitations. tranquilizarnos), because if we become upset, it is another ill at the same time. We have to calm down The first thing, you know what it is? One’s tranquilo) not only for one’s emotional wellbeing, but to lessen the effects of CD. Even otherwise positive emotions might be destructive if they impact the tranquilidad of a person with CD. “I can’t get excited,” says Pura. “I avoid anything like that. I can’t go to parties. Sometimes I get mad, I get really happy, and all that does damage. It does damage, so you have to be just normal.”These examples underscore the need to be calm  affirmed that they felt calm–Tranquila, I don’t feel palpitations in my heart. I think that–it’s like I don’t have the disease, no? So, I feel tranquila because others say they have Chagas and feel agitation in their hearts.Until now I feel that I don’t have Chagas. Tranquila, nothing more. As if it were nothing.”In part, the peace she feels is due to her lack of symptoms, but just as importantly, she does not permit herself to worry about the possibility of developing a more severe, symptomatic form of the disease. Alma perceives that such worrying would act as a trigger that aggravates her CD. In a similar example, Porfiria, 54, indicates her diagnosis of CD “didn’t affect me… I didn’t say, ‘Oh no, it’s going to kill me,’ or anything. Well, you’ve got Chagas. ranquilidad implies acceptance of the possibility of death which a diagnosis of CD entails. Hugo, a 62-year-old teacher, avows he feels, “Tranquilo. Because we have arrived in this world, and I am conscious that we have to depart as well. It’s not forever. Tranquilo. Tranquilo.” Elsa, a 35-year-old storekeeper, also considers herself “tranquila. Anyway, you’re going to die of something. You just have to wait for it.” Josefina, 53, who works cleaning houses, asks, “Why are we going to get upset? Tranquilo, you have to die from something.”Importantly, ttranquilidad.Five respondents (7.9%) mentioned faith in God or the Virgin helps bring about acceptance of the disease and I was sad a long time. I told my children and my husband that I had this disease, and they told me you have to have faith that God will protect us and care for us. From then on we always ask God and the Virgin to protect us and care for our health. Constanza “worried a lot” when she first found out she had Chagas, but later came to accept it, something she attributes to “God. Because God comes first, you see? And that gave me more courage with this disease.” In all these examples, faith helps individuals come to terms with having the disease, and in particular enables them to put an end to worrying.tranquilo, even though I had Chagas, I didn’t give it a lot of importance,” says Eduardo, a 54-year-old agricultural worker. “That was the good thing.” When asked what helped him feel calm, Eduardo responded, “being able to work, do something, clean or straighten up, I feel happy. And well, you get used to it.” “They told me it was better not to worry about the disease that I had,” recalls Anita, a 47-year-old vendor of salteñas, a local delicacy similar to an empanada. “I know it can get worse very quickly. I spent my whole life working, and in that I keep things normal. It’s as if I didn’t have the disease. I work and work and work, and I don’t even know if I’m sick or anything.” For these individuals, being able to continue functioning normally is the key to tranquilidad. At the same time, these patients perceive it is essential not to become too worried or upset, since this might set off the symptoms of Chagas disease.Two people mentioned they use work as a means of putting their minds at ease. “I felt tranquilidad stems from avoidance of dealing with CD. In an illustrative example, Blanca, 62, states that while she does not worry about her CD, she does pay attention to her diabetes: “No, I don’t think about [CD]. I think of it as nothing. What I think about is just my sugar, that it doesn’t go up on me.” Blanca’s CD is asymptomatic, so in part this might reflect a pragmatic choice on her part about where to focus her time and attention. On the other hand, there are specific actions she feels she can take to combat her diabetes, such as watching her glucose and controlling her diet. She is able to do something about her diabetes. With CD, often the only viable action is to avoid stress and worry.It would be incorrect to state that tranquila. Yes, because I can’t take any type of medicine. Here in the countryside we use sweetened teas to calm down and I can’t because of my sugar (diabetes).” Leticia feels she is unable to avail herself of medicinal teas  because they are typically prepared with sugar. Thus, trying to stay tranquila is the only option left to her. Importantly, her comment expresses the idea that medicinal teas themselves can help a person feel tranquilidad.Leticia, who is also both diabetic and Chagas-positive, states that “the only thing I do is try to stay tranquilidad helps patients obtain freedom from anxiety and regain a sense of normality in their lives. A return to normalcy could also result from successful biomedical treatment, but the path is much more difficult.tranquilo helps patients cope with their CD diagnosis and maintain a sense of normality in their lives. Kleinman distinguishes between “disease”: the biological impact of a particular pathogen or disorder, and “illness”: the wide variety of social, emotional, and other impacts experienced by patients in their daily lives[Tranquilidad helps patients manage illness by preventing disruption of daily activities, yet patients perceive it is also critical to preventing a worsening of disease symptoms, which patients feel would be triggered by strong emotions. Respondents indicate tranquilidad counteracts the agitation of the heart that often accompanies advanced chronic CD. An individual who is tranquilo is in control of worries about impending death or debilitation from CD.Remaining ily lives. IllnessIn the social environment, people with CD may suffer from stigmatization because of the disease’s association with poverty, 38, 39.tranquilidad as resignation or fatalism on the part of patients, it is more useful to consider it a coping strategy that enables patients to live with CD, especially given the significant barriers to accessing treatment. These include the distance of providers, hidden costs such as consultations and laboratory analyses, and conflicting messages about the curability of the disease[While it would be tempting to view e disease. While pe disease–43, thise disease. For inse disease. In othee disease.Other research has examined patient strategies for coping with CD, with similar findings. A study in La Plata, Argentina found that men with CD, while experiencing fear and worry about the disease, went through a process of normalization and acceptance. SimilarResearch has begun to explore the link between CD and emotional health, 50. In Tranquilidad thus represents a coping strategy which helps patients safeguard their health and return to a semblance of normal functioning. This strategy, along with ethnomedical remedies, permits patients to self-manage their CD, given the difficulties they face in securing biomedical treatment. Current treatment of CD often maintains a narrow, biological focus that disregards patients’ emotional and social well-being. There is a need for more holistic strategies that take into account the socioeconomic conditions and emotional health of patients, enabling them to minimize stigma and return to daily activities[In sum, patients in this study feel that excessive anguish or worry over CD would actually aggravate the disease. ctivities. While dBecause of the difficulties in confirmatory testing, most patients had only had one test for CD, whereas the World Health Organization recommends both an initial and a confirmatory test. I did not collect reliable data on how many patients were suffering from advanced symptoms of CD. While most were asymptomatic, some had clinically evident CD-related complications, and several suffered from comorbidities including diabetes, age, high blood pressure, and obesity. This study describes a convenience sample of patients whose experiences may not be representative of other patients with CD. Because the author was simultaneously serving as a volunteer at the CMHP, patients may have perceived him as part of the clinic’s staff, which could have influenced some of their responses. The author explained to patients at the beginning of each interview that he was an independent investigator and not a doctor with the clinic, but his positionality may have been unclear to some, especially since most patients were recruited within the clinic. This is essentially a cross-sectional study, so the author was unable to document changes in patients’ coping strategies over time. This is important because the department’s program of free treatment for Chagas disease was in a process of expansion during the study period, so patients may have greater awareness of this program now than they did in 2013 when the investigation took place.S1 Checklist(PDF)Click here for additional data file.S2 Checklist(PDF)Click here for additional data file.S3 Checklist(PDF)Click here for additional data file.


In clinical medicine, evaluation of loss of particular function(s) is fundamental to both diagnosis, and planning possible interventions that may be restorative. The term diagnosis is etymologically derived from the Greek diagignoskein, “to discern.” The root of the word, gignṓskein (γιγνώσκειν) means “to learn”; the implication being that one must gain knowledge about those aspects of the thing(s) that are important for it to be distinguished, identified and characterized. If, however, the focus of discernment is the clinical assessment of consciousness in a patient who is unable to respond and/or communicate through speech or overt behavior, then the act of discernment becomes complicated, given that the cardinal characteristics of consciousness are subjectivity and self-transparency, and to that extent not viable for direct objective evaluation.In any attempt to recover a loss, it becomes important to assess Indeed, consciousness remains the elusive, proverbial “hard question” of neuroscience, and in many ways, this is exactly the point. However, it is a question and problem that cannot simply be ignored, ascribed to mere speculative explanations, or devalued. To the contrary, it must be confronted, as the discernment of consciousness is not merely a philosophical matter, but also of practical importance in, notably, neurological diagnoses, prognoses and clinical care of patients with disorders of consciousness (DoC), such as coma, minimal consciousness or persistent unresponsive states. Such considerations of diagnosis and care are not limited to the clinical milieu. They generate ethico-legal issues that conjoin other professions  and the public in ongoing discourse and deliberation about how such patients are, or should be regarded  and treated in both the short- and the long-term, inclusive of current debates about the use of brain-machine interfaces , continuity and quality of care, and the justification of life-sustaining measures.per se  of consciousness; and “m” stand for material substrates  involved in particular functional domains “d” of these processes that are held to be important in and for clinical, ethico-legal, and social value. Given this, one methodological approach might be to assess key patterns  in specific functional domains (“m/d”) that are reflective of neural activity indicative of efficient processes of consciousness (“a/e”), such that Pm/d ≈a/e as obtained under empirically defined conditions in accordance with (the most applicably) current theoretical models of the relationship of brain function and consciousnessIn “putting such pieces together,” we see this both as representative of “tools-to-theory-to-tools” heuristics, and as a puzzle to be solved (and suggestion of methods toward such solution), rather than an enigma simply to ponder  neuroimaging could enable better conceptualization of structural and functional correlates of consciousness, which in turn could be useful to informing decisions about the kind and extent of care patients should be provided . We have previously emphasized the importance and need for patients  to be knowledgeable about what technologies are available and their possible uses, allowing them to express advance directives to state whether and which neurotechnologies they, should the situation arise, would wish to be used to define—and perhaps attempt to restore—their neurological integrity (Giordano, person in whom this brain~mindIn any case, current difficulties in correctly diagnosing DoCs should prompt new tools to be developed that decrease error or uncertainty, inclusive of assessment neurotechnologies, and more sensitive, standardized neurobehavioral metrics. But we suggest that an additional goal is not just to assess brain functions of patients with DoC, but to access them, and thereby engage the All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Many biologic products have improved the outcomes of cancer patients, but the costs can substantially burden healthcare systems. Biosimilar products can potentially reduce drug costs and increase patient access to beneficial treatments. Approval of a biosimilar product relies on the demonstration of “comparability” or “no clinically meaningful differences” as compared to its reference biologic product. Biosimilar products for erythropoietin, granulocyte colony-stimulating factor, trastuzumab, and rituximab are already available, and the regulatory processes in various countries are constantly evolving. It is important that oncologists be familiar with the potential issues surrounding the clinical use of biosimilar products. In this review article, we provide background information about biosimilar products and their regulatory approval processes, followed by a discussion of individual biosimilar drugs. Biologic therapies, such as the monoclonal antibodies (mAbs) trastuzumab and bevacizumab, have prolonged the survival of cancer patients, but their high costs have also resulted in substantial financial burdens being placed on individual patients and on healthcare systems as a whole . Because“Biosimilar”, as the name implies, is a biologic that is similar to the licensed “reference” drug. Before a biosimilar is incorporated into the clinical management of patients, clinicians should determine whether, based on efficacy and safety, the biosimilar can be used “instead of” or “interchangeably with” the reference biologic. Regulatory authorities play an important role in the development of biosimilars, and medical societies help guide clinicians on the use of biosimilars –5. Curre“Biosimilar” , 4, “simThe European Medicines Agency (EMA) is the body responsible for approving biosimilars in the European Union (EU), and it established the first legislative pathway for doing so. In 2005, it published guidelines governing the development of biosimilars; since then, it has developed individual guidelines for the biosimilars of granulocyte colony-stimulating factor (G-CSF), erythropoietin, and various mAbs . The EMAIn the EU, the first biosimilar product was approved in 2006 . Many coOverall, the USA and the EU have similar basic principles in terms of quality and clinical and non-clinical comparability testing strategies , 4. A siPharmacovigilance is “the science and activities relating to the detection, assessment, understand and prevention of adverse effects or any other drug related problem” . PharmacGeneric medications can be used interchangeably with their branded originators since they are considered therapeutically equivalent; often, pharmacists may substitute a prescribed drug for a generic medication without the prior consent of the treating physician (known as “automatic substitution”) . InterchSome biosimilars might not have all the same indications that the reference biologics are approved for, especially if the reference biologics have multiple mechanisms of action. Even if a biosimilar is approved for the same indication as the reference originator, it is generally recommended that automatic substitution not be allowed because the clinician should be fully aware of which drug is given to the patient. If, during a treatment period, alternation or switching between the biosimilar and the branded biologic cannot be avoided, this must be recorded accurately; maintaining pharmacovigilance by clearly delineating between biosimilars and branded originators is important.The USA allows an “interchangeable” designation for biosimilars, provided that, in any given patient, the biosimilar can be expected to produce the same clinical results as the reference product and that the safety and efficacy observed when alternating or switching between the two remain the same . Other cErythropoiesis-stimulating agents (ESAs), such as epoetin alfa , epoetin beta (NeoRecormon), and darbepoetin alfa (Aranesp), are approved for cancer patients with chemotherapy-induced anemia , 29. TheIn 2007, two biosimilar epoetins–epoetin alfa, marketed as Binocrit, Abseamed, and Epoetin Alfa Hexal; and epoetin zeta, marketed as Retacrit and Silapo were approved in Europe –35 Tabl. ComparaG-CSF is widely used in the field of oncology . It is uEnhanced by a 20-kDa polyethylene glycol molecule compared with filgrastim, pegfilgrastim has a longer half-life and is administered less frequently . PegfilgTrastuzumab is a humanized mAb that targets human epidermal growth factor receptor 2 (HER2). For patients with HER2-positive breast cancer, it is the standard of care in the neoadjuvant, adjuvant, and palliative settings , 58. It Rituximab is an mAb that acts against CD20 to treat non-Hodgkin’s lymphoma (NHL) and chronic lymphocytic leukemia. Marketed by Roche as Rituxan and MabThera, rituximab was originally approved for use in 1997 in the USA  and in 1The rationale for the development of biosimilar products is to improve patient access to biological therapies in a safe and cost-effective way. As the technology advances and evolves, so too do regulations governing the development of biosimilars. Oncologists should be aware that biosimilars are not generics of biological medicines, but they do have comparable efficacy and safety to the originators. Pharmacovigilance is critical to ensuring the long-term quality and safety of biosimilars. With biosimilars, it is important to be mindful of concerns about drug substitutions and extrapolation across indications.


Among these oxidized molecules, lipid peroxidation-derived products  or quinones are highly reactive and form adducts with amine moieties in DNA and amine/thiol moieties in proteins.–9 Oxidative stress is considered to be caused by and be a result of disease. Therefore, oxidized molecules could be factors in the initiation, promotion, and development of pathologies such as cardiovascular and neurodegenerative diseases. Therefore, the detection of these molecules could serve as a “torch” for researchers to reveal the picture of such diseases. These modified molecules are a kind of “rust” in living systems and could be indicators of a balance between oxidation and reduction.As a consequence of oxidative stress, lipids, proteins, and DNA are endogenously oxidized, and this leads to the formation of modified molecules in tissue and blood. These effects may be triggered by specific or non-specific reactions between ingested xenobiotics and various biological molecules within cells. Plant-derived isothocyanates can form adducts with proteins and thereby induce cellular responses, while some polyphenols can conjugate with proteins via their quinone moieties. These xenobiotics may act as “toxic chemicals” that induce self-defense mechanisms in the body.–17 Alternatively, some biological effects could be caused by “indirect” reactions between a phytochemical and a cell via a receptor. This means that the “brain-gut interaction” is a pathway to instantiate functions triggered by the ingestion of fruits and vegetables.We have food every day. Food contains nutrients . Plant-derived food, in particular, is also rich in non-nutrients such as phytochemicals . This indicates that “xenobiotics”, which are not true nutrients, are incorporated into our daily diets whether we like it or not. Nutritionally, we do not “digest” these xenobiotics , but some are circulated through the body either intact or in conjugated forms. These ingested phytochemicals could elicit biological effects. Indeed, epidemiological studies have shown the beneficial effect of polyphenols in, for example, the prevention of cardiovascular diseases.In these serial reviews, current progress in covalent modification of biomolecules by food or oxidatively generated compounds will be reviewed in six separate papers from different points of view. In particular, these reviews will focus on the biological relevance of the interaction of oxidized or ingested molecules with biomolecules and their role in triggering biological functions via shared pathways Fig. .


In many disciplines, data are highly decentralized across thousands of online databases . Wringing value from such databases depends on the discipline of data science and on the humble bricks and mortar that make integration possible; identifiers are a core component of this integration infrastructure. Drawing on our experience and on work by other groups, we outline 10 lessons we have learned about the identifier qualities and best practices that facilitate large-scale data integration. Specifically, we propose actions that identifier practitioners (database providers) should take in the design, provision and reuse of identifiers. We also outline the important considerations for those referencing identifiers in various circumstances, including by authors and data generators. While the importance and relevance of each lesson will vary by context, there is a need for increased awareness about how to avoid and manage common identifier problems, especially those related to persistence and web-accessibility/resolvability. We focus strongly on web-based identifiers in the life sciences; however, the principles are broadly relevant to other disciplines. For instance, the above URI could be represented as uniprot:A0A022YWF9. This deterministic conversion makes it easy for meta resolvers as well, .An tionable : it is aier (URI), of typepersistent identifiers is harder than it appears. There are a number of approaches that may be used for this purpose, but no single one is perfect. Identifiers are reused in different ways for different reasons, by different consumers. Moreover, digital entities , physical entities , living entities , and descriptive entities  have different requirements for identifiers [Suboptimal identifier practice is artificially constraining what can and cannot be done with the underlying data: it not only hampers findability, accessibility, interoperability, and reuse (FAIR principles) ,10, but ntifiers .The problem of identifier management is hardly unique to the life sciences; it afflicts every discipline from astronomy  to law . TowardsThroughout this document, the word “must” is reserved for practices that ensure against the collision, ambiguity, or inaccessibility of items referenced by identifiers; instances of “must” are also often specific to particular design choices. We use the word “should” to convey that the trade-offs must be understood and carefully weighed before choosing a different course .There is no one in science that is unaffected by identifiers. Many of the following recommendations are applicable during the planning and identifier conceptualization phase, i.e., before any identifiers are created. The retrofitting  of existing identifiers can sometimes be too difficult or may even make matters worse: for instance, changing existing identifiers introduces the need for systems that can recognize the variations for what they are; such overhead can outweigh potential benefits. Each of the lessons is relevant to the basic classes of identifier actions , consider its role in identifying and referencing the knowledge that it publishes. We advise that you only create your own identifiers for new knowledge . In spite of how important identifiers are to this process, the confusion with identifiers often starts with the basics, including what the “identifier” even is. A local ID  on the web. This is often accomplished through the use of a prefix, which should be documented. If this is overwhelming, don’t forget that there are meta resolvers and services built to help for exactly this reason (see Lesson 3).If you reference others’ data or anticipate your data being referenced by others, consider how you document your identifiers. Note that you may not know a priori how your data may be used. Data do not thrive in silos: they are most useful when reused, broken into parts, and integrated with other data, for instance in database cross references  standard and (b) the http URI’s uniqueness is ensured by a single well-established name–granting process (DNS). However, the length of URIs can make them unwieldy for tasks involving human readability even within structured machine-parsable documents. CURIEs [JSON-LD and RDFa) as they enable URIs to be understood and conveniently expressed. We, the authors, are not absolutist about anyone using CURIEs; however, we agree that the features that make for good URIs also happen to make CURIEs possible   URIs areprefix  the prefix  that you pattern . Your chA core component of persistent identification is redirection, the absence of which makes it extremely difficult to provide stable identifiers. When designing (or refining) your http URI strategy:http URI pattern , Identifiers.org, Handle.net, PURL (now via InternetArchive), EPIC, ARK) and be mindful of your constraints regarding cost, metadata ownership, turnaround time, versioning support, etc , query strings , and technology choices (“.php”). Never embed the local ID in the query part of a URI e.g., http://example.com/explore?record=A123456.Avoid inclusion of anything that is likely to change or lapse, including administrative details  or implementation details such as file extensions  and, if applicable, trailing slash or other delimiters. Trailing characters after the local ID are strongly discouraged as they unnecessarily increase the variability with which the identifier is represented and also complicate straightforward appending of the local ID .Omit trailing characters after the local ID. In all cases, the http://example.com/A123456 rather than http://example.com/vertebrates/mammals/rodents/rat/white-rat/A123456); see also Lesson 5 regarding types and meaning. The CURIE approach can work with any resolver(s): see for instance examples 4 and 5 in URI pattern, you make it possible for others to resolve your identifiers simply  can and should be conveyed in the HTTP header where possible [local ID (MUSG…ENS), or within the http URI path (gene/12345…/), or both. In any case, if you opt to include type in the identifiers you issue, avoid relying on type for uniqueness: that is to say, once a local ID  is assigned it should never be recycled for another entity, even an entity of a different type .When designing new possible . The strlabel, for instance as is done in model organism nomenclature such as by Mouse Genome Informatics . Labels are for human readability only; even if they are deemed durable, labels should not be treated as identifiers, nor should they appear within http URIs. URI patterns, if type-specific, require a corresponding type-specific prefix. For example, the Library of Integrated Network-based Cellular Signatures (LINCS) contains entities of several types including cells and proteins. Cell records are resolved using the pattern http://lincs.hms.harvard.edu/db/cells/, whereas protein records are resolved using the pattern http://lincs.hms.harvard.edu/db/proteins/; thus, if more than one lincs type is referenced/integrated in the same context, it requires the use of two different prefixes, e.g., such that lincs.cells:50001 → http://lincs.hms.harvard.edu/db/cells/50001 and lincs.protein:200001 → http://lincs.hms.harvard.edu/db/proteins/200001, respectively. By contrast, MGI implements a single prefix for all types of entities in their corpus ; accordingly, this prefix “MGI” corresponds to a single URI pattern (http://www.informatics.jax.org/accession/). Thus the single MGI resolver works for all accessions, regardless of type, and redirects them to their corresponding type-specific destination  all without the user needing to know the type beforehand. Dual approaches like MGI’s can be helpful to different kinds of consumers: type-agnostic resolution is useful in cases such as data citation in the literature where (a) the type of the identified entity is not of primary importance, or (b) the type of the entity is already conveyed contextually, and/or (c) where resolution is done systematically at scale and/or involves many and varied or volunteer contributors that may be difficult to coordinate. Type-specific resolution is useful in cases like bioinformatic research pipelines where embedded type may facilitate the human-led debugging process. If you support both kinds of resolution, it is best to document (a) whether you intend for both to be treated as persistent and (b) what mapping support you provide. Note that while type-agnostic resolution has important advantages, it must only be undertaken if all local IDs of any type, past and future, can be guaranteed to never collide.If you need the ability to convey meaning in a dense character space, you don’t need to do so in the identifier itself; consider instead implementing an entity local IDs include type, you should provide other ways for humans and machines to determine the type of entity that is being identified; this is most often achieved via web services , but ideally also within metadata landing pages [Whether or not your URIs or your ng pages ,25, if plocal IDs, there are some design decisions that can facilitate their use in diverse contexts .Preexisting identifiers should be referenced without modifications (see Lesson 10). However, if you create new local ID be in CURIE format because modern browsers resolve colons without having to encode them. Although characters “/” and “?” are technically URL-safe, they are very problematic when used within the local ID, as these characters are assumed to have special meaning and can complicate parsing of the identifiers, whatever forms they take. For the same reason, local IDs should ideally not contain the dot character (“.”) except to denote version where appropriate (see Lesson 7).Avoid problematic characters. Local IDs should, wherever possible, comprise only letters, numbers, and URL-safe delimiters. Omission of other special characters guards against corruption and mistranscription in many contexts; however, it is acceptable that the Local IDs must adhere to a formal pattern (regular expression); this facilitates the validation of URIs and improves the accuracy of mining identifiers from scientific text. Consider a fixed length of 8–16 characters . A pattern may be extended if all available identifiers are issued, but existing identifiers should not be changed. To minimize local ID collisions at a global scale, it is considerate to tightly specify your pattern . The regular expression should include a fixed, documented case convention. In most cases, it is advised that identifiers not rely on case for their uniqueness: if you assign ab-12345 to one entity and AB-12345 to a different entity, collisions due to mistranscription are more likely. Case-sensitive patterns are best reserved for when brevity is a constraint, and hand transcription is not .Define a formal pattern and stick to it. local ID, but if you do use both consider omitting characters that can be mistranscribed [prefix from the rest of the identifier. Therefore “_” or “:” should (a) occur no more than once per identifier, and (b) should only be used if local IDs are intended to be deterministically expanded to a resolvable http URI. For instance, if your intended prefix is “MyDB”, then either MyDB:gene-6622 or MyDB_gene-6622 are acceptable patterns, but MyDB_gene_6622 is problematic, as it could result in three possible conversions by others, even if these are not intended: MyDB_gene:6622, MyDB:gene_6622, MyDB:gene:6622. Whatever pattern you adopt, document which variations you support resolution of, if any.Avoid problematic patterns. Consider using both letters and numbers in the nscribed  This avonscribed , or as unscribed . A histoWhether you produce original data, or reference data from others, consider the impact of data changes. The nature, extent, and speed of data changes impact how data can be referenced and used. Document your chosen version management practice: if you issue identifiers, the change history for the entity should be either documented or retrievable with a URL-based query. Alternatively, the identifier itself can be versioned whether or not change history is also supported.“tombstone” page (Lesson 7). A resource should communicate clearly what a version change refers to. UniProt and RefSeq use versions to reflect changes in sequence. Ensembl uses versions to reflect changes in sequence and splicing for transcript records but sequence alone for protein records. In each of these examples, the changes in the annotations that are attached to a record does not alter the version.Embedding versioning in identifiers is recommended if the prevailing use of an unversioned identifier results in “breaking changes” . However, if new information about the entity emerges slowly and the changes are “nonbreaking”, it is reasonable to instead maintain a machine-actionable change history in the entity’s metadata. The history should be a list of changes categorized in such a way that users can assess the impact on their work. Versioning and change history work well together, especially when multiple types of changes overlap. Even where previous records are entirely removed, the URI should continue to resolve, but to a http://e85.ensembl.org/id/ENSMUSG00000033577). If you do not have the ability (or common use case) to maintain individually resolvable-archived records, we strongly recommend that you (a) support export to files so that users can archive the records they need, and (b) make snapshots available for the database, whether in whole or in parts [There are two approaches to versioning: record-level  and relein parts .local ID after the “dot”, as per UniProt in metadata-only versioning . Although the topic of when and how to version data is of great interest, use cases vary and consensus is elusive. Other groups have discussed change management consideration and “content drift” in more depth [If you version identifiers at the level of the individual record, the most common approach in the life sciences is to version in the shown in ; moreovere depth ,31.Identifiers that you have exposed publicly, whether as http URIs or via APIs, may be deprecated but must never be deleted or reassigned to another record. If you issue identifiers, consider their full life cycle: there is a fundamental difference between identifiers which point to experimental datasets  and identifiers which point to a current understanding of a biological concept . While experimental records are less likely to change, concept descriptions may evolve rapidly; even the nature and number of the relevant metadata fields change over time. Moreover, the very notion of identity is often strongly impacted by relationships .Extensive changes cannot be captured with numerical suffixing alone. For instance, taxonomists may split or merge species, pathologists may split or merge diseases, or hypothesized entities may be proven not to exist . Global initiatives  are actipermalink) together with a statement about persistence. E.g.:Persistent URIs almost always differ from the ephemeral URLs to which users are ultimately directed . Therefohttp://e85.ensembl.org/id/ENSMUSG00000033577 We aim to maintain all archives for at least five years; some key releases may be maintained for longer”“The permanent link to this page, which will not change with the next release of Ensembl is: http://www.uniprot.org/uniprot/P12345.1, for instance). Although it is good practice for each database website to include general citation guidance for users [For archived records that are out of date, make this clear to the user and provide a link to the updated version  and must not be used for referencing or linking in any context intended to persist.When external entities are referenced in narrative online text, they should be hyperlinked where possible, either to their URIs or to a destination that documents their URIs. Broader issues associated with citation of data and software in the traditional literature are outside of the scope of this paper, but Redistributors of data should monitor their references to other sources; any “dead” links should be reported to the original data provider. If the original provider does not fix the broken link, your reference to it should be marked obsolete both visibly (for user interaction and/or interpretation), and within any accompanying metadata . Differentiate the identifiers that are internally linked within your application from those identifiers linked outside your application. One way to do this is by using the linkout icon; consider opening all external links in a new browser window or tab in order to avoid confusion.Better identifier design, provisioning, documentation, and referencing can address many of the identifier problems encountered in the life science data cycle—leading to more efficient and effective science. However, best practices will not be adopted on the basis of their community benefit alone; the practices must be both easy and rewarding to the groups that do the implementing. In the broader context of scholarly publishing, this is just what DOIs afford; DOIs succeeded because they were well aligned with journals’ business goals (tracking citations) and because the cost was worth it to them. However, in the current world where everyone is a data provider, alignment with business goals is still being explored: meta resolvers can provide a use case for journals and websites seeking easier access to content, while software applications leverage these identifier links to mine for knowledge.We recognize that improvements to the quality, diversity, and uptake of identifier tooling would lower barriers to adoption of the lessons presented here . Those tS1 TableLocal ID that appear in fixed-width font throughout the paper) are formally defined herein.Technical terms relevant to web technology and identifiers (those such as (PDF)Click here for additional data file.S2 Table(A) Primary registries for prefixes and/or URI patterns and (B) secondary sources of same.(PDF)Click here for additional data file.S3 TableDespite its usefulness to data integrators, identifier documentation is often overlooked and underspecified. This table describes the questions that comprehensive identifier documentation should answer.(PDF)Click here for additional data file.S1 TextInternational initiatives that are working to address various issues related to identifiers in the scholarly landscape.(PDF)Click here for additional data file.S2 TextThe features that make for a good persistent URI also make for good CURIEs: desirable features include lack of semantics in both the URI pattern and the local ID (Lesson 4), absence of characters after the local ID (Lesson 5), omission of problematic characters, etc. (Lesson 5). CURIEs can complement http URIs in important ways for curators and data integrators: location-independence, brevity, clues to collapse equivalents. These features are described.(PDF)Click here for additional data file.S3 TextThere are basically three kinds of approaches to serving URIs on the web: (A) “native” URIs that require no redirection at all as in , ZFIN, , Ensembl(PDF)Click here for additional data file.S4 TextA list of current and future tooling and technologies that could help lower barriers to the adoption of identifier best practice.(PDF)Click here for additional data file.