The emergence of large language models has led to the development of powerful tools such as ChatGPT that can produce text indistinguishable from human-generated work. With the increasing accessibility of such technology, students across the globe may utilize it to help with their school work—a possibility that has sparked ample discussion on the integrity of student evaluation processes in the age of artificial intelligence (AI). To date, it is unclear how such tools perform compared to students on university-level courses across various disciplines. Further, students’ perspectives regarding the use of such tools in school work, and educators’ perspectives on treating their use as plagiarism, remain unknown. Here, we compare the performance of the state-of-the-art tool, ChatGPT, against that of students on 32 university-level courses. We also assess the degree to which its use can be detected by two classifiers designed specifically for this purpose. Additionally, we conduct a global survey across five countries, as well as a more in-depth survey at the authors’ institution, to discern students’ and educators’ perceptions of ChatGPT’s use in school work. We find that ChatGPT’s performance is comparable, if not superior, to that of students in a multitude of courses. Moreover, current AI-text classifiers cannot reliably detect ChatGPT’s use in school work, due to both their propensity to classify human-written answers as AI-generated, as well as the relative ease with which AI-generated text can be edited to evade detection. Finally, there seems to be an emerging consensus among students to use the tool, and among educators to treat its use as plagiarism. Our findings offer insights that could guide policy discussions addressing the integration of artificial intelligence into educational frameworks. In the realm of art, for example, Stable Diffusion and DALL-E have received significant attention for their ability to generate artwork in different styles3. Amper Music, another generative AI tool, is capable of generating music tracks of any given genre, and has already been used to create entire albums5. ChatGPT is the latest tool in this field, which can generate human-like textual responses to a wide range of prompts across many languages. More specifically, it does so in a conversational manner, giving users the ability to naturally build on previous prompts in the form of an ongoing dialogue. This tool has been described as an “extraordinary hit”6, and a “revolution in productivity”7, for its seemingly endless utility in numerous out-of-the-box applications, such as creative writing, marketing, customer service, and journalism, just to name a few. The capabilities of the tool have sparked widespread interest, with ChatGPT reaching one million users in only five days after its release8, and soaring to over 100 million monthly users in just two months.Generative artificial intelligence (AI) refers to the use of machine learning algorithms that build on existing material, such as text, audio, or images to create new content. Recent advancements in this field, coupled with its unprecedented accessibility, has led many to consider it a “game-changer that society and industry need to be ready for”11. Furthermore, as these tools continue to evolve, so does the challenge of identifying what is created by humans and what is created by an algorithm. In the context of education, ChatGPT’s ability to write essays and generate solutions to assignments has sparked intense discussion concerning academic integrity violations by school and university students. For instance, in the United States its use has been banned by school districts in New York City, Los Angeles, and Baltimore12. Similarly, Australian universities have announced their intention to return to “pen and paper” exams to combat students using the tool for writing essays13. Indeed, many educators have voiced concerns regarding plagiarism, with professors from George Washington University, Rutgers University, and Appalachian State University opting to phase out take-home, open-book assignments entirely14. In the realm of academic publishing, a number of conferences and journals have also banned the use of ChatGPT to produce academic writing16, which is unsurprising given that abstracts written by ChatGPT have been shown to be indistinguishable from human-generated work17. Yet, many have argued for the potential benefits of ChatGPT as a tool for improving writing output, with some even advocating for a push to reform the pedagogical underpinnings of the evaluative process in the education system19.In spite of its impressive ability, generative AI has been marred by ethical controversies. In particular, as generative AI models are trained on massive amounts of data available on the internet, there has been an ongoing debate regarding the ownership of this data23, computer and data science26, law28, business29, journalism and media30, and language learning32. While these studies demonstrated mixed results with regards to ChatGPT’s performance in standardized exams against that of students, studies which specifically compared the model’s performance against that of previous large language models all indicated a significant improvement in the task of question answering23. For instance, Kung et al. evaluated the performance of ChatGPT on the United States Medical Licensing Exam, where they show that ChatGPT performed at or near a passing level on all three stages of the test, without the need for additional specialized training or reinforcement21. Similarly, Pursnani et al. evaluated ChatGPT’s performance in the context of Engineering, testing the model on the US Fundamentals of Engineering exam33. In their study, they showed that the model’s performance varied across different sections of the exam, performing exceptionally in areas such as Ethics and Professional practice, while under-performing in others such as Hydrology. Despite these examples, the literature still lacks a systematic study comparing the performance of ChatGPT against that of students across various disciplines at the same institution. Moreover, the perspectives of students and educators around the globe regarding the usage of such technology remain unclear. Finally, the detectability of ChatGPT-generated solutions to homework remains unknown.Previous work has examined the utility and performance of large language models in the context of education in a number of different disciplines, including medicine and healthcareHere, we examine the potential of ChatGPT as a tool for plagiarism by comparing its performance to that of students across 32 university-level courses from eight disciplines. Further, we evaluate existing algorithms designed specifically to detect ChatGPT-generated text, and assess the effectiveness of an obfuscation attack that can be used to evade such algorithms. To better understand the perspectives of students and educators on both the utility of ChatGPT as well as the ethical and normative concerns that arise with its use, we survey participants (N=1601), recruited from five countries, namely Brazil, India, Japan, United Kingdom, and the United States. Additionally, we survey 151 undergraduate students and 60 professors more extensively at the authors’ institution to explore differences in perceptions of ChatGPT across disciplines. We find that ChatGPT’s performance is comparable, or even superior, to that of students on nine out of the 32 courses. Further, we find that current detection algorithms tend to misclassify human answers as AI-generated, and misclassify ChatGPT answers as human-generated. Worse still, an obfuscation attack renders these algorithms futile, failing to detect 95% of ChatGPT answers. Finally, there seems to be a consensus among students regarding their intention to use ChatGPT in their schoolwork, and among educators with regard to treating its use as plagiarism. The inherent conflict between these two poses pressing challenges for educational institutions to craft appropriate academic integrity policies related to generative AI broadly, and ChatGPT specifically. Our findings offer timely insights that could guide policy discussions surrounding educational reform in the age of generative AI.We start off by exploring the current capabilities of generative AI to solve university-level exams and homework. To this end, we reached out to faculty members at New York University Abu Dhabi (NYUAD), asking them to provide 10 questions from a course that they have taught at the university, along with three randomly-chosen student answers to each question. Additionally, for each course, ChatGPT was used to generate three distinct answers to each of the 10 questions. Both students’ and ChatGPT’s answers were then compiled into a single document in random order, labelled as “Submission 1” to “Submission 6”. Each of these submissions were then graded by three different graders, recruited by the faculty member who had taught that course; see Methods for more details, and Supplementary Table 34; see Table Having analyzed ChatGPT’s performance on different courses, we now perform an exploratory analysis of how its performance varies along different levels of cognition and knowledge. To this end, we asked participating faculty to specify where each of their questions fell along the “knowledge” and “cognitive process” dimensions of Anderson and Krathwohl’s taxonomyTo understand how the use of ChatGPT is perceived by educators and students, we fielded a global survey in five countries, namely Brazil, India, Japan, the UK, and the US, targeting a minimum of 100 educators and 200 students per country; see Methods for more details. A summary of our findings can be seen in Fig. Next, we compare the distribution of the educators’ and students’ responses to the following question: “What percentage of your students/peers do you think will use ChatGPT in their studies?”. The results are depicted in Fig. Finally, we perform an OLS regression analysis to explore which factors that might be associated with the students’ decision to use ChatGPT during their next term of studies. Figure Having analyzed the global survey, we now shift our attention to the second survey, which was conducted at the authors’ institution—NYUAD. While this survey is narrower in scope than the previous one, it focuses on university students and professors, allowing us to examine variances in responses with respect to GPA in the case of students, and type of appointment in the case of professors. Figure Figure Figure GPTZero35 and OpenAI’s own AI text classifier36, both of which are designed specifically to determine whether a body of text has been generated using AI. We use these classifiers to quantify the percentage of human submissions that are misclassified as ChatGPT, as well as the proportion of ChatGPT submissions that are misclassified as human. More formally, our goal is to construct a normalized confusion matrix for each classifier, and to quantify its false-positive and false-negative rates; see Methods for more details. Out of all 320 questions in our dataset, there were 40 questions whose answers consisted entirely of mathematical equations that are incompatible with the classifiers; those were excluded from our evaluation, leaving us with 280 questions. The results are depicted in Fig. We conclude our analysis by assessing the detectability of using ChatGPT. To this end, we use two classifiers, namely 37—a popular paraphrasing/re-writing tools utilized by students worldwide. Our “Quillbot attack” was chosen since it can be readily implemented by students, without requiring any technical know-how; see Methods for a detailed description of the attack, and see Fig. 38.Next, we analyze the degree to which these classifiers are susceptible to obfuscation attacks. We devised a simple attack that involves running the ChatGPT-generated text through Quillbot40.This study aimed to provide both quantitative and qualitative insights in the perceptions and performance of current text-based generative generate AI in the context of education. We surveyed educators and students at the authors’ institution and in five countries, namely Brazil, India, Japan, UK, and USA, regarding their perspectives on ChatGPT. There is a general consensus between educators and students that the use of ChatGPT in school work should be acknowledged, and that it will increase the competitiveness of students who are non-native English speakers. Additionally, there is a consensus among students that, in their future job, they will be able to outsource mundane tasks to ChatGPT, allowing them to focus on substantive and creative work. Interestingly, the majority of surveyed students expressed their intention to use ChatGPT despite ethical considerations. For example, students in India think the use of ChatGPT in homework is unethical and should not be allowed, while those in Brazil think it is ethical and should be allowed, yet in both countries, the vast majority of surveyed students (94%) indicated that they would use it to assist with their homework and assignments in the coming semester. The negative implications of this finding go beyond the university, as studies have demonstrated that academic misconduct in school is highly correlated to dishonesty and with similar decision-making patterns in the workplace43. Our results indicate that the majority of students plan to use ChatGPT to assist with their assignments, and believe their peers would approve of its use, implying that the use of ChatGPT is likely to emerge as a norm among students. Similarly, we find that the majority of professors plan to treat the use of ChatGPT as plagiarism tool, and believe their peers would approve of treating it as such, implying that the treatment of ChatGPT as plagiarism is likely to emerge as a norm among professors. The inherent conflict between these two could create an environment where students’ incentives suggest they should hide the use of ChatGPT while professors are eager to detect its use (and may expect their colleagues to do the same). This poses a major challenge for educational institutions to craft appropriate academic integrity policies related to generative AI broadly, and ChatGPT specifically.The NYUAD survey allowed us to examine whether there exists a descriptive norm among students backed by injunctive norms regarding the use of ChatGPT, and among professors with regards to treating ChatGPT’s use as plagiarism. More specifically, we investigate whether individuals prefer to conform to these behaviours on the condition that they expect the majority of their peers would also conform to them44, the degree to which these measures can be effectively enforced remains unknown. Treating the use of ChatGPT as plagiarism is likely to be the most common approach to counter its usage. However, as we have shown, current measures to detect its usage are futile. Other universities have opted to change their evaluative processes to combat the tool’s usage, by returning to in-person hand-written and oral assessments13—a step that may undo decades of advancements aimed at deploying alternatives to high stakes examinations that disadvantage some categories of students and minorities45. Our findings suggest that ChatGPT could disrupt higher education’s ability to verify the knowledge and understanding of students, and could potentially decrease the effectiveness of teacher feedback to students. In education, it is crucial to confirm that what is being assessed has been completed by the person who will receive the associated grades. These grades go on to influence access to scholarships, employment opportunities, and even salary ranges in some countries and industries46. Teachers use assessment to give feedback to learners, and higher education institutions use the assessment process to credential learners. The value of the degree granted is in part linked to the validity of the assessment process. Countermeasures to academic misconduct such as plagiarism include applying detection software, such as Turnitin, and emphasising honor codes in policy and cultural norms. Higher education, in particular, is susceptible to disruption by ChatGPT because it makes it very difficult to verify if students actually have the skills and competencies their grades reflect. It disrupts how learning is assessed because demonstrated learning outcomes may be inaccurate and thus, prolific use could diminish the academic reputation of a given higher education institution. Our findings shed light on the perceptions, performance, and detectibility of ChatGPT, which could guide the ongoing discussion on whether and how educational policies could be reformed in the presence of not only ChatGPT, but also its inevitably more sophisticated successors.When comparing ChatGPT’s performance to that of students across university-level courses, the results indicate a clear need to take “AI-plagiarism” seriously. Specifically, ChatGPT’s performance was comparable, if not superior, to students’ performance on 12 (38%) out of the 32 courses considered in our analysis. These findings, coupled with the clear ease with which students can avoid detection by current AI-text classifiers as demonstrated in our study, suggests that educational institutions are facing a serious threat with regards to current student evaluation frameworks. To date, it is unclear whether to integrate ChatGPT into academic and educational policies, and if so, how to enforce its usage. While several school districts, and even an entire country in the case of China, have already banned access to the tool50, including ChatGPT’s own successor, GPT 451. Hence, future work may examine the performance of newer and more complex models on courses from all levels of the educational ladder where such technology is likely to be utilized by students. Indeed, the vast majority of studies evaluating the performance of large language models have done so at a higher education level, with only a few doing so at a high-school level or lower52. Moreover, as such technology continues to evolve, so will the classifiers built to detect text, images, audio, and other forms of content created by generative artificial intelligence. Simultaneously, techniques used to evade such detection will also improve, thereby opening important research questions on the efficacy of such techniques, given their impact on regulating the use of large language models in educational contexts. While our study demonstrated potential issues that ChatGPT introduces in the process of student evaluation, it could also be potentially utilized as an assistive tool that can facilitate both teaching and learning55. For example, teachers could use large language models as a pedagogical tool to generate multiple examples and explanations when teaching55, and students could use them as a tutor that is available at their disposal to answer questions and provide explanations54. Hence, future work should examine both the issues such technology presents from the perspective of plagiarism as well as the opportunity this technology provides for students and educators in and outside of the classroom.Given the novelty of large language models such as ChatGPT, research on the impact of this technology on the education sector remains a potentially fruitful area of research. Future work may examine any temporal variation in sentiments towards such models as they become increasingly accessible across the globe. While our work has examined student and educator sentiments towards ChatGPT in the two months following its release, such sentiments may evolve over time, and hence, future work may evaluate any changes in these sentiments. Similarly, the utility and performance of large language models will also continue to improve over time. In the six months since the release of ChatGPT to the public, numerous other large language models have also been releasedAll faculty members at New York University Abu Dhabi (NYUAD) were invited to participate in this study and were asked to provide student assessments from their courses. In particular, they were asked to follow these steps: (i) select a course that they have taught at NYUAD; (ii) randomly select 10 text-based questions from labs, homework, assignments, quizzes, or exams in the course; (iii) randomly select three student submissions for each of the 10 questions from previous iterations of the course; (iv) convert any hand-written student submissions into digital form; (v) obtain students’ consent for the usage of their anonymized submissions in the study; and (vi) obtain students’ confirmation that they were 18 years of age or older.When choosing their questions, participating faculty were given the following guidelines: (i) Questions should not exceed 2 paragraphs (but the answer could be of any length); (ii) No multiple-choice questions were allowed; (iii) No images, sketches, or diagrams were allowed in the question nor the answer; (iv) No attachments are allowed in the question nor the answer ; (v) The question could include a table, and the answer could have a table to be filled; (vi) The question and the answer could include programming code; (vii) The question and the answer could include mathematics ; and (viii) Fill-the-gap questions were allowed.For each course, ChatGPT was used to generate three distinct answers to each of the 10 questions. More specifically, the question’s text was entered into ChatGPT verbatim, without any additional prompt or text. To generate unique responses, the “Regenerate response” button was clicked until a unique answer was generated. Any changes in the wording or phrasing in the answer was considered to be a unique response. This step was carried out between the 13th and 29th of January, 2023. Both students’ and ChatGPT’s answers were then compiled into a single document in random order, labelled as “Submission 1” to “Submission 6”. The faculty responsible for each course was asked to recruit three graders , and to provide the graders with rubrics and model solutions to each question. The participating faculty was instructed not to inform the graders that some of the submissions are AI-generated. Each grader was compensated for their time at a rate of $15 per hour. The participating faculty vetted the graders’ output and ensured that the grading is in line with the rubrics.34; (ii) Does the question or the answer involve mathematics? (Yes/No); (iii) Does the question or the answer involve snippets of computer code? (Yes/No); (iv) Does the question or the answer require knowledge of a specific author, scientific paper/book, or a particular technique/method? (Yes/No); and (v) Is the question a trick question? (Yes/No). To explain what a trick question is, the following description was provided to participating faculty: “A trick question is a question that is designed to be difficult to answer or understand, often with the intention of confusing or misleading the person being asked. Trick questions are often used in games and puzzles, and can be used to test a person’s knowledge or problem-solving skills”.For each of their 10 questions, the faculty were asked to answer the following: (i) Where does the question fall along the “knowledge” dimension and the “cognitive process” dimension of the Anderson and Krathwohl taxonomy60. The platform utilizes email and location verification for fraud detection and ID exclusions, thereby guarding against duplicate and bot-submitted responses. SurveyMonkey enables the targeting of samples with pre-specified demographics. Specifically, in the US, we identified educators as those whose job is “educator ,” and identified students as those who had a “student status.” As for the remaining four countries, the above filters were not available on SurveyMonkey. Therefore, we recruited 100 respondents whose primary role is “Faculty/Teaching Staff” and recruited 200 respondents whose occupation was “studies.” In addition, we ensured that the analytical sample only contains current students and educators who are either teachers or professors based on respondents’ self-reports. Respondents filled out the survey in the official language of their country. The global surveys were conducted on the 19th and 20th of January, 2023.Participants were recruited from Brazil, India, Japan, United Kingdom, and United States, with a minimum of 200 students and 100 educators per country; see Supplementary Table 61 related to the use of ChatGPT among students. As for faculty, we asked them about the number of years they have been teaching, their academic division, their appointment type , as well as their empirical and normative expectations surrounding the treatment of ChatGPT as plagiarism. We excluded student participants who indicated that they aware of our study, as well as faculty members who coauthored the paper. Descriptive statistics can be found in Supplementary Tables As for the NYUAD survey, it includes responses from 151 students  and 60 professors  collected between January 23rd and February 3rd, 2023. This survey used the same questions as those used in the global survey, but with a few additions. Specifically, for students, we asked them to specify their current cumulative GPA, year of study, as well as empirical and normative expectationsGPTZero35, and OpenAI’s own AI text classifier36. The former classifies text into five categories: (i) “likely to be written entirely by a human”, (ii) “most likely human written but there are some sentences with low perplexities”, (iii) “more information is needed”, (iv)“includes parts written by AI”, (v) “likely to be written entirely by AI”. As for the latter, it classifies text into five categories: (i) very unlikely, (ii) unlikely, (iii) unclear if it is, (iv) possibly, or (v) likely AI-generated. We start our analysis by examining the proportion of human submissions that are misclassified as ChatGPT, as well as the proportion of ChatGPT submissions that are misclassified as human. More formally, our goal is to construct a normalized confusion matrix for each classifier, to quantify its false-positive and false-negative rates. For both classifiers, ChatGPT submissions that are classified as categories (i) or (ii) are treated as false negatives, while human submissions that are classified as (iv) or (v) are treated as false positives. As for submissions classified as (iii), we consider them to be classified as human. This decision is motivated by our context, where the classifier is used to detect AI-plagiarism. Given that a student is arguably “innocent until proven otherwise”, having an inconclusive classification means that, as far as the teacher is concerned, there is no conclusive evidence incriminating the student of AI-plagiarism, implying that such a classification has practically the same consequences as being classified as human. Based on this, ChatGPT submissions that are classified as (iii) are treated as false negatives.Once all student and ChatGPT submissions were collected for all courses, they were used to test two classifiers designed specifically to determine whether a body of text has been generated using AI, namely, To evaluate the classifiers’ susceptibility to obfuscation attacks, we devised a simple attack that involves running the ChatGPT-generated text through Quillbot. Our attack starts by setting the “Synonym” slider in Quillbot to the highest level, which maximizes the number of modifications introduced to the text. Then, we run every ChatGPT-generated submission through each of the “Modes” available on Quillbot, each resulting in a different output. Each output is then re-analyzed on both classifiers, and if any of them manages to flip the text classification from category (iv) or (v) to any other category, the attack on this particular submission is considered successful.The research was approved by the Institutional Review Board of New York University Abu Dhabi (HRPP-2023-5). All research was performed in accordance with relevant guidelines and regulations. Informed consent was obtained from all participants in every segment of this study.Supplementary Information.


The possibility of mental illness caused by the academic emotions and academic pressure of graduate students has received widespread attention. Discovering hidden academic emotions by mining graduate students’ speeches in social networks has strong practical significance for the mental state discovery of graduate students.Through data collected from online academic forum, a text based BiGRU-Attention model was conducted to achieve academic emotion recognition and classification, and a keyword statistics and topic analysis was performed for topic discussion among graduate posts.Female graduate students post more than male students, and graduates majoring in chemistry post the most. Using the BiGRU-Attention model to identify and classify academic emotions has a performance with precision, recall and F1 score of more than 95%, the category of PA (Positive Activating) has the best classification performance. Through the analysis of post topics and keywords, the academic emotions of graduates mainly come from academic pressure, interpersonal relationships and career related.A BiGRU-Attention model based on deep learning method is proposed to combine classical academic emotion classification and categories to achieve a text academic emotion recognition method based on user generated content. In the pursing of a master or doctoral degree, graduate students have been actively carrying out scientific research achievements. They take advantage of their intelligence and make the best use of higher quality scientific and technological resources. At the same time, they are also facing great pressure . GraduatWhat is the current state and distribution of graduate students who publish information on social platforms?Based on these students’ posts, how can we discover the hidden academic emotions?What are these graduate students really talking about online?However, emotions are not easy to identify sometime. We cannot just ask some other graduate students with serial rude and personal questions such as “Are you happy doing your research today?” or “Why are you so angry about your teammates?” . So, if To address these issues, we conducted the following research. We performed our research on real Internet user generate contents. First, we conduct a descriptive statistical analysis on the collected data, to find out the distribution of posters’ information. Second, we transformed the task of discovering academic emotions from posters’ topics and replies into a supervised deep learning based classification model with natural language processing methods to recognize and classified the content into several emotion categories. At last, a semantic based topic evolution analysis has been conducted to find out what are the posters really discussing about specifically.2.2.1.Academic emotion recognition can be considered as the process of discovering hidden emotions from data and research in the studying process of students. Recent studies have been conducted primarily on specific groups or by incorporating specific environmental factors. Meanwhile, the research on academic emotion recognition from artificial intelligence approach mainly focuses on the practical exploration of emotion recognition in the fields of natural language processing , compute2.2.Sentiment classification and analysis are important research front in the field of natural language processing, and emotion analysis is considered an extension and complement . SentimeBoth unsupervised and supervised approaches are used in research of emotion detection. Unsupervised approaches mainly including dictionary-based and rule-based methods. Research on academic emotions has been widely carried out, and many questionnaires are used for different research subjects. However, these methods are not suitable for academic emotion recognition process based on large-scale datasets. Meanwhile, the achievement of text-based emotion analysis, we can easily use these methods and techniques for academic emotion recognition research.3.3.1.The dataset used in our research needs to meet two main requirements. First, to achieve opinion mining, it should be user generated content; Second, it should be posted by graduate students. We choose to collect the posts and comments from “Xiaomuchong”We collected the web page text content of the Xiaomuchong website through a web spider program written in python for 2 weeks, and the data set contains three types of useful information: topic information, detail information and poster information. The topic information is the dataset of topics posted by the website users. It contains the title of the topic, the post time, the latest reply time, the id of the topic poster and the volume of views and replies. The detail information contains the whole detail content of the dataset. Each reply to every topic in our dataset is collected. It contains the replied detail text content and the replier’s information such as id and reply time. The poster information contains the personal information of both posters and repliers, such as age, gender and what they are majoring in. Finally, we collected 18,831 topic items, 511,269 detail items and 131,755 poster information items and stored in MongoDB.3.2.3.2.1.We used Bi-GRU, a deep learning model, for training and testing to achieve the task of identifying academic emotions. The pipeline of the entire model includes three main steps, which is shown in The first step is data pre-processing. For Chinese text, before the word vector embedding task, it is necessary to perform a word segmentation on the text. The second step is data labeling, it is essential in supervised learning model training. In the word embedding process, due to the necessity to train and test on different models and compare the results, we used a pretrained word2vec model instead of randomly initialized word vectors. We selected a Chinese word vector library developed by Tencent Artificial Intelligence Laboratory . This li3.2.2.We labeled the dataset into four categories with academic emotion tags among “Positive activating,” “Positive deactivating,” “Negative activating” and “Negative deactivating.” Due to the large amount of data, perform manual labeling method on global data set is not appropriate. To ensure the efficiency and accuracy of data labeling result, we divide the process into two steps. The first round of labeling process is unsupervised labeling, and the second round is conducted by combining unsupervised and manual labeling. Unsupervised labeling is to determine whether the target text contains emotional words corresponding to the dimension. We first constructed an emotional word list based on academic emotion categories  and an e今天做完了实验，真开心, 但导师却挑剔的说结果不够好 ,” there are two emotional word <开心 (happy), 挑剔 > in this text which belongs two emotional word categories < happy, derogatory > and two academic emotion dimensions < Positive activating, Negative deactivating >. According to the emotional ontology, emotional strength of word “happy” is 7 and “critical” is 5, thus the academic emotion dimension strength of this text can be calculated as < Positive activating:7, Positive deactivating: 0, Negative activating: 0, Negative deactivating:5 >, 7 is the max strength value, so this text will be labeled as “Positive activating.”Three conditions occurred during the process. First, there is only one emotional word in the text, which can be directly labeled with the corresponding dimension. Second, the text does not contain any emotional words, and it will be marked as invalid data. The third condition is the most complicated. If the text contains multiple emotional words, a method for calculating academic emotional strength was conducted to present the emotional tendency. For an unlabeled text, the academic emotional strength of one dimension can be calculated as the sum of the emotional strength of all emotional words belongs to this dimension, and the text can be labeled as the academic emotion dimension with the max emotional strength. For example, in an unlabeled text “However, this unsupervised data labeling method is suitable for conditions where the text length should not be too long. If there are more than 10 emotional words in one unlabeled text, then it can be classified as long text . We manu3.2.3.zt and a reset gate rt. At time t, according to the output of previous stateℎ ht−1 and current input xt, zt and rt can be formulated as:From natural language processing aspect, the task of identifying academic sentiment from textual content can be considered as feature based text sequence classification, so among deep learning models and frameworks, a recurrent neural network (RNN) that can extract and process sequence features is conducted. GRU (Gated Recurrent Units) and LSTM (Long-short Term Memory) are both optimized recurrent neural network models , which aℎt−1 × xt), then combine the new data with the current input xt, a gated recurrent unit calculates the new state ht as:After inputting the previous gated state, a reset gate can calculate an updated  classification with the best effect has a score of more than 98% in all three indicators. This category has the largest sample size and a larger range of emotional words, which can have better recognition effect. The worst result of the four categories is PD (Positive Deactivating), with a F1 score of 93% and a recall 89%. Just focus on the score of this one category, the performance of the model is good enough, but there is a gap of close to 5 to 10% with the scores of the other three categories. This phenomenon occurs because this category PD (Positive Deactivating) is one of the academic emotions which is most difficult to articulate or to define. According to In general, in such a high-dimensional multi-classification natural language processing task, using a deep learning model based on a recurrent neural network combined with an attention mechanism to build a classification model can achieve good performance with more than 90% of the comprehensive scores. The performance indicators can support some Chinese text-based academic emotion recognition applications based on such a model.4.4.Discovering hidden academic emotions from posters can effectively help us deal with the academic stress of graduate students, but it is not enough to know that what are they really talking about or worrying about. To have a further understanding of current graduate students using online platforms, it is necessary to bring up this discussion to semantic and topic level.In addition to identifying the hidden graduate academic emotions from the website’s posts, to truly understand the specific topics that the graduates are discussing, we have carried out more detailed discussions and analysis. After the text pre-processing operation, we not only construct the recognition model of the obtained corpus, but also conduct a simple topic analysis on the collected topics. Through word frequency statistics, we excluded keywords with term frequency less than 1,000, finally, we can know high-frequency words as shown in We continue to use the Chinese word vectors obtained in the word vector embedding step and conducted a k-means algorithm to cluster the words in each topic and post, determined 4.4.1.Academic pressure is a major aspect of postgraduate students expressing academic emotions. The academic emotional pressure of postgraduates is not only good or bad academic exam score performance, but also includes the pressure of scientific research achievements, such as experiments and papers. The high-frequency words related to the academic pressure of graduate students include “papers,” “projects,” “laboratories,” and “scientific research,” etc. Academic pressure, that is, the dual pressure from study and scientific research, is the topic that postgraduates are most concerned about, and may also be the major source of postgraduate pressure. Accompanied by such academic pressure, psychological problems or practical problems may occur among graduate students. Psychological problems can lead to mental illness, such as depression, obsessive–compulsive disorder, anxiety disorder, and even lead to schizophrenia, etc. . Suicida4.4.2.Another important emotion topic comes from lack of social interaction. The high-intensity research and study of graduate students makes their life trajectory very simple. The daily life of most graduate students during the semester is to commute among the laboratory or classroom, dormitory and canteen. They do not have the opportunity, nor the extra energy, to engage in social activities. This situation is more prominent among Chinese students . The hig4.4.3.Finally, one of the most frequently discussed topics online for graduate students is related to personal career development. These topics are like the WFC mentioned in the previous section, but more about the concerns of the graduate students themselves about their personal career development. The high-frequency words related to the career pressure of graduate students include “work,” “career,” “job,” and “income,” etc. These pressures and emotions arise from comparison. We all know that academic qualifications and degrees can determine the type and starting point of your future work, but in actual working environment, personal career development is often influenced by a variety of factors, among which working years is a very important consideration . This br5.Measuring academic emotions is an important way to discover graduate students’ learning status and mental health. Due to the concealment and diversity of academic emotions, it is difficult to discover hidden them from texts using traditional methods. The academic emotions and academic pressures of graduate students is a long-standing concern and is receiving increasing attention. The peculiarity of graduates’ academic emotions is that their stress sources are not only from their studies, but also from research, family, and career planning. At the same time, with these academic pressures, there is no effective way to express and talk about these academic emotions that graduate students generate. In the long run, it is very easy to form psychological problems and lead to serious consequences. Many studies have analyzed and excavated academic emotion.To address the three research questions we raised, we conducted a series of academic emotion recognition and analysis methods on large-scale datasets. For our first research question, our research conducted a statistic analysis of the collected postgraduates’ posts on Xiaomuchong platform, mainly gender and major, and find out excludes users who do not want to disclose their gender or who do not fill in their gender, female post more on the platform. According to the majors marked by posters, we found that graduate students in science and engineering published most of the posts on the platform, especially majoring in chemistry. This is determined by the features of the major and the way in which the research work undertaken is carried out. For our second research question, we transform the academic emotion recognition task into a series process of constructing, training, and testing an emotion classification model based on user-generated text content. Aiming at the shortcomings of traditional academic emotion recognition research in the application of large-scale data sets, we constructed a pipeline based on recurrent neural network, which can identify and classify academic emotions unsupervised, and has a relatively ideal model performance. At last, for our third research question, based on the word vectors, we performed a topic analysis among the graduate students’ posts. We clustered graduate posts on the Xiaomuchong platform into three main categories: academic pressure related topics, interpersonal related topics, and career related topics. We also discussed the main problems and sources of stress faced by graduate students from these three main categories. There are also deficiencies in our research that need to be improved in future research. The first is the problem of data labeling. The use of vocabulary-based heuristic rules may be insufficient. Consider using a decision tree model instead. Second, the topic of posts is not necessarily a simple emotional expression, but also a relevant topic discussion. The information organization provided only through the website may be insufficient. Consider a better way to filter the data set.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.QX: research design and paper writing. SC: data collection and data analysis. YX: paper revision and editing. CM: paper writing. All authors contributed to the article and approved the submitted version.This work is supported by the National Natural Science Foundation of China (Nos. 72104219 and 62207016), the MOE Project of Humanities and Social Sciences (No. 21YJC870013), and Major Humanities and Social Sciences Research Projects in Zhejiang higher education institutions (No. 2023QN129).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This discourse analytical study explores how health professionals (HPs) construct burnout as a form of mental distress in the context of Finnish burnout rehabilitation framed with a particular rehabilitation ethos. Burnout is a fuzzy concept and lacks a disease status. Therefore, it calls for context-specific definition and justification. By highlighting the socially and interactionally produced character of categories of mental distress, the study investigates the kinds of discourses HPs use to formulate “the problem” and its solutions, and how people dealing with burnout are categorized in these discourses. The data consists of field notes from the observation of group discussion sessions in two 1-year burnout rehabilitation courses. As a result of the analysis, five partly overlapping discourses were identified: psychological, evolutionary, healthy lifestyle, biomedical, and welfare. Within these discourses, people who experience burnout were categorized as over-conscientious employees, “good girls,” “primitive people,” self-responsible rehabilitees, patients, and (aging) employees with social and legal rights. Burnout rehabilitation and HPs’ views reproduce a cultural and clinical discourse around burnout in which work-related problems are treated as individual-level problems and individuals are responsibilized for the management of mental distress. Based on the results, it is concluded that the hybrid type of interventions that attempt to influence both individual- and work-related problems behind burnout would help to prevent people dealing with burnout from being over-responsibilized for solving problems at the workplace. This discourse analytic study is concerned with exploring how health professionals (HPs) construct burnout as a form of mental distress in the institutional setting of rehabilitation. Drawing on the field notes from the observation of group discussions in two Finnish group-based burnout rehabilitation courses, we analyse the kinds of discourses with certain categorizations of people dealing with burnout that are constructed by HPs. In previous discourse analytical (DA) research on mental distress, the focus of the analysis has often been on the ways by which individual patients react to receiving a diagnosis and construct “diagnostic identities” in light of such a diagnosis . This stThe starting point of this study is, first, that the way mental distress is discursively constructed in institutional contexts is crucial to the positions that become available to clients—and thereby the type of support they may encounter : 162. Ho . The study is part of a larger follow-up research project, which was conducted in 2016–2018. Both courses, funded by the Social Insurance Institution of Finland (Kela), lasted about 1 year and included three 5-day group-based rehabilitation periods at the rehabilitation center. The rehabilitation was targeted at employees whose working ability was already jeopardized by symptoms of burnout (vs preventive). Its official aim was to enhance and maintain the participants’ well-being at work and acknowledge the significance of a healthy lifestyle and the importance of active self-responsibility for recovery.The setting of the study consists of two Finnish group-based burnout rehabilitation coursesEach 5-day rehabilitation period in both courses included a total of 22–24 hours of group discussion sessions guided by different HPs see . The invBoth the rehabilitation participants and HPs involved in the study were recruited in cooperation with the rehabilitation center and the responsible actors for the chosen courses (a psychologist and two physiotherapists). Research permission was obtained from other HPs with the help of these key actors, who also forwarded the research information letter to the clients prior to the beginning of the rehabilitation. In the information letter, participants were briefed on the aims and venue of the study, voluntary participation, their right to withdraw from the study at any stage, data usage, processing and archiving, and confidentiality. Moreover, the letter highlighted that participation in the study or refusal to participate in it did not affect the implementation of the rehabilitation program. Ethical approval for the research was given by the University of Eastern Finland Committee on Research Ethics.The data used in this study—field notes gathered by means of observations of group discussion sessions on both rehabilitation courses—is drawn from data from a larger project, which also consists of follow-up interviews with rehabilitation participants. Altogether 19 people (18 women and one man) participated in two courses. All of them granted permission for the observation of the group discussions, and 15 (14 women and one man) participated in follow-up interviews. They worked in the fields of health care and social work, technology, education, media, and the service sector. Most of the participants were highly educated. They had had a relatively long and stable career and they perceived themselves as qualified employees. Subjectively reported in the interviews, heavy workloads, long hours, a pressured work environment and poor management had determined the participants’ situation.The participants were 34–62 years old (at the beginning of rehabilitation) and represented various occupations.The observation data comprises a total of 324 pages (A4) of hand-written field notes, some of which were later transferred to a computer for analytical purposes. The observation was conducted by the first author. During the group sessions, the researcher avoided any comments in the discussions, as was agreed in advance with the rehabilitation staff. The researcher’s non-involvement in discussion made it easier for her to hear different voices and minimize the effect she had on the setting and the people being studied. Because it was impossible to collect data on everything that was said and that was going on, the researcher needed, by necessity, to be selective in her writing . It shou She also made notes with a more reflective or analytical tone. The researcher had previously worked as a psychologist on similar rehabilitation courses. Her prior experience partly guided (and possibly restricted) the ways she viewed, understood, and mapped out talk by HPs  habits, 2) personality traits, 3) things beyond your control, circumstances.” Refers to studies: “The typical profile of someone with burnout is a kind woman with difficulties drawing boundaries, saying no, standing up for her rights.” Instructs every rehabilitee to draw “the crossroads of their own life” on paper. The psychological discourse occurred particularly in one group discussion in which the HP introduced a popular psychology book entitled After instructing the participants to consider the crossroads of their lives, the HP started to speak about “traits” and “personality” that are characteristic of each “path,” thereby placing the roots of burnout within the individual. HP gave a short list of personality characteristics, such as “niceness” and “inability to set boundaries,” “softness,” and “consuming professional ambition,” which were construed as problematic from a well-being point of view. Thus, the idea of burnout as a crossroads worked to construct a view concerning an individual-level problem and acted as an invitation for the participants to observe and reflect on their psychological characteristics, in particular.over-conscientious employees whose certain psychological characteristics had made them vulnerable to burnout. Moreover, the psychological discourse produced a gendered categorization of burnout women as “good girls” who have a tendency to take excessive responsibility and care for other people and sacrifice their own needs. Through this categorization, many of the participants’ feminine-assumed psychological characteristics and behavioral patterns appeared deviant and, therefore, in need of change. Accordingly, recovery from burnout was seen to require learning to become “merciful” to oneself and learning to become more “assertive” because circumstances at work “cannot be influenced”:HP notes that “burnout leads to depression” and suggests that all of the rehabilitees “should give yourselves time to heal, you have to be merciful.” Speaks about “learning assertiveness.” Says that assertiveness “is being firm, and sometimes you have to upset someone else, when you set boundaries.” Continues that “there’s nothing wrong with being nice, but martyrdom, self-sacrifice is its opposite.” HP says that “since you can’t influence your work, it’s important to learn how to identify the things that you can affect with your thinking.” Continues that “taking care of yourself is the greatest form of selflessness” and that ”it’s important to set boundaries in life in general”. Within this discourse, people dealing with burnout were categorized as causes of burnout: stress was seen as a result of a discordance between humans’ ancient, genetically determined biology and the activity patterns of contemporary people  as well as undesired intra-psychological outcomes. Thus, the HP constructed the nature of the illness to be fought against:HP speaks of “resting the body” and “resting the brain.” Asks the group “What nutrient does the brain need to function?” Goes on to say that these are complemented by sleep and speaks of the “brain maintenance perspective.” Continues by saying that “humans are a day species” and that “daily routines are important from an evolutionary perspective.” Later also speaks of naps, meditation, and memory, referring to the evolution of the “human species,” for example the significance of memory to the “survival of the species.” Continues about the importance of exercise by saying: “It would be natural for humans to walk or run 20 km a day.” The occupational therapist also drew on all three discourses while he encouraged rehabilitees to “take care of their brains” through healthy life habits and referred to an evolutionary perspective:HP: “Modern research has shown that just 10–15 minutes of light exercise has an effect comparable to antidepressants – you can tell yourself that you’re setting out to rehabilitate your brain.” Talks about the activation of the sympathetic nervous system and the fight or escape reaction to stressful situations. Provides instruction in relaxing the body and introduces mental self-care methods: “This is number one,” (writes the word “exercise” on the flip chart). Continues by talking about the effects of exercise on mood. Later lists other means of mental self-care, such as “rest,” “meaningful activities,” “taking a moment” and “idleness” (refers to “positive idleness” and stresses that “passivity is not empowering”), “relationships” , “nourishment” and “nature experiences.” In another group discussion, the psychotherapist highlighted the significance of physical exercise, rest, and relaxation for recovery by utilizing the biomedical and healthy lifestyle discourses. The psychotherapist talked about the “fight or flight-reaction” thereby referring to popular stress theories that regard the fight-or-flight response as our ancient response to stressful situations. By relying on the evolutionary discourse, HP suggested that social interaction with other people is a “species-specific need” for human beings:Moreover, HP pointed out the difference between the right kind of physical activity and “being passive” and construed an ideal of active self-management of mental health and performance, in particular, by emphasizing “rehabilitation of the brain.” Thus, the biomedical and healthy lifestyle discourses resonated with the neuro-discourse and related idea of self-optimization, which has become prevalent in workplace health interventions e.g. .In the following example, the psychotherapist utilizes the healthy lifestyle discourse and suggests the learning of rest and relaxation skills for the management of work stress. She denies the possibility of being able to affect the working conditions. The example illustrates that within the healthy lifestyle discourse, people who experience burnout were responsibilised for the control and management of their minds and bodies in the face of work-related difficulties.Laura  says that she went back to work for a week after her holiday and before this rehabilitation period. She says: “It was anything but smooth sailing!” Laura ironically recounts a newspaper’s tips on an easy start for work after the holidays. HP: “There’s nothing you can do about your work, but you can keep the feeling of not having enough time under control, so that it doesn’t get under your skin. That’s when it consumes your strength.” Talks about how “important it is to be aware whether you are giving yourself the opportunity for sufficient rest.” Later stresses the fact that “work doesn’t change.” Explains that there have been changes in the ways sick leave due to incapacity for work are addressed in the workplace, and this results from new legislation on health insurance and occupational health care. “When the employee’s illness is prolonged, the remaining work capacity must be assessed by an occupational health doctor. In addition, the employer, the employee, and the occupational health service work together to determine the employee’s chances of continuing to work.” The group discussion held by a social worker was the only session in which recovery from burnout was approached from the point of view of work arrangements and the related solutions that could support the individual’s working ability. A social worker gave a lecture about social benefits, rehabilitation options, social security allowances and different statutory options for lightening workloads, such as applying for a part-time pension, disability pension, alternate leave, and part-time work. The excerpt below comes from part of that discussion:welfare discourse, rehabilitees were categorized, first, as employees with social and legal rights who are entitled to collective, system- and organization-level support for their ability to work and, second, as employees with reduced working ability. Moreover, occupational health care and employers were responsibilized for finding solutions that support the employee’s working ability. In the welfare discourse, the participants were addressed especially as aging employees. This can be explained by the fact that in the Finnish social insurance system access to many social benefits to maintain working ability requires certain criteria related to the number of working years carried out and the age of the employee.Within the Although our focus in this study is on the ways HPs as institutional authorities constructed burnout and recovery, we also take a brief look at the rehabilitation participants’ own meaning making, within which they both adopted and challenged the discourses and categorizations created by the HPs.life-historical discourse, participants constructed causal explanations for their burnout. They presented themselves as hardworking and conscientious employees, categorizing themselves as lifelong good employees. They stated things such as, “Work used to be my love, my friend, I devoted myself to work,” or, “I always worked long hours, diligent by nature, did what I was told,” .The first group session of both rehabilitation courses focused on sharing personal burnout narratives. Within a The categorization of rehabilitees as lifelong good employees occurred also in the group sessions that dealt with personal work histories. Almost everyone emphasized hard work since childhood and the centrality of work in their lives: “All my life I have worked long hours,” and, “I was born in 1982, and that’s when I started to work,” .Although in both group sessions the participants perceived themselves as people for whom work had been the foundation of their identity, in the face of burnout their lifelong commitment to work became regarded as an individual flaw. In other words, the participants located the causes of burnout in themselves by characterizing themselves as over-conscientious “workaholics,” for example, and by presenting an ironic critique of the qualities of a good employee perceived in themselves. In this respect one of the participants stated: “It must be a mental disability then, stretching it out [work],” . Thus, in the same vein as the HPs, the rehabilitees portrayed themselves as over-conscientious employees.Everyone’s work history has been discussed in turn. HP continues by asking: “Can you recognize some characteristic, something to improve on, that you ought to do something about?” One of the rehabilitees mentions “saying no” and “drawing boundaries” and another says that they are “too nice, a doormat” at the workplace. HP: “This all began with learning to say no, you remember.” Whereas the HPs’ discursive practices emphasized individual-related causes of burnout, in their personal burnout stories and work histories the rehabilitees brought up adversities at work that they had faced at different times. They pointed out long-lasting and cumulative negative processes at work, such as increased workloads and poor working arrangements. They said: “No real breaks in my work” or, “Every year, we’ve had redundancy negotiations, people let go,” and, “Working life went off kilter after the depression” . However, the response from the HPs framed the discussion toward prioritizing individual attitudes and behaviors whilst downplaying the participants’ work-related concerns. For instance, one HP started to speak about the ability to “say no” in the face of difficult work situations and emphasized that solving problems at work was not a part of the aims of rehabilitation: “We may not be able to fully solve these problems here. Maybe it’s the person drawing the boundaries” . Instead, rehabilitation was intended to identify those “traits” in oneself that needed to be changed:In this article we have analyzed how burnout as a problem is discursively constructed, how recovery is viewed and how people dealing with burnout are categorized by HPs in the institutional context of burnout rehabilitation. The study shows that the discursive practices of rehabilitation reproduce the individual-centered discourse around burnout and there is a paucity of working life perspectives. Psychological, evolutionary, biomedical, and healthy lifestyle discourses encourage people who experience burnout to engage in an array of mind- and body-focused practices of self-care as means to work toward better management of mental distress.the self, in particular that was subjected to “change work” healthy subjects; what are the lived effects of specific problematizations of burnout on people’s lives) and HPs . Finally, further research is needed to analyse how HPs perceive men’s burnout and whether gendered categories related to masculinity and burnout are constructed.per se (cf. Although the understanding of burnout in the context of Finnish burnout rehabilitation is individual-centered, it is not neoliberal r se cf. . Rather,r se cf. . Accordir se cf. : 414, “tr se cf. : 17 put can address them when there is a lack of both resources and approaches to try to influence clients’ actual work environments.We suggest that HPs employ discourses which emphasize self-care to address individual responsibility in the name of patient-centredness and empowerment  – the vaAn approach that emphasizes burned out people as self-responsible managers of their mental distress may indeed appear at first glance to be respectful of individual agency. In previous research on Finnish burnout rehabilitation individuals’ enhanced sense of self-responsibility has also been linked to the strengthening of personal agency as a desirable outcome of rehabilitation . As BechWe argue, however, that the major problem of individual-centered discourses around burnout is that they ignore the problems of working life, everyday stressors at work as well as social power structures and hierarchies behind the burnout syndrome. Moreover, there is research evidence suggesting that compared to person-directed interventions, hybrid type interventions with a combination of both person- and organization-directed approaches have longer-lasting positive effects in terms of alleviating burnout symptoms .Based on this study, we argue that hybrid type interventions that attempt to influence both individual- and work-related problems behind burnout would also help to prevent people dealing with burnout from being over-responsibilized for solving problems at the workplace. However, the implementation of such interventions requires financial resources and is, therefore, not just in the hands of HPs but a matter of political priorities and decisions.


The Global Research on the Impact of Dermatological Diseases (GRIDD) team is developing the new Patient-Reported Impact of Dermatological Diseases (PRIDD) measure of the impact of dermatological conditions on the patient’s life, in partnership with patients. To develop PRIDD, we conducted a systematic review, followed by a qualitative interview study with 68 patients worldwide and subsequently a global Delphi survey of 1,154 patients to ensure PRIDD items were meaningful and important to patients.To pilot test PRIDD with patients with dermatological conditions, focusing on its content validity , acceptability, and feasibility.We conducted a theory-led qualitative study using the Three-Step Test-Interview method of cognitive interviewing. Three rounds of semi-structured interviews were conducted online. Adults (≥ 18 years) living with a dermatological condition and who spoke English sufficiently to take part in the interview were recruited through the International Alliance of Dermatology Patient Organizations’  global membership network. The topic guide met the gold-standard COSMIN  standards for cognitive interviewing. Analysis followed the thematic analytical model of cognitive interviewing.Twelve people  representing six dermatological conditions from four countries participated. Overall, patients found PRIDD to be comprehensible, comprehensive, relevant, acceptable, and feasible. Participants were able to discern the conceptual framework domains from the items. Feedback resulted in: the recall period being extended from 1 week to 1 month; removal of the ‘not relevant’ response option; and changes to the instructions and item ordering and wording to improve clarity and increase respondents’ confidence in their ability to respond. These evidence-based adjustments resulted in a 26-item version of PRIDD.This study met the gold-standard COSMIN criteria for the pilot testing of health measurement instruments. The data triangulated our previous findings, in particular the conceptual framework of impact. Our findings illuminate how patients understand and respond to PRIDD and other patient-reported measurement instruments. The results of comprehensibility, comprehensiveness, relevance, acceptability, and feasibility of PRIDD provide evidence of content validity from the target population. The next step in the development and validation of PRIDD is psychometric testing. The The Global Burden of Disease (GBD) project , 11 is tThe Global Research on the Impact of Dermatological Diseases (GRIDD) project, the first patient-initiated and led impact research project in dermatology, is collecting global data on the impact of living with dermatological conditions. These data will support local, national, and international advocacy work to prioritize dermatological conditions more accurately in the global health debate.To address GRIDD’s aim, a comprehensive measure of the impact of dermatological conditions on the patient’s life is required. Our systematic review  evaluateWe are developing the Patient-Reported Impact of Dermatological Diseases (PRIDD) measure in close collaboration with patients and according to best practices in PROM development  18, 19,–34. PRIDContent validity, “the degree to which the content of an instrument is an adequate reflection of the construct to be measured” , is consFollowing best practice, the next phase in PRIDD development is to pilot test the measure with dermatology patients and make any necessary adjustments , 39. TheCognitive interviewing is a pilot testing method that use a semi-structured topic guide to direct the interview according to Tourangeau’s four-stage model of question response , 41 to oThe aim of the current study was to pilot-test PRIDD by qualitatively exploring whether the measure  is comprehensive, comprehensible, relevant, feasible, and acceptable to people with dermatological conditions through cognitive interviews.2.2.1.We conducted a theory-led, qualitative study using the TSTI method of cognitive interviewing to pilot test PRIDD. This study was tested against the latest COSMIN guidance on the pilot testing of PROMs , 33, 45 2.2.Participants met the inclusion criteria if they were an adult (≥18 years) with a dermatological condition from anywhere in the world and spoke English sufficiently to take part in the interview and complete PRIDD independently (without a translator). Those who required translation to complete PRIDD were excluded as construct equivalence, the assumption that items in the translation measure the same construct in the same way as in the original language –49, coulParticipants were drawn from PRIDD’s target population via the International Alliance of Dermatology Patient Organizations’  global membership network using purposive sampling to achieve maximum variation according to dermatological condition and demographic factors: age, gender, and country of residence. Participants were directed to a secure online platform which included the participant information sheet (PIS), electronic consent form, and interview booking information. Twelve patient organizations were invited to recruit to the interviews; 8 (66.7%) agreed to participate. Reasons for non-participation included lack of staff capacity, scheduling conflicts, and non-response. Recruitment ceased at the point of data saturation; when there was sufficient evidence that most problems had been detected and/or resolved , 50.2.3.We tested the first draft of PRIDD , a 27-itA topic guide was developed detailing the interview procedures, instructions, and questions . Two ver2.4.Twelve interviews were conducted via Zoom video conferencing software across three rounds from 2 August to 1 September 2021 with one of four researchers , at a mutually convenient time. All interviewers were trained in cognitive interviewing by the study co-investigator (CB) and had backgrounds in healthcare practice and/or research.Interviews were approximately 1 h long and followed the three steps of the TSTI method see . They we2.5.Data collection and analysis were interrelated and concurrent, with analysis beginning after the completion of the first interview. Accordingly, generated themes and edits made to PRIDD were incorporated into subsequent interviews.Quantitative data were uploaded to SPSS version 26 and sample characteristics were summarized for clinical and demographic variables. Qualitative data were exported to NVivo 12 qualitative data software package. RP independently analyzed the data. NTS reviewed the coding and results reporting for accuracy. Analysis followed the thematic analytical model of cognitive interviewing  see Sup. This pr3.Eighteen people completed the online consent form and demographics questionnaire. Of these, three people were excluded because they were not sufficiently proficient in English to complete PRIDD independently and three did not respond to invitations to schedule an interview. In total, 12 people (response rate = 67%) across six dermatological conditions  and four3.1.Participants praised the comprehensiveness, comprehensibility, and relevance of PRIDD to their lived experiences:I’ve completed a lot of dermatological questionnaires, but I don’t think I’ve ever seen them all integrated like this in such a questionnaire … I’m very, very happy with this. It has stirred my heart … There are things here that I wanted to discuss with my dermatologist. PID5, Patient with hidradenitis suppurativa, IrelandThey were short questions, and they were quite easily answered. PID15, Patient with alopecia, UKPRIDD also appeared to be acceptable and feasible for patients. The average time taken to complete the questionnaire was 4.11 min .The questions are so concise. You can quickly fill that in, in the waiting room. PID12, Patient with extensive linear porokeratosis, IrelandWhile no participants found any of the items offensive or objectionable, they felt that others might be “uncomfortable”  with item 26 (“it has been difficult to be intimate with a partner”) as it referred to intimacy, but stressed it was important to include. Instead, participants felt that completing PRIDD initiated a process of reflection on their experiences with their condition:I wasn’t offended by any of them [items] … It actually made me aware of how much this is actually controlling my life again. PID11, Patient with discoid lupus, IrelandA minority of participants, most with alopecia, questioned the focus on the negative impacts of dermatological conditions and felt that positive impacts should be included too.I think it's sometimes nice to balance the negatives out with positives … in the past week, I've felt a lot of empowerment, I've felt a lot of like confidence, I've felt a lot of people praising me for something I've tried to hide away for so long, so it's not just negatives that you could capture as well, having a separate question saying I've felt confident, or I've felt empowered or something. PID15, Patient with alopecia, UKSome participants wanted to further elaborate on items with qualitative data.You could even go deeper than that … you could even have … a box to maybe put is there anything you'd like to add … that [you] feel is relevant … because everybody isn't the same PID16, Patient with Pityriasis Rubra Pilaris (PRP), UK3.2.Overall, the instructions appeared easy to comprehend as participants were able to summarize them accurately and succinctly.It was asking me to answer the below questions based on my condition, how it's affected me in the last week, and answer them with what's relevant to me and my experience. … I felt the instructions were really clear … it's a fairly straightforward questionnaire PID15, Patient with alopecia, UKHowever, some “did not read that part [instructions]” , which affected the validity of their answers, particularly in relation to the recall period. On this basis, several sections of the instructions were emboldened to draw respondents’ attention to the instructions and their most important aspects.Some suggestions to improve clarity were provided. First, participants felt that the example “because you do not work” created confusion as it led participants to believe that the items should be answered in relation to both their dermatological condition and employment. As a result, this example was removed from the instructions.Going through the questions in my head, I don’t know how work would have anything to do with the questions that were asked … I don’t even really think you need it all. PID14, Patient with psoriasis, CanadaSecond, some participants suggested alternatives to the term “dermatological conditions,” feeling it was wordy. A minority of participants with conditions primarily affecting the skin suggested using ‘a simple word like skin’  instead. Others with conditions that did not primarily affect the skin such as alopecia felt that dermatology implied a focus on the skin and would prefer another word, but could not provide a suitable alternative:Is there another word to say skin, which includes hair, nails, whatever, you know? I don't think there is, but that's the only thing that I would maybe look into, but I don't think there is a synonym. PID18, Patient with alopecia, UKBecause most participants found it acceptable and it is more inclusive than “skin,” the term ‘dermatological conditions’ was retained.You need it to be applicable to several different conditions, not just one and … there is the difficulty. So with that in mind, your questions are brilliant. PID18, Patient with alopecia, UKI am sure phrases like dermatological, I mean will be familiar to anyone with any sort of conditions PID13, Patient with psoriasis, UK3.3.Participants were almost unanimous in their criticism of the one-week recall period. Many felt that a longer recall period was required to accurately reflect the impact that their dermatological condition has had on their lives, largely due to the relapse and remitting nature of many of these conditions:Seven days isn’t long enough for someone with … [a] condition that they’ve no control over, and people can see it. Because that’s another thing like, lupus can flare, and it’ll go back down, and I can have three good weeks and then one really crap week where it’s just blown up on my face. So, I still think that the past week is too short a term to ask someone how it is. PID11, Patient with discoid lupus, IrelandParticipants also explained that many people do not engage in some of the experiences captured by the items  on a weekly basis and so expanding the recall period would likely increase item relevance to a higher proportion of patients and consequently more accurately capture disease burden:If the timeframe had been three months, six months, a year, or your lifetime … the feedback would be very different. So, you talk about relationships, intimate with [a] partner, all of these kinds of things, you know, social interactions, if somebody hasn’t had a social interaction in the last week they’re going to say never, whereas if the timeframe is much larger, you’re going to get a more realistic feedback. PID12, Patient with extensive linear porokeratosis, IrelandA 2-week and 1-month recall period were tested. These were generally more acceptable to participants than the 1-week recall. A 1-month recall period was adopted, having been suggested as an alternative to the 1-week recall period by multiple participants.3.4.A summary of the evidence of comprehensibility, relevance, and problems detected for each item is presented in Sixteen items were modified to align them more closely with the intended concept of interest outlined in the item definition list or to reduce conceptual overlap with other items.One item, “I have felt dismissed or abandoned by others,” was deleted because it was not easily understood by participants and was felt to be highly similar to “I have been excluded, bullied or discriminated against.”Overall, participants found the item ordering acceptable. The five domains of the conceptual framework were evident in the items, as participants correctly recognized categories of items, providing evidence in support of the suitability of the item ordering as well as the conceptual framework.They seemed to be grouped together quite well and I think the order of them was fine. PID15, Patient with alopecia, UKThe order of seven items was changed to enhance understanding. For example, the item “my everyday choices have been affected ” was listed before the item “my life goals and choices have been affected ” to highlight that the latter does not include everyday choices, which some participants subsumed under life goals and choices.Subgroup differences were found on four items. People with alopecia felt that the item “my treatment has caused practical problems ” was not relevant to them as they had no treatments. They also differed from people with other conditions on three items as alopecia appeared to have a positive impact in terms of timesaving, reduced financial costs and feelings of attractiveness.It positively impacted it [daily routine], because I don't have to mess about with my hair as much in the morning… it's a bit of a blessing. PID15, Patient with alopecia, UK3.5.Participants found the response options  to be appropriate, cover the full range of experience, and comprehensible.I found it quite easy, I think it gives a good range of options. Obviously always and never are complete extreme [s] and then a couple of the intermediates of different intensity, is fine. I think it is a really good way of asking questionnaires and usually makes it quite easy to answer. PID13, Patient with psoriasis, UKSome participants had difficulty interpreting the “not relevant” response (NRR) option. NRRs caused confusion, especially for items where the condition had been an obstruction to engaging in the relevant life area. This was most clearly discussed in relation to the item on intimacy, for example:‘It's been difficult to be intimate with a partner’ … this is the question that I always struggle with. [I chose] not relevant because I've not got a partner, but that can also be always, because I haven't got a partner, it can be both … the reason why I've not got a partner is because it's been difficult … I think it's an important question … I think [psoriasis is] … probably the reason why I'm single. In my formative years between when I started getting psoriasis and in hospital, was years when all my mates were getting wives and babies and all that. All of a sudden it had just passed me by, it had gone, you know. And it's like all of a sudden, I'm 40 odd and I'm like all my mates are married and having kids and I seem to have missed that bit. PID1, Patient with psoriasis, UKOthers could not distinguish the NRR option from the ‘never’ response option.There were some questions where it was never or … not relevant and I’m thinking … what did not relevant mean? … you might think not relevant is fairly self-explanatory but it’s not in my case … what’s not relevant? … I’ve ticked it and it’s not relevant because it never occurred … [so] you’d say never, wouldn’t you? PID9, Patient with psoriasis, UKThe edits to items 12 (“I have struggled to concentrate”) and 26 (“I have been prevented from or found it difficult to be intimate with another person”) reduced the need for the not relevant option as these items could now apply to all respondents, regardless of employment or relationship status. This option was no longer necessary and was therefore removed to simplify.4.This study tested the content validity, acceptability, and feasibility of PRIDD. It met the highest standards for cognitive interviews outlined by COSMIN  19, 27), 2719, 24.1.4.1.1.With the International Classification of Disease (ICD)-10  classifyWhile PRIDD appears to be relevant to people with dermatological conditions overall, some sub-group differences were found. The physical, psychological, and social impacts were generally consistent across conditions but practical impacts such as time and financial resources differed for people with alopecia. They emphasized the positive impacts of their condition, for example, regaining time lost to styling hair. Nevertheless, the feedback indicated a consensus that PRIDD was relevant and accepted across conditions. We believe this shows that, despite their inherent challenges, dermatology-specific PROMs are appropriate but need to be developed carefully with high levels of patient involvement throughout.4.1.2.The findings of this cognitive interview study support our conceptual framework of the impact of dermatological conditions . First, Given the importance patients placed on the influence of healthcare in determining the impact of their condition in our previous work , 37, we 4.1.3.We pilot tested a 5-point rating scale with an additional NRR option for each item. NRR options are common in dermatology. The DLQI, the most widely used PROM in dermatology , for exaacross the target population. Such high frequency and differing rates of NRRs suggest content validity issues with the DLQI as certain items are not relevant to a significant proportion of patients and subgroups groups , supporting the concept of Cumulative Life Course Impairment in dermatology –68. Part4.1.5.During PROM development, a balance is evident between maximizing the information gained about the construct of interest and reducing respondent burden, meaning that every item in a PROM must earn its place . Where pPRIDD appears to be feasible for use, with participants taking an average of approximately 4 min to complete. The average time to complete PRIDD in research and clinical practice is likely to be lower because, in most cases, participants were thinking and responding aloud while completing PRIDD and edits were consequently made to improve the comprehensibility of the measure.4.2.Though it is an important step in the development and validation of new PROMs , the pilThe main strength of this pilot test is the use of qualitative methods. The interviews followed the COSMIN-recommended TSTI method of cognitive interviewing , elicitiParticipants were sampled purposively through GlobalSkin’s unique global network to achieve diversity in terms of clinical  and demographic variables . This strengthens the content validity of PRIDD for global use across dermatological conditions. However, our participants were mainly patient organization members and therefore may not represent the experiences or views of non-members. With a sample size of 12, we were able to demonstrate data saturation as no major problems that could be resolved were identified in the final round of interviews. Still, rarer problems or those pertaining to conditions not represented may have gone undetected. Five of the 12 participants had psoriasis and we were unable to recruit participants with other common conditions such as acne, eczema, and vitiligo or those who do not speak British, American, or Canadian English, reducing the transferability of PRIDD. The next stage of development is psychometric testing. Here, we will be able to further test PRIDD with a larger sample of patients representing a wider array of dermatological conditions and global regions and who speak other variations of English.4.3.replace the discussion of the wider impact of the disease during the clinical consultation, but facilitate and frame patient-centered discussions  and interpretability information  is required . While c5.In this final step in the content validity phase of development, PRIDD was pilot tested through cognitive interviews with the target population. The data triangulated previous findings, recommendations, and the conceptual framework of impact. The results provide insight into how patients understood the items in PRIDD and shed light on the patient perspective on current debates regarding PROMs in dermatology. The resultant confirmation of the comprehensibility, comprehensiveness, relevance, acceptability, and feasibility of PRIDD provides evidence of content validity from the target population. The next step in the development of PRIDD is the psychometric testing phase.pattinsonr@cardiff.ac.uk.The datasets presented in this article are not readily available because, due to the nature of this research, participants of this study did not agree for their data to be shared publicly. Requests to access the datasets should be directed to RP, This study involves human participants was reviewed and approved by Cardiff University School of Healthcare Sciences Ethics Committee. The patients/participants provided their written informed consent to participate in this study.RP: conceptualization, methodology, validation, formal analysis, investigation, resources, data curation, writing—original draft, writing—review and editing, visualization, supervision, and project administration. NT-S: methodology, validation, and writing—review and editing. RH, MV, and NT: investigation and writing—review and editing. JA: conceptualization, resources, writing—review and editing, project administration, and funding acquisition. AF: resources, writing—review and editing, and project administration. NC: methodology, writing—review and editing, and supervision. MA: conceptualization, methodology, writing—review and editing, and supervision. CB: conceptualization, methodology, validation, resources, writing—review and editing, and supervision. All authors contributed to the article and approved the submitted version.This study was funded by GlobalSkin as a part of its Global Research on the Impact of Dermatological Diseases (GRIDD) project.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


It was in the late eighties and early nineties that the Leuven's group led by Paul Rutgeerts published a series of landmark studies describing the natural history of postoperative Crohn's disease (CD). These authors introduced the concept of postoperative recurrence (POR) as defined by the development of disease‐related mucosal lesions at the neoterminal ileum after a “curative” ileocecal resection. They showed that this phenomenon occurs early after surgery in up to 80% of patients within the first 12 months and that there is a clear correlation between the severity of these lesions seen at ileocolonoscopy and the risk of developing symptoms , which in some patients may lead to a new intestinal resection .Since then, many randomized, controlled studies have been performed to assess the efficacy of a number of drugs to prevent POR. To date, only thiopurines and anti‐TNF agents have been demonstrated to be useful in the prevention of early endoscopic POR.But bearing in mind that not all the patients will develop POR  and that the use of immunosuppressive therapies carries a risk of adverse effects, the best strategy after intestinal resection is still under debate.Systematic prevention with thiopurines and anti‐TNF agents is supported by their demonstrated efficacy. Moreover, most of these patients already developed disease‐related complications that drove them to surgery; therefore, leaving them without any maintenance therapy is against the current therapeutic goals that promote intensive treatment in those patients at risk of disabling disease.On the other hand, some authors propose the so‐called “endoscopy‐driven strategy”. Based on the benefits of treatment escalation in case of advanced endoscopic lesions,Finally, given that there are some epidemiological and clinical features that have been associated with a higher risk of early POR, the third strategy defends that the decision to start prevention or wait for endoscopic monitoring should be based on the presence or absence of risk factors. However, with the available risk factors in daily clinical practice, this strategy seems to be useless in improving postoperative outcomes.United European Gastroenterology Journal, Dragoni et alIn this issue of Although the study has some gaps related to its retrospective design , the observed results warrant a prospective evaluation of new decision‐making approaches in the postoperative setting of CD. Both primary prevention and endoscopy‐driven therapy strategies seem to be useful in the suitable patient…but how do we identify the patient? Maybe it is time to bring in histologicThe corresponding author confirms on behalf of all authors that there have been no involvements that might raise the question of bias in the work reported or in the conclusions, implications, or opinions stated. Eugeni Domènech has served as a speaker or has received research or educational funding or advisory fees from AbbVie, Adacyte Therapeutics, Biogen, Celltrion, Galapagos, Gilead, Janssen, Kern Pharma, MSD, Pfizer, Roche, Samsung, Takeda, Tillots; Míriam Mañosa has served as a speaker and has received research or educational funding from MSD, AbbVie, Takeda, Janssen, Faes Farma, Ferring and Pfizer; Margalida Calafat has served as a speaker for Takeda, Janssen, Faes Farma, and MSD.