To analyze tear function outcomes following collagen cross-linking (CXL) treatment in ectatic corneas.  Fifty-seven eyes of 34 patients were included, and patients with keratoconus who underwent epithelium-on (epi-on) or epithelium-off (epi-off) CXL were evaluated. The following tests were performed preoperatively and at 1, 3, 6, and 12 months postoperatively: best-corrected visual acuity (BCVA), maximum keratometry value (Kmax), ocular surface disease index (OSDI) questionnaire, slit-lamp examination, tear meniscus height, first noninvasive Keratograph breakup time (1st NIKBUT), average NIKBUT, and bulbar redness.  BCVA improved in both epi-on and epi-off groups at most follow-up points, but was not significantly different between groups. At 12-month follow-up, Kmax in the epi-on and epi-off groups improved after CXL, but there was no significant difference between the groups. The OSDI in both groups decreased after operation compared with before surgery, and there was no significant difference between the two groups. Comparing the two groups, only the change in the tear meniscus height at 6 months postoperatively was statistically significant, and the pre- and postoperative values of the two groups were within the normal range (>0.20 mm). The change was small and had no clinical significance. There was no change in the 1st NIKBUT and average NIKBUT between the epi-on and epi-off groups. A change in bulbar redness was significantly better in the epi-off group than in the epi-on group at 3 months postoperatively. Comparing the effects at 1 year postoperatively, both groups had positive results in OSDI, NIKBUT, tear meniscus height, and bulbar redness.  Both epi-on and epi-off CXL can control the progression of keratoconus, although epi-off CXL is more effective. Both methods have a positive effect on dry eye, which can improve the condition of the tear film and reduce dry eye symptoms in patients with keratoconus. Keratoconus is a progressive primary ectatic disease, which involves irregular astigmatism and corneal thinning due to structural and biomechanical changes in the stroma, leading to vision loss , 2. KeraThis prospective study protocol was approved by the Institutional Review Board of Tianjin Medical University Eye Hospital . The clinical study registration number was ChiCTR2000032444. Patients or their parents provided written informed consent for study procedures and protocols, which complied with the principles of the Declaration of Helsinki.Keratoconus frequently begins in puberty and progresses until the age of 30–40 years. Due to the influence of age on tear film function, this study only included patients <40 years of age. Patients were prospectively recruited; 57 keratoconic eyes of 34 patients from the Tianjin Medical University Eye Hospital  between May 2020 and September 2020 were included. Of the 57 keratoconic eyes, 26 were included in the epi-on group and 31 in the epi-off group.Patients were excluded if they had a history of systemic disease, any ocular autoimmune disease, keratitis, glaucoma, ocular injury or surgery, any other ocular surface disease, such as meibomian gland dysfunction, or previous treatment with systemic steroids or topical medication for artificial tears.D) corneal change in the maximum keratometry value (Kmax) and a 20-μm reduction in the central corneal thickness, which were determined using a Pentacam Scheimpflug tomography analyzer  [Progression of keratoconus was defined as a 1.00-diopter  . CXL wasAt each visit, data were collected regarding slit-lamp examination findings, best-corrected visual acuity (BCVA) using a conventional Snellen chart and described in the decimal visual acuity style, and Kmax value in the 3-mm diameter central zone, which were measured using corneal topography . Each patient was asked to complete an ocular surface disease index (OSDI) questionnaire, which is reliable and valid for evaluating the severity of dry eye disease , 12. TheThe examinations were performed chronologically by the same ophthalmologist. Tests were performed for the right eye and then the left eye preoperatively and at 1, 3, 6, and 12 months postoperatively. All visits were conducted in the morning, between 10 and 12 AM. The patient stopped wearing RGP contact lenses for 1 week before each visit to avoid the influence of RGP on the ocular surface. The incidence of complications was recorded at the 12-month follow-up.μm. Enough ParaCel solution  was applied to completely cover the cornea, and this process was repeated every 90 s for 4 min. The cornea was fully flushed using VibeX Xtra , which was then applied to completely cover the cornea; this process was repeated every 90 s for 6 min. Ultraviolet light treatment was commenced using the KXL System, after which the procedure was completed using standard techniques.The epi-on CXL procedure began by measuring the corneal thickness to ensure that it was >400 μm; thereafter, riboflavin ophthalmic solution was applied to completely cover the exposed stroma. Ultraviolet light treatment was commenced using the KXL cross-linking device . The procedure was then completed using standard techniques.The epi-off CXL procedure was commenced by removing the corneal epithelium over the desired area. A sufficient amount of riboflavin ophthalmic solution  was applied to completely cover the exposed stroma. This procedure was repeated at 2-min intervals up to a total of 30 min based on the desired depth of cross-linking. The corneal thickness was subsequently measured to ensure that it was >400 A bandage contact lens was applied for the patients after the procedure. During the postoperative period, patients were instructed to use fluorometholone eye drops  4 times daily for 1 week after the procedure. The eye drop dosage was tapered over the following 4 weeks. Patients were also instructed to use levofloxacin eye drops  for the first 3 days after the procedure. The bandage contact lens was removed on postoperative day 3.t-test (tear meniscus height and bulbar redness) and Mann–Whitney U test . The generalized estimating equation was used to compare outcomes preoperatively and at 1, 3, 6, and 12 months postoperatively. This equation was calculated to account for intereye correlation for eye-specific measurements. The lower and upper bounds of the 95% confidence intervals (95% CI) are shown in Tables 1P < 0.05 was considered statistically significant.Statistical analyses were performed using SPSS . All data are expressed as mean ± standard deviation. Normal data distribution was tested using the one-sample Kolmogorov–Smirnov test. Changes between different groups were analyzed using independent samples P < 0.076), respectively. There was no significant difference between the two groups with regard to BCVA, OSDI, tear meniscus height, 1st NIKBUT, average NIKBUT, or bulbar redness before surgery (P > 0.05).No patient experienced treatment-related adverse events, and all patients initiated wearing their RGP lenses at 6 weeks postoperatively without any sign of intolerance. During the study period, no infectious keratitis was observed. No patient dropped out during the study.The demographic characteristics and baseline findings of the patients are shown in r = −0.772, P < 0.01) .A highly significant inverse correlation was observed between the value of Kmax and BCVA in the two groups ( < 0.01) . A positP > 0.05) (P > 0.05) (P > 0.05)  . The BCV > 0.05) . There w > 0.05) .P=0.028) and 12  months postoperatively, while at 1  and 6 months  postoperatively, Kmax was steeper than at preoperatively (Tables P=0.294) (Tables P=0.048). And Kmax also decreased at 12  months postoperatively. However, there was no significant difference in pre- and postoperative Kmax values between the groups at each follow-up points (P > 0.05).In the epi-on group, Kmax decreased at 3  Tables . Kmax vaP=0.048) (Tables P > 0.05)  Tables . The oth Tables 1.P=0.331) at the 6-month follow-up was significantly different (P=0.036) from the epi-on group .There was a slight change in the tear meniscus height between the two groups at each follow-up compared with that before surgery Tables –3, FigurP=0.001) (Tables P = 0.939) and 6 months  postoperatively compared with the preoperative values, while it increased at 3  and 12 months  postoperatively compared with the preoperative values (Tables P > 0.05)  Tables . The 1st > 0.05) .P=0.021) and was statistically significant, and then gradually increased from the third month onward (P > 0.05) (Tables P < 0.01)  Tables . The ave) Tables .P=0.289) and 12 months  postoperatively and increased at other time points compared with that of the preoperative period (P > 0.05) (Tables P=0.001) (Tables P=0.021)  Tables . The bul) Tables . There wP=0.021) .The standard technique for corneal stabilization in cases of keratoconus is CXL, which aims at increasing biomechanical stromal stability and stopping disease progression via the formation of new chemical bonds between collagen fibrils and extracellular matrix proteins in the stroma –16. It iD preoperatively and 61.42 ± 8.92, 61.17 ± 7.96, and 60.02 ± 7.58 D at 1, 6, and 12 months after accelerated transepithelial CXL (P=0.05), respectively [In our study, the BCVA of the epi-on and epi-off groups increased at most follow-up time points, indicating that both epi-on and epi-off CXL are beneficial to vision improvement. BCVA and Kmax also had a negative correlation; therefore, improvement in visual acuity after CXL was consistent with the decrease in Kmax in the two groups. However, this conclusion did not apply to the changes of BCVA and Kmax in the epi-on and epi-off groups at 1 and 6 months postoperatively in the epi-on group. Kmax increased at 1 month after epi-on and epi-off CXL, but BCVA also increased. In Weijun Jian's study, the Kmax values were 60.10 ± 7.51 ectively . Kmax inectively . In our P  >  0.05). This result is consistent with that of a previous study [The cornea is one of the most densely innervated tissues in the human body. Corneal nerve damage caused by trauma, disease, or surgery can reduce corneal sensitivity and affect the functional integrity of the ocular surface . The difus study . The staus study .Loss of corneal sensation is expected to adversely affect blinking and basic tear secretion. There was no significant difference in our study in tear meniscus height pre- and postoperatively, indicating that CXL had no effect on tear secretion . The senThe 1st NIKBUT in the epi-on group was significantly increased 12 months postoperatively than that before surgery. Moreover, the average NIKBUT in the epi-off group was significantly increased at 12 months postoperatively when compared to its preoperative values. The Kmax values of the two groups also decreased at 12 months postoperatively than that before surgery. Dogru et al.  demonstrP=0.001). In Uysal's study [The bulbar redness scores were unchanged in the epi-on group but significantly decreased in the epi-off group at 12 months postoperatively compared with that before surgery ('s study , tempora's study , and theOur study suggests that both epi-on and epi-off CXL can control the progression of keratoconus, and epi-off CXL is more effective. Both epi-on and epi-off have a positive effect on dry eye, which can improve the condition of the ocular surface and reduce the symptoms of dry eye in patients.Cessation of RGP use for more than 1 week will significantly affect the patient's visual quality and quality of life. Therefore, in this study, the patient only stopped wearing RGP for 1 week. Future studies can add a control group, wherein patients with keratoconus will not undergo surgery, but the changes in tear film function before and after wearing RGP alone will be compared. In our study, only the OSDI questionnaire, tear meniscus height, 1st NIKBUT, average NIKBUT, and bulbar redness were included. Other tear film parameters, such as tear osmolarity, lid wiper epitheliopathy, tear clearance, and fluorescein staining, would have provided a broader perspective to the study. In addition, the inclusion of both eyes of the same patient in our study may cause potential statistical bias. Finally, the findings of the present study are limited by the small sample size. Thus, a large sample size study with monocular inclusion of study subjects and different age groups is required to confirm how CXL affects the ocular surface in patients with keratoconus, which may guide the selection of individualized treatment plans.


P = 0.0358). The effects of TBs on long-term therapeutic efficacy were further studied based on event-free survival (EFS) and overall survival (OS) profiles; both the OS and EFS of the low TB group were better than those of the high TB group, but the differences were not statistically significant. Importantly, the time points of TB measurement did not significantly affect the OS and EFS profiles regardless of whether the TBs were measured before or after fludarabine-cyclophosphamide (FC) preconditional chemotherapy. On the other hand, the severity of CRS was significantly correlated with the TB level (P = 0.0080), and the incidence of sCRS was significantly related to the TB level . Unexpectedly, the ssCART-19 cell expansion peaks were not significantly different (P = 0.2951) between the study groups. Patients with a low r/r B-ALL TB yield more net benefits from CAR-T treatment than those with a high TB in terms of safety and CR rate. These findings are critical and valuable for determining the optimal CAR-T cell treatment window for r/r B-ALL patients and will further the development of comprehensive and reasonable CAR-T cell treatment plans for r/r B-ALL patients with differential TBs.The tumor burden (TB) is significantly related to the severity of cytokine release syndrome (CRS) caused by CAR-T cells, but its correlation with therapeutic efficacy has not been systematically studied. This study focused on the effects of the TB level on both the safety and efficacy of ssCART-19 as a treatment for r/r B-ALL. Taking the 5% tumor burden as the boundary, the study participants were divided into 2 groups, high and low tumor burden groups. Under this grouping strategy, the impacts of differential r/r B-ALL TBs on the clinical therapeutic efficacy  and safety profiles after ssCART-19 cell treatment were analysed. 78 patients were reported in this study. The differential B-ALL TBs significantly affected the complete remission (CR) rates of patients treated with ssCART-19, with rates of 93.94% and 75.56% in the low and high TB groups, respectively (Trial registration: ClinicalTrials.gov identifier, NCT03919240. However, despite the continuous development of CAR-T cell therapy, some insufficiencies remain regarding its long-term effectiveness and safety. For example, recurrence can occur in patients (more than 50%) treated with CAR-T cells within 12 months after the achievement of CR3. Moreover, serious side effects, such as cytokine release syndrome (CRS) and neurotoxicity, often occur during CAR-T cell therapy and can even cause patient death8. These problems have largely limited the clinical application of CAR-T cell therapy in r/r B-ALL. Therefore, identifying predictive and early indicators of the clinical efficacy, prognosis, and safety of CAR-T cells is of great significance for formulating a reasonable clinical CAR-T cell treatment plan.As a new immunotherapy, chimeric antigen receptor T (CAR-T) cell therapy has shown remarkable effects in refractory or relapsed B-cell acute lymphoblastic leukemia (r/r B-ALL). Studies have shown that the complete remission (CR)/CR with incomplete haematologic recovery (CRi) rate of CAR-T cell therapy can be as high as 90%14. For instance, Pan J reported that the threshold dose for CAR-T cell infusion was related to cell expansion and therapeutic efficacy15, and higher remission rates (55–94.3%) were shown to be achieved with doses exceeding 1 × 106/kg3. In addition, Fraietta, J. A found higher expression levels of memory-related genes in CAR-T cells from responsive chronic lymphocytic leukemia (CLL) patients, while the genes with increased expression in CAR-T cells from nonresponsive patients were shown to be involved in effector cell differentiation, glycolysis, depletion, and apoptosis15. Qin, J. S also found that ibrutinib could transform CAR-T cells into memory cells, which thereby increased the clearance rate of CD19+ tumors in mice and prolonged the survival times of tumor-bearing mice16. Although some studies have reported that tumor burden factors are correlated with CR after CAR-T cell therapy and with the survival rate, their inconsistent patient treatment methods and CAR-T cell infusion doses potentially affect the accuracy of statistical analyses17. Therefore, standardizing and systematically studying the tumor burden impacts on the CAR-T cell treatment response and long-term efficacy are necessary and important.Multiple studies have shown that numerous factors are potentially related to the efficacy of CAR-T cells19. Studies have also shown that the tumor burden is significantly related to the safety of CAR-T cell clinical treatment20. Davila, M. L analysed the correlations among 39 cytokines, the tumor burden prior to treatment and sCRS and found that the increased levels of 7 cytokines (interferon (IFN)-γ, interleukin (IL)-5, IL-6, IL-10, FIt-3L, granulocyte macrophage-colony stimulating factor (GM-CSF) and fractalkine) were related to the pretreatment tumor burden and to sCRS. Further analysis showed that the severity of CRS was significantly correlated with the tumor burden prior to 19-28z CAR-T cell infusion19. Similarly, Brentjens, R. J reported that the cytokine levels were correlated with the B-ALL tumor burden before treatment21. Moreover, Teachey, D. T found high levels of 24 cytokines, including IFNγ, IL6, sgp130, and sIL6R, within the first month after CAR-T cell infusion, which implied an intricate and high correlation with CRS7.Severe CRS (sCRS) and CAR-T cell-related neurotoxicity are common side effects of CAR-T cell treatment. Many factors have been reported to be associated with these side effects, including the dose used for CAR-T cell reinfusion, the costimulatory domain molecules on CAR-T cells, and the types and amounts of proinflammatory factors releasedBased on the duality of the efficacy and side effects of CAR-T cell treatment, the ultimate benefit of CAR-T cell treatment depends on the net benefit yielded from balancing treatment efficacy and side effects. The tumor burden affects the safety of CAR-T cell clinical treatment, but its impact on CAR-T cell efficacy has not been investigated in a systematic and standardized manner. Therefore, we analysed the therapeutic outcomes of patients with r/r B-ALL treated with ssCART-19 cells, researching the correlations of the tumor burden with the CR rate, long-term survival  and event-free survival (EFS)), and clinical risk. This study aimed to elucidate reliable indicators that are predictive of clinical treatment efficacy and risks and to guide the formulation of a reasonable overall clinical treatment plan for patients with varying tumor burdens to maximize the benefit of CAR-T cell treatment.This study was approved by the ethics committee of the First Affiliated Hospital of Soochow University and conducted to assess the safety and efficacy of ssCART-19 cells  in patients with relapsed or refractory CD19+ B-cell malignancies. The primary endpoint was the safety of treating relapsed or refractory CD19+ B-cell malignancy patients with ssCART-19 cells, and adverse events were graded according to the Common Terminology Criteria for Adverse Events (CTCAE) version 4.0.3. The secondary endpoint was the objective response rate at the end of the study. All the patients provided written informed consent, and the enrolled patients met the following inclusion criteria: diagnosed with CD19+ r/r B-ALL with sufficient organ function  and a Karnofsky performance status ≥ 60 or an ECOG performance status ≤ 2. The study was registered at ClinicalTrials.gov (NCT03919240). First Submitted Date: 15/04/2019. First Posted Date: 18/04/2019.All patients underwent bone marrow evaluations before or after interim therapy and immediately before the T cell infusion.6 cells per kilogram of body weight with split doses of 10%, 30%, and 60%. The patient's body temperature, blood pressure, heart rate and pulse oxygen were monitored, and blood was drawn according to the clinical trial process to monitor the patient's cytokine and CAR-T cell expansion levels in the peripheral blood. Blood test parameters, blood biochemistry parameters, ferritin, and C-reactive protein were also routinely monitored according to the protocol.After leukapheresis and ssCART-19 cell manufacturing, patients received conditioning chemotherapy consisting of fludarabine (30 mg per square metre of body surface area per day) and cyclophosphamide  on days − 5, − 4, and − 3 (FC treatment) before the administration of ssCART-19 cells at a dose of 5 × 1017. The analysis of overall survival used death as the event, and the analysis of event-free survival used the earliest of no response, relapse, or death as the event. Patients who did not have an event had their data censored for the analyses at the date at which they were last known to be alive.CR was defined as less than 5% blasts in the bone marrow without myelosuppression, no circulating blasts in peripheral blood, and the absence of extramedullary disease, regardless of cell count recovery. A negative MRD status was defined as less than 0.01% bone marrow blasts assessed by multiparameter flow cytometry, and absence of genetic aberrants assessed by karyotype analysis or molecular detection. Relapsed disease was defined as loss of CR status after the achievement of CR, including hematological relapse (bone marrow ALL blasts > 5%) or extramedullary relapse CRS was graded according to the National Cancer Institute CTCAE version 4.03 and was considered to be severe at grades higher than or equal to 3.CAR-T cell-related neurotoxicity was assessed according to the National Cancer Institute CTCAE version 4.03, and severe neurotoxic effects were defined as a seizure of any grade or a toxic effect of grade 3 or higher.22. Briefly, peripheral blood mononuclear cells were purified from the patient’s blood by gradient centrifugation using Lymphoprep , and their CD3+ T cells were then enriched by positive selection using magnetic bead separation . The T cells were transduced with lentiviral supernatant, followed by activation in vitro using anti-CD3/CD28 monoclonal antibodies (Miltenyi Biotec) at 5% CO2 and 37 °C for 24 h. After transduction for 48 h, the CAR-T cells were cultured and expanded in AIM-V T cell medium , which contained 100 IU/ml recombinant human IL-2 , 5 ng/ml recombinant human IL-7 (PeproTech), 5 ng/ml recombinant human IL-15 (PeproTech) and 10% autologous plasma, at 5% CO2 and 37 °C for 14 days.The ssCART-19 cell manufacturing procedures were described previouslyCytokines were measured using the Th1/Th2 Cytometric Bead Array Kit II (BD Bioscience) according to the manufacturer’s instructions. In brief, serum was collected from patients at different time points. The cytokine-capture microspheres were first mixed and then incubated with serum samples and fluorophore-labelled antibodies for 3 h before being washed and evaluated by flow cytometry (Thermo Fisher). The concentration of each cytokine was calculated from standard curves.23.The expansion and persistence of the ssCART-19 cells were monitored by real-time quantitative PCR (qPCR). All patients peripheral blood samples have obtained at different time points. The corresponding DNA has amplified with primers and probes complementary to specific sequences within the lentiviral vector followed by extracted from the peripheral blood samples. qPCR has performed using the ABI 7500 Real-Time PCR System , and calculation of the CAR-T copy number depends on the standard curve established with the plasmid encoding the transgene24.CAR-T cell differentiation stages were assessed by flow cytometry. CAR-T cells were harvested and washed twice with 1 ml of phosphate-buffered saline containing 2% foetal bovine serum , followed by incubation with the following antibodies: APC-Cy7-conjugated anti-CD8 , Alexa Fluor 700-conjugated anti-CD4 , PE-conjugated anti-CCR7 , PerCP-Cy5.5-conjugated anti-CD45RA , and PE-Cy7-conjugated anti-CD127 The EFS and OS data in this study were analysed using the Kaplan–Meier log-rank test. T tests (paired or unpaired), the chi-square test and Fisher's exact test were also used in this study. P values less than 0.05 indicated statistical significance. This study was performed in accordance with the Declaration of Helsinki and was approved by the ethics review committee of the First Affiliated Hospital of Soochow University. All participants provided written informed consent.A total of 107 patients with r/r B-ALL were enrolled between February 2017 and May 2020. Among them, 78 patients were treated with ssCART-19 cells and were included in the statistical analysis, while the remaining 29 patients were not included in the analysis. The specific exclusion criteria are shown in Fig. All 78 patients were treated with FC. According to the patient's tumor burden before CAR-T cell reinfusion, the 78 patients were divided into two groups. Thirty-three patients were included in the low tumor burden group and had minimal residual disease (MRD) with bone marrow blast percentages ranging from 0.01 to less than 5; the remaining 45 patients were in the high tumor burden group, with bone marrow blasts percentages equalling or exceeding 5.Among the 33 patients in the low tumor burden group, 25 (75.76%) received ssCART-19 cell therapy as a third or later salvage treatment, 26 (78.79%) were treated with other approaches for more than half a year, and 11 (33.33%) received bone marrow transplantation before ssCART-19 cell treatment. Nine (23.27%) patients were diagnosed with Philadelphia chromosome (Ph)-positive ALL, 28 (84.85%) relapsed before ssCART-19 cell treatment, 1 (3.03%) had a TP53 gene mutation, and 11 (33.33%) were evaluated as poor cytogenetic risk patients.Among the 45 patients in the high tumor burden group, 30 (66.67%) received ssCART-19 cell therapy as a third or later salvage treatment, 28 (62.22%) were treated with other approaches for more than half a year, and 8 (17.78%) received bone marrow transplantation before ssCART-19 cell treatment. Five (11.11%) patients were diagnosed with Ph-positive ALL, 29 (64.44%) relapsed before ssCART-19 cell treatment, 2 (4.44%) had a TP53 gene mutation, and 13 (28.89%) were evaluated as poor cytogenetic risk patients. Table It is worth noting that the time of tumor burden evaluation in this study was not completely uniform. The evaluation time points were distributed before and after FC treatment. Among the study cohort, the tumor burdens of 25 (75.76%) people in the low tumor burden group were evaluated prior to FC treatment, while those of 32 (71.11%) people in the high tumor burden group were evaluated before FC treatment. According to the results of this study, the time point of tumor burden assessment had no significant effect on the long-term efficacy , 36 achieved MRD-positive or MRD status-unknown CR (46.15%), and 13 patients had no response (NR) (16.67%). The overall CR rate reached 83.33% . IL-6 is closely related to the CRS response22, suggesting that it is the main factor underlying the high sCRS response rate in the high tumor burden group. Based on the dynamic results for these four cytokines, like TNF-α, IL-2, IL-6 and IFN-γ were secreted at higher levels in the high tumor burden group than in the low tumor burden group, but the differences were not obvious  conditioning chemotherapy before CAR-T infusion, and another 10 patients received Fludarabine + Cy chemotherapy. Secondly, the CAR-T infusion dosage was also different, 33 patients received different dosages of CAR-T cells. Some patients with a higher tumor burden received a lower dose of 1 × 106 19-28z CAR-T cells/kg. Some patients with a lower tumor burden received a higher dose of 3 × 106 19-28z CAR-T cells/kg. For the study of Brentjens, R. J, only 5 patients were treated during the study, from a statistical point of view, it is hard to obtain an accurate conclusion with this limited number of study subjects. Because CRS often occurs only during CAR-T cell treatment, the correlation between the tumor burden and CRS is easily studied. In contrast, drawing effective conclusions from the correlations of the tumor burden with EFS and OS often requires a sufficient number of patients and a long follow-up time. Therefore, a more systematic and reasonable approach to analysing clinical research data is critical and necessary.CAR-T cell therapy triggers a cellular immune response via the specific activation of CAR-T cells by specific antigens on target cells. Therefore, the number of target cells affects the degree of CAR-T cell stimulation and the intensity of the immune response. B-ALL is a type of haematological tumor, and the tumor burden is equivalent to the target cell load. Therefore, the B-ALL tumor burden is one of the main factors that determines the strength of the bodily CAR-T cell-target cell-mediated cellular immune response and the performance and clinical efficacy of CAR-T cells during clinical treatment. The CAR-T cell treatment of B-ALL in patients with a high tumor burden is more likely to cause sCRS than that of patients with a low tumor burdenIn this study, 78 patients were treated with ssCART-19 cells and we systematically evaluated baseline parameters in the study participants in a standardized manner prior to the statistical analysis to ensure that the baseline data were consistent and that the various tumor burden analyses were as reliable as possible. At the same time, we followed a consistent ssCART-19 treatment process including the conditioning chemotherapy, CAR-T dosage to reduce the impacts of factors other than the tumor burden.17 reportedly observed no significant correlation between the tumor burden and the CR rate, but the tumor burden did significantly affect the OS and EFS. Although the conclusions of the two studies are not completely consistent, the overall efficacy (both CR rate and OS/EFS) trends are the same, and despite the significant differences, both studies suggest that a lower tumor burden leads to higher CR rate and better OS/EFS. Furthermore, this conclusion is basically consistent with our previous study27. The differential conclusions may be due to the different treatment plans utilized in the two studies. The pretreatment chemotherapy regimens were different for the patients in the high and low tumor burden groups, and the CAR-T cell doses were also different. These differences can obviously significantly impact the treatment effects15. Brentjens, R. J reported that the tumor burden was not related to clinical efficacy21. We believe that his study did not include a sufficient number of patients, thereby leading to a lack of sufficient data to draw definitive conclusions.Based on the above research and our research results, we conclude that ssCART-19 cells have good curative effects and that the CR rate is significantly related to the tumor burden. The OS and EFS results demonstrated that patients with a low tumor burden benefited more than those with a high tumor burden. Park, J. H21. We also found that the incidence of sCRS in patients with a high tumor burden was significantly higher than that in patients with a low tumor burden, indicating a lower safety risk of CAR-T cell therapy in patients with a low tumor burden. Additionally, we found a significant correlation between the IL-6 peak and tumor burden, which potentially explains why patients with a high tumor burden are more likely to develop sCRS. Based on this, we also further divided the patients into 5 subgroups according to their tumor burden, and the CRS reaction became increasingly severe as the tumor burden increased. This finding has not yet been reported and may provide a more accurate clinical reference which may serve as an important early warning and estimated reference value for the occurrence of sCRS risk during clinical CAR-T cell treatment.. In addition, we statistically analysed the number of adverse events during the clinical treatment process, and the results were positively correlated , suggesting that the relationship between adverse events and the tumor burden is basically consistent with that between CRS and the tumor burden. Laboratory indicators such as red blood cells, hemoglobin, lymphocyte percentage, C-reactive protein, ferritin, albumin, and γ-glutamyl transpeptidase levels are also correlated with the tumor burden . Similarly, Pan, J found that patients had fever and elevated ALT/AST levels and that the incidence of hypoxemia and coagulopathy were potentially related to the leukemia burden14.The impact of the tumor burden on the safety of the treatment process observed herein is consistent with previous reports29, this study did not identify a correlation between CAR-T cell pharmacokinetics and the severity of CRS . These results suggest that the treatment of patients with a high tumor burden is not as safe as that of patients with a low tumor burden, which may not simply be related to the number of CAR-T cells.We also herein investigated the pharmacokinetics of CAR-T cells in r/r B-ALL patients with varying tumor burdens, and the CAR-T cell expansion peak was not significantly correlated with the tumor burden. Therefore, ssCART-19 cells are mild and stable in vivo. Although the number of CAR-T cells in the high tumor burden group was higher than that in the low tumor burden group, the cells did not overreact even with the high tumor burden and more target antigens, demonstrating the improved safety of ssCART-19 cells. This result might also explain why the CR rate in the high tumor burden group was significantly lower than that in the low tumor burden group. A dynamic CAR-T cell curve was analysed using a paired t test. Although the copy numbers were determined at different time points, the overall statistical results showed significant differences. This result demonstrates that while ssCART-19 cells prompt differential responses in patients with different tumor burdens, the response are not overly intense, thereby ensuring effective treatment while considering safety. Unlike other reports30.This study revealed that the tumor burden is a significantly important reference for the efficacy and safety of CAR-T cell treatment. Before CAR-T cell infusion, the tumor burden should be reduced to the greatest extent possible to improve the treatment effect and reduce the safety risk. In addition to conventional chemotherapy, chemotherapy combined with radiotherapy can potentially be utilized to improve the treatment effect, thereby reducing the tumor burden. For example, we have attempted to use chemotherapy and radiotherapy to reduce the high burden in patients with diffuse large B-cell lymphoma (DLBCL) before CAR-T cell treatment , and found that radiotherapy is highly recommended before CAR-T cell treatment for R/R DLBCL patients21. Most of these centres use a “split dose” regimen to reduce the clinical risk, and patients may benefit more when this strategy is utilized.For extremely refractory patients, other methods to reduce the tumor burden may not be effective, and clinicians can consider adjusting the treatment plan to improve the treatment effect and reduce the safety risk. For example, Memorial Sloan-Kettering Cancer Center (MSKCC) and Fred Hutchinson Cancer Research Center (FHCRC) adopt risk-adapted CAR-T cell dosing strategies to choose the best dose for balancing efficacy and safetyThe impacts of other factors on survival were evaluated when assessing the patients’ baseline information in this study, but no significant correlations were found (data not shown). Whether the tumor burden assessments at different time points, such as before and after FC pretreatment, have differential effects on the CAR-T cell treatment response, severe CRS, EFS, and OS was previously unknown. However, this study showed that under uniform FC pretreatment conditions, assessing the tumor burdens before and after FC treatment did not affect the final statistical results Fig. ; Fig. S1Finally, the sample size may not be large enough after more subgroup are divided, therefore, the more case data are needed to consolidate our research conclusions. And with the more related factors and more types of cytokines related to the CRS reported, the cytokines and biochemistry marker reported in this study may not be enough. Further studies may require more testing items.The CAR-T cell treatment of r/r B-ALL patients with a low tumor burden has a obviously higher CR rate than that of patients with a high tumor burden. The time point of tumor burden assessment does not affect the predictions of CR rate, survival time and safety (incidence of sCRS) based on the tumor burden. When making an overall CAR-T cell treatment plan, minimizing the tumor burden may become an important goal. The results of this study provide an important basis for formulating a reasonable plan for the CAR-T cell treatment of r/r B-ALL to obtain the greatest benefits. They may also provide an important basis for rationally formulating an overall CAR-T cell treatment plan for other cancers to obtain greater benefits.Supplementary Information.


The three-parameter Logistic model (3PLM) and the four-parameter Logistic model (4PLM) have been proposed to reduce biases in cases of response disturbances, including random guessing and carelessness. However, they could also influence the examinees who do not guess or make careless errors. This paper proposes a new approach to solve this problem, which is a robust estimation based on the 4PLM (4PLM-Robust), involving a critical-probability guessing parameter and a carelessness parameter. This approach is compared with the 2PLM-MLE(two-parameter Logistic model and a maximum likelihood estimator), the 3PLM-MLE, the 4PLM-MLE, the Biweight estimation and the Huber estimation in terms of bias using an example and three simulation studies. The results show that the 4PLM-Robust is an effective method for robust estimation, and its calculation is simpler than the Biweight estimation and the Huber estimation. One of the major concerns in item response theory (IRT) is the robust estimation of latent ability. Response disturbances such as guessing, cheating, carelessness, and transcription errors often cause biases in estimated latent ability, are determined by a maximum likelihood estimator (MLE). Waller 1974), with Wainer and Wright (1980), proposed approaches to the robust estimation of latent ability 974, with, includiθv is the ability of subject v, ai is the discrimination parameter of item i, and bi is the difficulty parameter of item i.As we know, the IRT models include the Rasch model and the two-parameter Logistic model(2PLM) . The 2PLConcerning guessing responses in tests, Birnbaum 1957) proposed a three-parameter Logistic model (3PLM) Reviewers' comments:Reviewer's Responses to QuestionsComments to the Author1. Is the manuscript technically sound, and do the data support the conclusions?The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: NoReviewer #2: NoReviewer #3: Partly**********2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: NoReviewer #2: NoReviewer #3: Yes**********3. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #1: NoReviewer #2: YesReviewer #3: Yes**********4. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #1: NoReviewer #2: NoReviewer #3: No**********5. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #1: I have reviewed the manuscript 'The Robust Estimation of Examinee Ability Based on the Four-parameter Logistic Model When Guessing and Carelessness Responses Exist'. Although this manuscript addresses an interesting topic, I have detected (what for me are) several important limitations.* I am not a native English speaker. I usually need the assistance of a professional translator with my own manuscripts. Having said that, I have found tha English level below what could and should be expected. For instance, I'm not sure about what this is supossed to mean: "According to their investigation, the Huber estimation, rather than the Biweight estimation, should be applied when reducing sampling variability was prior to reducing biases."* The structure should be improved. For instance:- "Waller (1974) and Wainer and Wright (1980) proposed approaches to the robust estimation of latent ability, including traditional correction for guessing, Jackknife, AMT-Robustified Jackknife, and WIM. AMT-Robustified Jackknife was generally considered the best and most efficient method for tests comprising forty or fewer items[2]." The authors here list several alternatives (from rather old sources) that are never described. Why do the reader need this apparently irrelevant information? Why, if the AMT-Robustified Jackknife was considered the best alternative, is not used in the present study?- Much of the details of the estimation approach described in the simulations studies should be placed in the Introduction, where those models are supposed to be described.* In the section 'Recent literature on the 4PLM', the authors should clearly differentiate the results from computerized adaptive testing from those from non-adaptive applications. Up to now, those two lines of research are combined.* Authors duplicate information. For instance, r = a(theta - b) can be found in page 4, 6, 9...* Equations in page 7 cannot be properly read, as some elements have been moved.* From my point of view, authors should not use BILOG to estimate trait levels. Or, at least, they should provide evidence that their Visual Basic routines, when programmed for 2PLM, BIW, and 3PLM, lead to the same results as those from BILOG. Otherwise, we don't know if the differences can be attributed to software differences or to the estimation methods in themselves.* Authors should justify why they are using maximum-likelihood and not any of the Bayesian alternatives, as expected a posteriori with a uniform prior.And, more importantly:* I really don't know what can be learnt from the 'Example' study. If there is no correct theta level, what are the readers expected to learn from this? The authors used clearly incorrect models (that would have never been estimated if the correct model was 2PLM), as the 3PLM with c = .20. What is the point of doing that? Isn't obvious that using an incorrect model can distort trait level estimates?* With respect to the simulation studies, it was difficult for me to correctly evaluate them, as the provided description is very difficult to follow. If the new method is correctly described in the Introduction, the description of the design of those studies could be greatly simplified.Apparently, the authors ahve found that the parameters are better recovered when the model used to generate the data is also the model used to recover data. Perhaps I'm missing something relevant, but, again, this seems as obvious.* I would greatly recommend this article:Green, B. F. (2011). A comment on early student blunders on computer-based adaptive tests. Applied Psychological Measurement, 35(2), 165-174.There some very pertinent points about this area of research were raised. I would not repeat them here.Reviewer #2: The paper introduces robust estimation method for the four-parameter logisitic model (4PLM) to account for guess and careless responses. While I think I could understand the premise and purpose, I am afraid I became lost quite early either due to mistakes, or lack of clarity in derivations and reasoning. I think the paper would benefit from being much clearer about what is new here, and what others have done, but also please have a close look at formatting, potential errors in equations and maybe notations. Have some comments below that could perhaps help start this, and if I wrong with these then perhaps just more clarity is needed.Main comments1. Formatting of (5), (6) and (7) needs fixing. It is difficult to determine what the equations should be.2. On line 117: why is P(ri) now exp(ri)/(1 - exp(ri))? Isn't is exp(ri)/(1 + exp(ri)) since it is a probability. Then line 119 onwards as well do maybe I am missing something here. With the formatting issues for (5), (6) and (7) I can't tell what is happening here, and whether this makes a difference.3. I got lost with some of the notation, which had me second guessing at times. E.g. what is eta_i? E.g. in (5) isn't is just the residual ri?4. In the two sections starting Line 105, it wasn't clear to me what was new to this paper, and what has previously been introduced by others. Is it just (8)?Minor (example minor points)The paper needs to some careful attention to grammatical errors and formatting issues. I have some examples below.While 3PLM and 4PLM are defined in the abstract, they also need to be defined in the intro before use. 2PLM also needs to be defined.Shouldn't rv on line 62 also be indexed by i? Later, ri used to indicate item i and an arbitrary subject.line 106: "approaches" should "approach"Eq 4: uj?line 117: Stat of new sentence, "let" should be "Let" and need to end the sentence after Eq. 4.Line 119: new sentence, "if" to "If"Line 119 onwards: again exp(ri)/(1 - exp(ri)) so maybe I am missing something hereReviewer #3: The manuscript addresses the problem of estimate the ability parameter in the presence of response disturbance, such as guessing, cheating, careless and transcription errors. The authors propose a new model that would be robust in the presence of that kind of occurrence and compare its results with the standard models present in Item Response Theory literature: 2PLM-MLE, 3PLM-MLE, the 4PLM-MLE, the Biweight estimation and the Huber estimation. It is worth mentioning that provide accurate ability estimation even in the presence of disturbance is a core matter in education assessment. The article is able to present the methodology and results quite clear. Therefore, I would recommend the publication if the following changes and some clarifications are met:#Major changes ###########################################################The major shortcoming of the manuscript is that the simulation study does not provide a fair comparison between the models. That is, it assumes that the items parameters are all known in advance but, for some models, it does not match with the true generated data. Indeed, the best results are obtained, in each case, as expected, by the model that matches with the true generated data. For example, is assumed in advance that de c parameter in the 3PL-MLE model follows a normal distribution N. That is, the c parameter will almost surely be located between 0.17 and 0.23. Nevertheless, in simulation Case II the c parameter of the true generated model is always zero. In a real situation, the items parameters are obtained previously through a pre-test and c parameter could be near zero in 3PL-MLE. Indeed, the parameters of each item are supposed to represent the empirical reality. I would recommend that, for each Case, the authors generate two data sets. The first data set would be used only to estimate the items parameters for each model. Therefore, these items were pre-tested. Then, you can use the second data set to estimate the ability parameter using the items parameters previously obtained. This would reproduce what actually occur in practice. Anyway, this is only a suggestion; the important point is to provide a fair comparison, using this solution or another.I also suggest to include a simulation (Case IV) in which the true generated model follow exactly the 3PL-MLE or the 4PL-MLE to see the behavior of the other models in that situation. Finally, I recommend to include a standard 4PLM-Robust in all simulation Case, for example 4PLM-Robust to see its behavior, since the authors did not provide a way to estimate p* and p** and, in practice, the user would need to inform this values.#Minor changes ##########################################################Introduction: There is no mention about the proposal model in the introduction or why is necessary a new approach to deal with ability estimation in presence of response disturbance . It’s not clear whether there are shortcomings in the current models used to address the problem.Line 84: It’s quite strong to state that anyone can hinder the development of a statistical model for a quarter of a century.The follow mathematical equations/symbols are misconfigured: 6 to 7 are (lines 119 – 125); equations 9 to 10 (lines 164 to 167); equation 17 (line 422); line 424 – 425, line 496 - 500.Line 161: “r, and”?Line 171: “4:0” = 4.0Line 172: If B =0 instead of If r_i = 0?Line 252 and Line 286: d = γ? Keep the same notation throughout the manuscript.In Table 2 (line 325), Table 3 (Line 382), Table 4 (line 424) the bias is provided, I would recommend also include the root mean square error (RMSE).Line 402: the value -0.128 is considered statistically significant, please provide further information about that statement. That is, did the authors perform any statistical test?Line 433: The statement “are below zero, and their absolute values are larger than 0.100” is not correct, according to Table 4.Line 437: The statement “and below -0.500” is not correct, according to Table 4.Line 459: the word “certainly” may imply in a too strong statement.**********what does this mean?). If published, this will include your full peer review and any attached files.6. PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool,  25 Jan 2021Response to ReviewersPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Review Comments to the AuthorReviewer #1: I have reviewed the manuscript 'The Robust Estimation of Examinee Ability Based on the Four-parameter Logistic Model When Guessing and Carelessness Responses Exist'. Although this manuscript addresses an interesting topic, I have detected (what for me are) several important limitations.* I am not a native English speaker. I usually need the assistance of a professional translator with my own manuscripts. Having said that, I have found that English level below what could and should be expected. For instance, I'm not sure about what this is suppossed to mean: "According to their investigation, the Huber estimation, rather than the Biweight estimation, should be applied when reducing sampling variability was prior to reducing biases."Response：" According to their investigation, the Huber estimation, rather than the Biweight estimation, should be applied when reducing sampling variability was prior to reducing biases." The sentence is the original views of Schuster and Yuan (2011).* The structure should be improved. For instance:- "Waller (1974) and Wainer and Wright (1980) proposed approaches to the robust estimation of latent ability, including traditional correction for guessing, Jackknife, AMT-Robustified Jackknife, and WIM. AMT-Robustified Jackknife was generally considered the best and most efficient method for tests comprising forty or fewer items[2]." The authors here list several alternatives (from rather old sources) that are never described. Why do the reader need this apparently irrelevant information? Why, if the AMT-Robustified Jackknife was considered the best alternative, is not used in the present study?Response：(1) The author just expounded the history of the researches about several alternative robust estimations in the introduction. (2) The AMT robustified jackknife is just the better one of the several previous methods. AMT robustified jackknife is only suitable for short tests. Moreover, the AMT robustified jackknife method is more complex, and the meaning of the AMT robustified jackknife is not exact. After the Biweight estimation was proposed, researchers do not use the AMT robustified jackknife method.* Much of the details of the estimation approach described in the simulation studies should be placed in the Introduction, where those models are supposed to be described.Response： It is revised in the new manuscript. The details of the estimation are already placed in the Introduction.* In the section 'Recent literature on the 4PLM', the authors should clearly differentiate the results from computerized adaptive testing from those from non-adaptive applications. Up to now, those two lines of research are combined.Response：It is revised in the new manuscript.* Authors duplicate information. For instance, r = a(theta - b) can be found in page 4, 6, 9...Response：The two sentences are deleted in the new manuscript. Those are:The sentence on page 4 is deleted : Let ; thus represents the distance between subject and item in the units of the standard deviation of item .The sentence on page 6 is deleted: indicate the residual between estimated ability and item difficulty. * Equations in page 7 cannot be properly read, as some elements have been moved.Response：It has been all revised in the new manuscript.* From my point of view, authors should not use BILOG to estimate trait levels. Or, at least, they should provide evidence that their Visual Basic routines, when programmed for 2PLM, BIW, and 3PLM, lead to the same results as those from BILOG. Otherwise, we don't know if the differences can be attributed to software differences or to the estimation methods in themselves.Response： When programmed for 2PLM, BIW, and 3PLM using the Visual Basic program, it is found that the ability estimation results using the Visual Basic program are the same as those using BILOG. Subject# Responses Visual Basic Program /BILOG Visual Basic Program 2PLM-MLE BIWB=4 3PLM-MLE 4PLM-MLE 4PLM-Robust HuberH=11 1111100000 0.061 0.048 -0.221 -0.133 0.068 0.0522 1111110000 0.763 0.710 0.538 0.627 0.724 0.7113 1111101000 0.763 0.746 0.436 0.494 0.762 0.7114 1111100100 0.763 0.653 0.262 0.131 0.206 0.6665 1111100010 0.763 0.421 0.061 -0.060 0.096 0.3466 1111100001 0.763 0.163 -0.080 -0.114 0.073 0.2577 1111000000 -0.755 -0.726 -0.999 -0.886 -0.709 -0.7208 1110100000 -0.755 -0.798 -1.239 -1.352 -0.684 -0.7209 1101100000 -0.755 -0.617 -1.437 -1.817 -0.479 -0.43510 1011100000 -0.755 -0.160 -1.570 -0.535 -0.022 -0.22411 0111100000 -0.755 0.048 -1.642 -0.189 0.056 -0.146* Authors should justify why they are using maximum-likelihood and not any of the Bayesian alternatives, as expected a posteriori with a uniform prior.Response： The equation of expected a posteriori estimation is shown as  (1)And , , is called a "node." Each node has an associated weight that takes into account the height of the density function in the neighborhood of , and the width of the rectangles. The values of , and are found by solving a set of equations that involve the continuous distribution to be approximated and the specified number of nodes.  However, in Biweight estimations, the weight function is shown as  (2)How did the add to the equation (1) is still to be discussed to confirm, thus the Biweight estimations using the expected a posteriori with a uniform prior is in doubt. so is the Bayesian alternatives using the expected a posteriori with a uniform prior.So the ability is estimated only by using maximum-likelihood estimation in the manuscript.And, more importantly:* I really don't know what can be learnt from the 'Example' study. If there is no correct theta level, what are the readers expected to learn from this? The authors used clearly incorrect models (that would have never been estimated if the correct model was 2PLM), as the 3PLM with c = .20. What is the point of doing that? Isn't obvious that using an incorrect model can distort trait level estimates?Response：(1)the 'Example' study in the manuscript is similar to the 'Example' in Table 1 in the Schuster and Yuan (2011), and is also similar to the 'Example' in Table 1 in the Mislevy and Bock (1982).The authors designed that the response pattern of subject #1 is designated as the referenced pattern. His estimated ability is 0.061 using the 2PLM-MLE, which is perceived as the referenced ability value.The item difficulty of the 10-th item  is greatly smaller than the ability of Subject #6, but Subject #6 gives a correct response to the 10-th item by guessing probably and gets a same ability value as Subject #2. So there is a guessing response probably, and the ability estimation of Subject #6 is not robust.The item difficulty of the 1-st item  is greatly larger than the ability of Subject #11, but Subject #11 gives a wrong response to the 10-th item because of carelessness or other error probably and gets a same ability value as Subject #7. So there is a carelessness response probably, and the ability estimation of Subject #11 is not robust.In summary, the 'Example' can show the guessing and carelessness responses, and the authors try to compare the several robust estimation methods.Table 1. The Biases of the Ability Estimates of the Five ApproachesSubject# Responses Visual Basic Program /BILOG Visual Basic Program 2PLM-MLE BIWB=4 3PLM-MLE 4PLM-MLE 4PLM-Robust HuberH=11 1111100000 0.061 0.048 -0.221 -0.133 0.068 0.0522 1111110000 0.763 0.710 0.538 0.627 0.724 0.7113 1111101000 0.763 0.746 0.436 0.494 0.762 0.7114 1111100100 0.763 0.653 0.262 0.131 0.206 0.6665 1111100010 0.763 0.421 0.061 -0.060 0.096 0.3466 1111100001 0.763 0.163 -0.080 -0.114 0.073 0.2577 1111000000 -0.755 -0.726 -0.999 -0.886 -0.709 -0.7208 1110100000 -0.755 -0.798 -1.239 -1.352 -0.684 -0.7209 1101100000 -0.755 -0.617 -1.437 -1.817 -0.479 -0.43510 1011100000 -0.755 -0.160 -1.570 -0.535 -0.022 -0.22411 0111100000 -0.755 0.048 -1.642 -0.189 0.056 -0.146Note. 3PLM-MLE indicates c=0.20 for items 1~10 using the 3PLM.4PLM-MLE indicates c=0.20 , =0.95 for items 1~10 using the 4PLM. 4PLM-Robust indicates 4PLM-Robust .* With respect to the simulation studies, it was difficult for me to correctly evaluate them, as the provided description is very difficult to follow. If the new method is correctly described in the Introduction, the description of the design of those studies could be greatly simplified.Apparently, the authors have found that the parameters are better recovered when the model used to generate the data is also the model used to recover data. Perhaps I'm missing something relevant, but, again, this seems as obvious.Response：Firstly, the model used to generate the data of the guessing and carelessness response disturbances is also used or acknowledged by the researchers in the past year. Secondly, the 4PLM-Robust estimation and several other robust estimation methods are compared by the same simulation data.Thirdly, the form of the 4PLM-Robust estimation is only similar to the test data generated model, but not the same. The 4PLM-Robust estimation and the test data generated model originate in the same Production principle about guessing and carelessness response. So, the form of the 4PLM-Robust estimation is only similar to the test data generated model. * I would greatly recommend this article:Green, B. F. (2011). A comment on early student blunders on computer-based adaptive tests. Applied Psychological Measurement, 35(2), 165-174.There some very pertinent points about this area of research were raised. I would not repeat them here. Response：Green’s research is cited in the new manuscript. In Green’ views, there are rare errors on middle difficulty items for very proficient test-takers, and the so-called bias is one of selective data analysis. The authors believe that whether the score penalty for very proficient test-takers is statistically significant, the faultiness of underestimating them in the Rasch model or 2PLM in the test has been found, and the researchers should admit it but not overlook it. Furthermore, the reasons and methods should also be found in different research angles.Reviewer #2: The paper introduces robust estimation method for the four-parameter logisitic model (4PLM) to account for guess and careless responses. While I think I could understand the premise and purpose, I am afraid I became lost quite early either due to mistakes, or lack of clarity in derivations and reasoning. I think the paper would benefit from being much clearer about what is new here, and what others have done, but also please have a close look at formatting, potential errors in equations and maybe notations. Have some comments below that could perhaps help start this, and if I wrong with these then perhaps just more clarity is needed.Main comments1. Formatting of (5), (6) and (7) needs fixing. It is difficult to determine what the equations should be.Response：The sequence numbers of the following mathematical equations are misconfigured, including Equation (4), (5), (6) and (7). They are all revised in the new manuscript.2. On line 117: why is P(ri) now exp(ri)/(1 - exp(ri))? Isn't is exp(ri)/(1 + exp(ri)) since it is a probability. Then line 119 onwards as well do maybe I am missing something here. With the formatting issues for (5), (6) and (7) I can't tell what is happening here, and whether this makes a difference.Response： “exp(ri)/(1 - exp(ri))” is a mistake in the original manuscript. It is revised in the new manuscript. Line 119 onwards，the authors aimed to explain that 4PLM-Robust estimation originates in 2PLM and can be simplified as 2PLM. In other words, 2PLM is one special case of 4PLM-Robust estimation. so in the new manuscript，it is revised as： , (8)where and . If guessing and carelessness response disturbances don’t occur simultaneously on the -th item，then and . If and , then and in Equation 8. If and , 4PLM-Robust estimation becomes 4PLM. That is, 4PLM-Robust estimation originates in 2PLM and can be simplified as 2PLM. In other words, 2PLM is the special case of 4PLM-Robust estimation.3. I got lost with some of the notation, which had me second guessing at times. E.g. what is eta_i? E.g. in (5) isn't is just the residual ri?Response：They are all revised in the new manuscript. indicates the residual between estimated ability and item difficulty. in Equation (5), (6), (7) are deleted. (5) (6) (7)4. In the two sections starting Line 105, it wasn't clear to me what was new to this paper, and what has previously been introduced by others. Is it just (8)?Response：The authors aimed to explain that 4PLM-Robust estimation originates in 2PLM and can be simplified as 2PLM. That is, 2PLM is the special case of 4PLM-Robust estimation.  And more, the Biweight and Huber robust estimation are introduced in Line 163-178 in the original manuscript (Line 192-217 in the new manuscript). The Biweight and Huber robust estimation can be simplified as 2PLM according to Equation 5. In other words, 2PLM is also the special case of Biweight and Huber robust estimation. Minor (example minor points)The paper needs to some careful attention to grammatical errors and formatting issues. I have some examples below.While 3PLM and 4PLM are defined in the abstract, they also need to be defined in the intro before use. 2PLM also needs to be defined.Response： It is revised in the abstract .Shouldn't rv on line 62 also be indexed by i? Later, ri used to indicate item i and an arbitrary subject.Response： is revised to line 106: "approaches" should "approach"Response： It is revised.Eq 4: uj?Response： It is revised to . (4)line 117: Stat of new sentence, "let" should be "Let" and need to end the sentence after Eq. 4. Response： It is revised.Line 119: new sentence, "if" to "If"Response： It is revised.Line 119 onwards: again exp(ri)/(1 - exp(ri)) so maybe I am missing something hereResponse：In Line 119 in original manuscript, is revised to Reviewer #3: The manuscript addresses the problem of estimate the ability parameter in the presence of response disturbance, such as guessing, cheating, careless and transcription errors. The authors propose a new model that would be robust in the presence of that kind of occurrence and compare its results with the standard models present in Item Response Theory literature: 2PLM-MLE, 3PLM-MLE, the 4PLM-MLE, the Biweight estimation and the Huber estimation. It is worth mentioning that provide accurate ability estimation even in the presence of disturbance is a core matter in education assessment. The article is able to present the methodology and results quite clear. Therefore, I would recommend the publication if the following changes and some clarifications are met:#Major changes ###########################################################The major shortcoming of the manuscript is that the simulation study does not provide a fair comparison between the models. That is, it assumes that the items parameters are all known in advance but, for some models, it does not match with the true generated data. Indeed, the best results are obtained, in each case, as expected, by the model that matches with the true generated data. For example, is assumed in advance that de c parameter in the 3PL-MLE model follows a normal distribution N. That is, the c parameter will almost surely be located between 0.17 and 0.23. Nevertheless, in simulation Case II the c parameter of the true generated model is always zero. In a real situation, the items parameters are obtained previously through a pre-test and c parameter could be near zero in 3PL-MLE. Indeed, the parameters of each item are supposed to represent the empirical reality. I would recommend that, for each Case, the authors generate two data sets. The first data set would be used only to estimate the items parameters for each model. Therefore, these items were pre-tested. Then, you can use the second data set to estimate the ability parameter using the items parameters previously obtained. This would reproduce what actually occur in practice. Anyway, this is only a suggestion; the important point is to provide a fair comparison, using this solution or another.I also suggest to include a simulation (Case IV) in which the true generated model follow exactly the 3PL-MLE or the 4PL-MLE to see the behavior of the other models in that situation. Finally, I recommend to include a standard 4PLM-Robust in all simulation Case, for example 4PLM-Robust to see its behavior, since the authors did not provide a way to estimate p* and p** and, in practice, the user would need to inform this values. Response：(1) In the original manuscript, the item parameters are known and are not changed in the three simulation cases. The design of the original manuscript is similar tothat of Schuster and Yuan (2011) , and Mislevy and Bock (1982).If the item parameters are changed for each case, then the item parameters in different test cases will likely influence examinees’ ability. Then, it will be more complicated for the simulation study. (2)If c parameter in the 3PL-MLE model follows a normal distribution N in the test simulation, thenthere will be another different simulation way of guessing responses generating model in the test.  Firstly, as we know, if the examinee knows the answer to the item or is sure to answer correctly , then the examinee will not guess the answer of the item randomly. It is be defined as Guessing Principle here.Secondly, in the manuscript, random guessing is generated by the following model, Equation (12) : (12)The generation model, Equation (12) tallies with the Guessing Principle.In another generating model of guessing responses in 3PL-MLE model, for all subjects, including the high-ability subjects, the expected probability of correct responding will include the probability of randomly guessing, 0.20 or 0.25. So 3PL-MLE model does not tally with the Guessing Principle.(3) if the true generated model follows exactly the 3PL-MLE or the 4PL-MLE in the test, the design and the aim of the manuscript will be complicated and blurry, and the first aim will becomea comparison between the two different generation models of guessing responses. Moreover, there is few common points between the two different generation models. (4) In Table 2 and Table 3，the simulation data about 4PLM-Robust is added.#Minor changes ##########################################################Introduction: There is no mention about the proposal model in the introduction or why is necessary a new approach to deal with ability estimation in presence of response disturbance . It’s not clear whether there are shortcomings in the current models used to address the problem. Response： The generated model about the carelessness error in the Schuster and Yuan (2011), , is accurate. For instance,for the low-ability examinees, if the response probability and , the probability will be , that is, . As is known , is paradoxical and illogical. The generated model about the carelessness error in the Schuster and Yuan (1982) only includes the high-ability examinees. In the article, the authors believe that the generated model about the carelessness error is proposed according to the Principle about Guessing and carelessness error, and so should the Robust estimation. Line 84: It’s quite strong to state that anyone can hinder the development of a statistical model for a quarter of a century.Response：It has been revised. Perhaps, their comments influence the research. The research on the 4PLM stagnated for a quarter of a century.The follow mathematical equations/symbols are misconfigured: 6 to 7 are (lines 119 – 125); equations 9 to 10 (lines 164 to 167); equation 17 (line 422); line 424 – 425, line 496 - 500.Response： All has been revised.Line 161: “r, and”?Response：“ , and“ is deletedLine 171: “4:0” = 4.0Response：It is revised.Line 172: If B =0 instead of If ri = 0?Response： B is a divider in Equation 10, so B can’t be 0. Line 252 and Line 286: d = γ? Keep the same notation throughout the manuscript.Response：It is revised.In Table 2 (line 325), Table 3 (Line 382), Table 4 (line 424) the bias is provided, I would recommend also include the root mean square error (RMSE).Response： There is a small difference between the root mean square error(RMSE) of the five examinees, and it is not statistically significant. Line 402: the value -0.128 is considered statistically significant, please provide further information about that statement. That is, did the authors perform any statistical test?Response：The word “statistically significant“ is revised to “relatively larger”.Line 433: The statement “are below zero, and their absolute values are larger than 0.100” is not correct, according to Table 4.Response： The sentence “ Simultaneously, all the biases of the 2PLM-MLE at high ability ( =2) are below zero, and their absolute values are larger than 0.100. “ is deleted in the revised vision.Line 437: The statement “and below -0.500” is not correct, according to Table 4.Response：“and below -0.500” is revised to “and below -0.350” Line 459: the word “certainly” may imply in a too strong statement.Response：“certainly“ is revised to “probably“________________________________________AttachmentResponse to Reviewers 22.docSubmitted filename: Click here for additional data file. 16 Feb 2021PONE-D-20-32376R1The Robust Estimation of Examinee Ability Based on the Four-parameter Logistic Model When Guessing and Carelessness Responses ExistPLOS ONEDear Dr. Jian,Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.Please submit your revised manuscript by Apr 02 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocolsIf applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: We look forward to receiving your revised manuscript.Kind regards,Lianmeng JiaoAcademic EditorPLOS ONEAdditional Editor Comments (if provided):The reviewers still have critical comments for your work. Please revise the paper by addressing all the comments.[Note: HTML markup is below. Please do not edit.]Reviewers' comments:Reviewer's Responses to QuestionsComments to the Author1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.Reviewer #3: (No Response)**********2. Is the manuscript technically sound, and do the data support the conclusions?The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #3: No**********3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #3: No**********4. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #3: Yes**********5. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #3: Yes**********6. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #3: The manuscript was revised and improved in many aspects, however the major changes required was not properly addressed. The major shortcoming of the manuscript is that the simulation study does not provide a fair comparison between the models. Using 3PLM-MLE and 4PLM-MLE the parameters c and γ was considered known, but it doesn’t match with the true generated data. For example, in simulation case 2, γ=1-p** for the 4PLM-Robust, which matches the true generated data, but it was set to γ=0.97 for 4PLM-MLE, regardless of the true generated data. Since c and γ are parameters that should be estimated, doesn’t seem to be fair to assume a wrong value to them when it’s possible to inform the true value. In the previous example, γ should be 1-p** for 4PLM-MLE.The manuscript is very interesting in the sense the present the 4PLM-Robust model, which seems to be more flexible to accommodate many patterns of response disturbance. However, the article does not contain simulations to verify that it is robust enough when the disturbance pattern does not follow its assumptions. For example, a simulation case in which the true generated model follow exactly the 3PL-MLE or the 4PL-MLE to see the behavior of the 4PLM-Robust. Therefore, the main conclusion is that the best model is the one that corresponds to the true generated-response model, as should be expected.#----------------------------------------------------------------------------------------------------------------------------------Minor changes:Introdution – There is no mention to the proposed method.Page 7 (lines 122-130) ,9 (line 180), 10 , 26 (line 446) – The equations are misconfigured.Line 198: donot = do notLine 247: Replace smaller for larger ?Line 250: Replace larger for smaller?Line 257: “The response of Subject #6 most seriously deviates from the referenced pattern of subject #1”. Is not the opposite?**********what does this mean?). If published, this will include your full peer review and any attached files.7. PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool,  29 Mar 2021nothingAttachmentResponse to Reviewers 3.25.docSubmitted filename: Click here for additional data file. 5 Apr 2021The Robust Estimation of Examinee Ability Based on the Four-parameter Logistic Model When Guessing and Carelessness Responses ExistPONE-D-20-32376R2Dear Dr. Buyun,We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at onepress@plos.org.If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact Kind regards,Lianmeng JiaoAcademic EditorPLOS ONEAdditional Editor Comments :Reviewers' comments: 12 Apr 2021PONE-D-20-32376R2 The Robust Estimation of Examinee Ability Based on the Four-parameter Logistic Model When Guessing and Carelessness Responses Exist Dear Dr. Buyun:I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. onepress@plos.org.If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact plosone@plos.org. If we can help with anything else, please email us at Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staffon behalf ofDr. Lianmeng Jiao Academic EditorPLOS ONE