Systematic reviews (SRs) are cornerstones of evidence-based medicine and have contributed significantly to breakthroughs since the 1980’s. However, preclinical SRs remain relatively rare despite their many advantages. Since 2011 the Dutch health funding organisation (ZonMw) has run a grant scheme dedicated to promoting the training, coaching and conduct of preclinical SRs. Our study focuses on this funding scheme to investigate the relevance, effects and benefits of conducting preclinical SRs on researchers and their research.We recruited researchers who attended funded preclinical SR workshops and who conducted, are still conducting, or prematurely stopped a SR with funded coaching. We gathered data using online questionnaires followed by semi-structured interviews. Both aimed to explore the impact of conducting a SR on researchers’ subsequent work, attitudes, and views about their research field. Data-analysis was performed using Excel and ATLAS.ti.Conducting preclinical SRs had two distinct types of impact. First, the researchers acquired new skills and insights, leading to a change in mindset regarding the quality of animal research. This was mainly seen in the way participants planned, conducted and reported their subsequent animal studies, which were more transparent and of a higher quality than their previous work. Second, participants were eager to share their newly acquired knowledge within their laboratories and to advocate for change within their research teams and fields of interest. In particular, they emphasised the need for preclinical SRs and improved experimental design within preclinical research, promoting these through education and published opinion papers.Being trained and coached in the conduct of preclinical SRs appears to be a contributing factor to many beneficial changes which will impact the quality of preclinical research in the long-term. Our findings suggest that this ZonMw funding scheme is helpful in improving the quality and transparency of preclinical research. Similar funding schemes should be encouraged, preferably by a broader group of funders or financers, in the future. Keeping up to date with health/medical the literature can be challenging due to the vast number of new articles published every year. Scholarly peer-reviewed journals produce over 3 million articles annually, with a 56% increase since the last decade ,2. ConseInterest in preclinical SRs is slowly growing within academia and amongst other stakeholders –16. SincWhile the benefits of clinical SRs are beyond doubt, the impact of conducting a preclinical SR on researchers and their research field remained unknown. Hence, to investigate these effects, we evaluated the impact of ZonMw funded preclinical SRs on researchers and (their) research.Our study aims to assess the impact of conducting preclinical SRs on researchers, their research and their field through the following objectives:What impact does conducting a preclinical SR have on a researcher in terms of planning, conducting, reporting and appraising their research projects?What impact does conducting a preclinical SR have on a researcher’s views about research, their own research, and their field more generally?What impact does preclinical SRs have on preclinical research in general ?https://www.zonmw.nl/nl/actueel/nieuws/detail/item/systematisch-literatuuronderzoek-vervangt-vermindert-en-verfijnt-proefdieronderzoek/.This project is a research impact case study aiming to evaluate the impact of an intervention  on individuals (researchers), products (their research), and environment (their research field). Therefore, it relies on theories and knowledge of research impact assessment. An in-depth description of our theoretical framework can be found in the original ZonMw internal report made for this case study, available at: Briefly, research impact assessments rely on the evaluation of research impacts, defined as “contributions that research makes to the economy, society, culture, national security, public policy or services, health, the environment, or quality of life, beyond contributions to academia” . These eOverall, research impacts will lead to demonstrable and beneficial changes in behaviours, beliefs, and practices . ConsequThe research impact framework was developed in 2007 by Kuruvilla et al., as a checklist to guide researchers in selecting and evaluating the impacts of their work and interventions . It highThe behaviour wheel of change created by Michie et al., in 2011 presents the idea that behavioural change occurs as a result of good pre-dispositions, interventions, and policies (which enable or support the particular intervention and are created [in part] by responsible authorities)  See S2 . These gAs a result, we hypothesise a pathway by which our intervention  in a set context  would create impacts (for researchers and their research) .This study complies with the standard for reporting qualitative research and the consolidated criteria for reporting qualitative research ,33. ChecThe primary investigator is a research assistant with a Master’s Degree in Science, Innovation and Management (Radboud University). She has experience in preclinical SRs and qualitative research via her Master’s Degree and two long-term Master’s internships. She had no relationship with any of the participants and had not taken part in the training or coaching provided by the funding scheme being studied, although she had taken part in one of the workshops. Participants were not aware of her personal goals or personal motivation to conduct this research project.Her interest in preclinical SRs and qualitative research led her to conduct this research project. She was supported and/or supervised by a research fellow with a background in ethics and philosophy, a professor in Evidence-Based Laboratory Animal Science and a ZonMw program manager, all of whom helped create a robust framework for the project.This study uses a mixed-method approach combining questionnaires and semi-structured interviews. We focused on researchers who had followed ZonMw’s workshops and who (had) received coaching to perform their own preclinical SR. This enabled us to get a snapshot of the preclinical SR field and fairly evaluate the experience of these researchers within a given time period.We used purposive sampling to select our participants. The target population for the questionnaires comprised researchers who had participated in a ZonMw workshop and who had either started (currently conducting or prematurely stopped) or completed a preclinical SR with funded coaching. Those who had completed their SRs were approached to take part in an interview to evaluate the impact of the SR on their subsequent research.Participants were recruited by e-mail and received a reminder two weeks after the first invitation , with the online questionnaires being available for one month (23/07/2020–23/08/2020). Data analysis was performed for a period of almost one month (24/08/2020-18/09/2020).https://www.questionpro.com/), a free questionnaire platform. Our questionnaire consisted of both closed  questions and open-ended questions. For rating questions, seven-point Likert scales were chosen , with the extremities being “completely disagree” and “completely agree”  Based on the findings of this study, I’d use the most robust models that will result in the reduction of the use of animals. I’m also confident that people who read this article will employ a similar approach and modify the design of their studies, thus overall, I think this study will have a big impact in the field”–Respondent 76823076Of the five people who performed research projects using non-animal methods, two affirmed that their SRs had positively influenced their choice of approach. The other three had already decided to use a non-animal approach prior to conducting their SRs. For the six who performed clinical research projects, three stated that the SR had impacted their clinical research; one with respect to the questions they chose to address, another in terms of translating preclinical to clinical research, and the third in terms of highlighting the need to investigate the role of a specific mutant phenotypeAll participants answered questions about appraising research, regardless of the stage of their SRs. These questions aimed to determine the effect of conducting a preclinical SR on the way they conducted appraisals.Overall, the results were equal across groups . FurtherParticipants from both groups stated that they gained new skills or improved existing skills in the process of conducting their preclinical SRs. Some of these skills are exclusive to research , while others go beyond research . We identified and categorised four main types of skills: 1) research skills directly linked to the SR stages, 2) Research skills for planning/conducting subsequent research, 3) Critical appraisal, and 4) Interpersonal skills .Reviews (and reviews steps) often took longer than expected (26/30 participants for the completed group and 8/15 participants for the ongoing group). For instance, researchers often experienced the number of studies during screening, data extraction or data analysis as overwhelming, but also suffered in getting their protocols, manuscripts or revision for publication reviewed. Some had to wait for second or third assessors/reviewers and experienced delays when asking authors for further information. Also, three participants (both groups combined) mentioned the COVID-19 pandemic as a delaying factor. Even though many participants experienced extensive delays while conducting their reviews, most considered it likely that they would conduct another SR or advise colleagues and peers to conduct a preclinical SR. Numbers wise, 34/46 participants mentioned that they would be likely to conduct another SR (18 “yes” and 16 “maybe”), while 45/46 would potentially recommend colleagues to do their own SRs (37 “yes” and 8 “maybe”). Eight participants from the completed SR group had already conducted a second SR after completing their first coached review, illustrating how useful they considered this. Additionally, in the SR completed group, 20/28 participants reported that they would like to receive similar coaching for their next review. Fifteen of these wanted a more personalised approach, with a focus on certain stages of the SR, while 5 wanted coaching for the whole process.The interviews afforded greater insight into the sort of impact that conducting a preclinical SR has on both researchers and their research, as well as how this happens. We identified that impacts occurred in two steps and influenced three given levels .First, impacts occur at the level of the researchers; conducting the SRs contributes to an evolution in participants’ thinking, providing them with important insights and skills. This first step leads researchers to step back from their usual point of view and triggers the realisation of specific issues and shortcomings in their own work, research field, and science community.lab level), 2) regarding how they appraise their field and advocate for change (field level), and 3) promoting change on a broader scale (science community level).Second, impacts on research can occur now that a new mind-set has developed. Researchers actively modify their activities 1) relative to their own work  increased awareness of their own field, 2) identification of data gaps and topics for new primary studies, 3) discovery of heterogeneity and incoherencies, and 4) discrediting or confirming theories.“You retrieve papers that you were not aware of even though you’re quite familiar with literature on the topic, you will always find new things”–ZonMw80“I really have more, much more transparent ideas about what studies are missing, and need to be done”–ZonMw23“What was striking for this model is that everybody in the field was using the model in a different way.”–ZonMw84“We did encounter unexpected findings, which we couldn’t even explain why it was the case. And it really contradicts some basic theories of (field of interest), of how our intervention works.”–ZonMw 91Overall, participants were convinced about the value of preclinical SRs, and were positive about their experiences with the funding scheme including the coaching. Moreover, many participants admitted that they had underestimated the difficulty of the SR process and were glad to have received proper guidance.“I had underestimated really the efforts needed to make a strong and a good systematic review”–ZonMw72“I really felt I needed this help”–ZonMw23Research impacts were identified within the participants’ teams and labs and were categorised as follows: 1) direct impacts on experiments, planning, conducting, reporting, appraising research; execution of other types of research, switching fields, conducting meta-research, and 2) advocacy for better research within their teams, promoting networking and collaboration.The first category of impact was frequently mentioned in the questionnaires. Similar outcomes were reported in the interviews, namely that the planning, conduct and reporting of research projects was performed better after (or while) conducting the SR. The participants were unanimous about the change and progress they experienced thanks to their review:“I was just much more mindful about the blinding, randomisation, the sources of bias. We put enormous amount of efforts into doing that properly”–ZonMw91“A lot of details would be missing if I hadn’t done this (systematic review), cause now I know how much is missing in the studies”–ZonMw23Aside from doing new experimental research differently, participants’ SRs fuelled ideas for other types of studies in meta-research and non-animal approaches. For some of them, the SR was a game-changer that led them to switch fields. Some now pursue their career in human research, some in meta-research  and some have distanced themselves from academia to work towards improving the system:“Step by step, you can do a lot of research in humans that is also very helpful. I did a lot of animal experiments and yeah (pause). I’m not going to do that anymore; I’m not going back to animal experiments”–ZonMw84“I realised then that open science is what I’m interested in. That’s also why I wanted to switch to a different working environment. Not doing research myself anymore that much but actually working on trying to improve the system”–ZonMw57The second category of impact emphasises our participants’ enthusiasm about sharing their newly acquired knowledge with colleagues and peers. Some passed on their knowledge by teaching their new skills to students or by advising colleagues on how to improve their research, as the following quotes show:“I had this little extra time to teach students how to do it, and it becomes part of their package as well. Because in reality there’s nothing about this type of science in education, and that’s such a shame. Experimental design, statistics, and so, it’s completely missed in education, and I feel that’s what I want to teach my students, how to do good science”–ZonMw91“I also used this to give presentations and make my colleagues at the department aware of how they should perform animal experiments. So, I try to implement this guideline of ARRIVE also in our department.”–ZonMw84However, such improvements at a team or individual level may require changes from higher up in organisations. Several interviewees mentioned how supervisors’ lack of familiarity with SRs could impede their execution and the transfer of skills and knowledge within a team. The fact that some professors and supervisors realised the value of SRs by the time they had completed the SR does, however, provide hope.“I have an assertive personality. […] But many other people that are not as assertive might just not do a systematic review in the first place because of how many criticisms there is from the more managerial position in the department. Meaning professors themselves don’t understand, then why should a PhD student do it?”–ZonMw91Resistance to change remains in the lab, both towards acceptance of preclinical SRs but also to other new methods that can improve research quality. Lab conventions, resistance from colleagues and supervisors, time pressure, funding and competition were the main obstacles experienced by the participants.The two important ways by which our participants impacted their field with their preclinical SR were 1) by dissemination of their findings and 2) by promoting the value of SRs to improve the quality of research, for example, by writing an opinion paper.The usual way of disseminating results is via publications and conferences. Interviewees’ experiences highlighted that getting preclinical SRs published can be challenging. Some participants had difficulties finding journals that would accept their SRs or, once found, faced repeated rebuttals. Moreover, the opinions of peer reviewers were quite influential in the editorial process. Responses about peer reviewers’ input were mixed; some participants had a generally negative experience with peer reviewers who did not understand the SR process or the value of SRs, while others received beneficial input that improved their SRs:“This is I think the biggest frustration of the whole process; that the journals and the reviewers really underestimate the significance, the work, that is in it.[…] they said no ‘you have to do a letter to the editor, it’s not worth an original article’–ZonMw23This dissonance directly echoes the lack of acceptance, support and education regarding preclinical SRs as mentioned earlier in “Step two. impact on research (lab level)”. Nevertheless, once in the open, preclinical SRs can influence a field see . Of courEXAMPLE 1• A participant discovered that the mode of drug administration significantly impacted the results of an intervention, both in clinical and in preclinical experiments. With this issue identified, there could be a change in how this particular intervention is administered across the whole field.“With regard to the patient’s studies, was that everybody just depleted these XX and saw what happened to XX, and then said, ‘oh XX are bad’ or ‘oh XX are good’. But our review made I hope, make people realise when it gets published that it totally depends onhowyou administer your agent to deplete your XX. So, you can’t really say ‘oh they are good’ or ‘they are bad’ because it depends totally on how you do it.” – ZonMw23”EXAMPLE 2• A participant’s SR triggered a whole series of new SRs in the same direction as their own; as a result ofShowin the value and insight a preclinical SR can provide, other researchers in their field began performing their own preclinical SRs.“People really loved the paper and found it really informing to have a first good overview. And then, multiple research groups actually ‘copy pasted’ our paper, but put another cell type than the cell type we used […] for the research field it was good, because everybody started to systematically assess all the evidences for certain cell lines”- ZonMw15To press for change, certain participants chose an innovative route: some wrote opinion papers or letters to reach peers and editors, while some others took the matter a step further by creating tools  specific to their field.“We wrote a paper in a XX journal for animal technicians, and describing all the differences that everybody does with the model. […] also concerning housing and all kinds of things that are important for animal studies. […] I also wrote a paper for the XX organisation, about the use of animals and the 10 sorts of pitfalls, mistakes about experimental models”–ZonMw84Impacts at a broader level were also identified, i.e., increased awareness and acceptance of preclinical SRs, which could potentially improve the overall quality of preclinical research. These impacts included 1) the influence of stakeholders at a higher level and changing the status quo, 2) increasing training for researchers, and 3) improving and investing in the education of the next generation of researchers.Observing that their SR findings did not bring about changes in their field, certain participants would have liked to pursue the issues at a higher level, for example by involving ethical committees and funders. It was suggested that preclinical SRs could become a requirement prior to performing an animal study and that this could be directed/checked by animal ethical committees. Funders could also be involved by emphasising the value of grant applications that include systematic overviews.As noted above, questionnaire respondents were likely to recommend SRs to their colleagues and peers. This observation was confirmed in the interviews, with participants recommending that their peers or colleagues should receive proper training  in the same way that they had. Participants made many suggestions about promoting preclinical SRs in their field, including advertising on websites, in universities, on social media and at conferences, and by providing financial support. It was suggested that more investment in education would be beneficial in terms of promoting the value and acceptability of SRs, since they do not appear to be sufficiently understood nor employed in many fields.Finally, participants reflected on the younger generation of scientists, suggesting that preclinical SRs and related training would be beneficial for new researchers.“I think training is most powerful, because it’s difficult to change the behaviour of the old researchers […]. But the young researchers you can definitely train, they can become knowledgeable in this field.”–ZonMw72Our findings have highlighted how conducting preclinical SRs impacted both research and researchers as a result of the ZonMw funding scheme, MKMD “Knowledge infrastructure” module. Both questionnaire and interview data indicated that conducting preclinical SRs provided researchers with awareness, skills, and insights that 1) were used and promoted in the planning, conducting and reporting of their subsequent projects, 2) influenced their critical stance when appraising research, and 3) were used and promoted in their research fields . Ultimately, these impacts could lead to more transparent, high-quality research. To our knowledge, this is the first time qualitative research has been employed to identify the research impacts of preclinical SRs.At a lab level, we saw that conducting preclinical SRs not only provided insights and skills but also contributed to long-lasting behavioural changes amongst our participants. Furthermore, most participants were willing to improve their work and advocate for change in their teams (via support and teaching). We might extrapolate that if properly supported, coached, and empowered, a significant number of researchers might re-evaluate their research practices and follow the same path of reflection as our participants. This could lead to more teams wishing to improve knowledge transfer within their labs and among students, functioning a SR feedback loop. Moreover, if universities invest in teaching about SRs and meta-analyses ), as suggested by our participants, this type of knowledge would become more universally accepted and might increase standards and expectations within the entire preclinical field.However, knowledge transfer both within and between teams depends on several factors and antecedents, which can both promote and impede the dissemination of knowledge . A legalReturning to the behaviour wheel of change framework, with both legislation and intervention in place, only the predispositions need to be set to trigger change. But as highlighted in the interviews, several factors currently hinder the adoption and acceptance of preclinical SRs, including standard lab conventions, resistance from colleagues and supervisors, time pressure, lack of funding, competition, and lack of awareness of their value. There are international hurdles too, such as journals not yet being willing to accept preclinical SRs and a lack of familiarity with SRs among journal editors and peer reviewers. Consequently, the emergence of preclinical SRs falls within a research culture paradox. On the one hand, high quality, transparent, and reproducible research is and should be standard. Previous research highlighted that  SRs can play a useful role in this context as they can provide helpful information and insights for policy decision making and translation from preclinical to clinical studies ,51,52. ORegardless of all positive aspects, we should bear in mind that SRs rely on the studies they include and may therefore be of limited value. Like primary studies, they may also be constrained by poor reporting or poor conduct. Such limitations are highlighted with respect to clinical systematic reviews, for example with protocols in PROSPERO not corresponding to the PRISMA-P guidelines, and outcome discrepancies between the protocol and final publication ,55. TherThe major strength of this study lies in its mixed methodology, using both questionnaires and interviews to identify research impacts. This triangulation of data increases the validity of our findings. The choice of design and methodology was carefully considered by researchers from different academic disciplines, helping to ensure the study’s robustness.Regardless of a relatively limited set target population, we collected sufficient data using both questionnaires and interviews. The impacts we identified mapped onto the Research Impact Framework for the categories: “type of knowledge/problem”, “research methods”, “publication and papers”, “translatability potential”, “research network”, and “communications”, showing strong coherence between our findings and the Research Impact Framework. Our findings provide insight into how preclinical SRs have created impacts as a result of the ZonMw MKMD programme.Despite significant strengths, we identified limitations in the methodology. First, our sample consisted mainly of Dutch researchers and was, of course, limited to researchers who were awarded a ZonMw grant within the knowledge infrastructure module. Consequently, our findings–enthusiasm and drive for improvements–may be related to the cultural context. The Netherlands are well known for their innovation-driven enterprise, so we cannot ensure that similar findings would emerge in a different context or with a foreign funding agency.Second, our data collection took place during the COVID-19 pandemic and the study design we used limits the generalisation of our findings. Our findings are reported impacts, thus they are subjective to the perceptions of the participants, in contrast to measurable, tangible impacts. As mentioned earlier, the fact that (some of the) participants conducted their SR during the COVID-19 pandemic may have impacted their experience and influenced their responses. On the one hand, researchers might have had more time or opportunities to work on meta-research during the pandemic. On the other hand, the pandemic created delays in conducting the SRs and could have impacted the researchers personally, including their capacity and opportunity for professional development.Moreover, our study design does not include a control group nor a pre-post intervention assessment, which limits the certainty one can have on the causality of our intervention. An example of this uncertainty can be observed in Taken together, the intervention seems promising but our findings need to be interpreted within the narrow scope of the case study.Lastly, because this study had to be conducted over the summer, it is possible that, despite our best efforts to obtain a high response rate, the summer holidays reduced the availability of participants. Furthermore, the time scale for this study was rather short (3 months), which limited the time we could spend collecting and analysing our data. Finally, the data collection, coding and analysis of the transcripts were all performed by one researcher. The participation of one or multiple additional investigators would have strengthened the trustworthiness of our findings and potentially provided other interpretations. Investigator triangulation should be advocated when possible.Nevertheless, to our knowledge, this is the first case study on such a topic, and in the future, when preclinical SRs become more established, it would be interesting to repeat this study on a broader scale, for example within European institutes and/or the Ensuring Value in Research Funder Forum. In addition, a pre-post test study could be performed to evaluate change in skills and behaviour before and after the intervention. A wider population would also allow analysis by demographic group, e.g., the comparison of senior vs younger researchers, comparison between research field.This case study has provided important insights into the impacts of training, coaching and conducting preclinical SRs on both research and researchers. Not only did SRs impact research at a lab level, they led to changes in researchers’ views and critical abilities, and spurred efforts to advocate for improvement in their fields. Our project highlights the necessity and importance of supporting preclinical researchers to perform SRs, and demonstrates the impact of this on the quality and transparency of research, as well as on researchers’ awareness and motivation to change the status quo. Our findings suggest that support such as that provided here by the Dutch funding agency ZonMw, is relevant and should be encouraged on an international scale, to improve the quality and translation of preclinical research.S1 Appendix(PDF)Click here for additional data file.S2 Appendix(PDF)Click here for additional data file.S3 Appendix(PDF)Click here for additional data file.S4 Appendix(PDF)Click here for additional data file.S5 Appendix(PDF)Click here for additional data file.S6 Appendix(PDF)Click here for additional data file.S7 Appendix(PDF)Click here for additional data file.S8 Appendix(DOCX)Click here for additional data file.S9 Appendix(PDF)Click here for additional data file.S10 Appendix(PDF)Click here for additional data file.S11 Appendix(PDF)Click here for additional data file.S12 Appendix(PDF)Click here for additional data file.S13 Appendix(XLSX)Click here for additional data file. 2 Aug 2021PONE-D-21-16684The impact on research and researchers of conducting preclinical systematic reviews: a mixed methods case studyPLOS ONEDear Dr. Julia Menon,Thank you for submitting your manuscript to PLOS ONE. This is generally a well written paper but after careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.Please address the peer reviewers' comments which offer suggestions for improving readability and grammatical checks. Where applicable, we ask you to provide a point by point response the comments provided. As PLOS ONE does not copy edit accepted manuscripts we ask you to conduct a thorough spell and grammatical check before resubmissionplosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.Please submit your revised manuscript by 30 August 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.We look forward to receiving your revised manuscript.Kind regards,Eleanor OchodoAcademic EditorPLOS ONEJournal requirements:When submitting your revision, we need you to address these additional requirements.1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found athttps://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf andhttps://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.2. Thank you for stating the following in the Competing Interests/Financial Disclosure * (delete as necessary) section:“I have read the journal's policy and the authors of this manuscript have the following competing interests:Julia Menon declares that she worked for the Department “Health Evidence” within the Radboudumc, in the same team as the coaches who provided training and support for the “knowledge infrastructure” module. Since February 2021, she has a paid position via ZonMw.Merel Ritskes-Hoitinga, who supervised this study, is the head of the SYstematic Review Center for Laboratory  Experimentation (SyRCLE) team.Pandora Pound declared no competing interest.Erica van Oort, who supervised this study, is project manager for ZonMw and is in charge of the “knowledge infrastructure” module, as part of the ZonMw MKMD programme.However, we all sincerely declare that we did our utmost to remain impartial when conducting, analysing and supervising this study.” We note that you received funding from a commercial source: [Name of Company]Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” . The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #1: YesReviewer #2: Yes**********4. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #1: YesReviewer #2: No**********5. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #1: 1. Yes, this manuscript technically sound, and its data support the conclusions reached.2. Yes, statistical analysis were performed appropriately3. Yes, all data was available4. Yes, it is presented in an intelligible fashion and written in standard English5. Comments have been attached.Reviewer #2: I would like to thank the editor for the opportunity to review this very interesting paper; and would also like to thank the authors for conducting and reporting on this important and fascinating research topic on preclinical evidence synthesis; that is, relating to meta-research of laboratory experiments on animals. The paper investigates the impact of conducting a preclinical systematic reviews (SR), with funded coaching and workshops, on researchers and their subsequent research. While the limitations in terms of generalisability should be, and are, acknowledged, this is an important first step in reducing animal waste while improving and standardising preclinical work.The work shines the light on a very important and often neglected aspect of research waste: the waste of animal subjects in research. The authors are commended for this. It is our responsibility as scientists to conduct research on animals as ethically, rigorously and efficiently as possible; to ensure we obtain the correct results the first time, with minimal animal use. This may very well be a first-order seminal paper in the establishment of SRs as the top tier of the evidence hierarchy in this field. I offer the following thoughts with regards to the manuscript for the consideration of the editor and authors. I have marked the most pressing issues, ones I would classify as major in a differently designed study, with 'IMPORTANT' throughout the review.OverallThe manuscript would benefit from editing for minor spelling, punctuation and grammatical errors throughout, but readability in general is good. I have attempted to identify the most glaring errors in this review.AbstractLines 17-18: It is not clear from this statement whether the focus of the ZonMw grant scheme is preclinical SRs specifically, or SRs in general.Lines 22-23: This sentence does not accurately reflect that recruited researchers could be those who had finished a preclinical SR as well as those still busy conducting a preclinical SR .Lines 36-39: Though the authors do couch their language here, a word of caution: in the absence of measurements in a control group which did not receive the funded interventions and conducted a preclinical SR; the authors are requested to exercise caution in attributing the reported changes to the interventions alone. Even though researchers may themselves attribute these changes to the intervention, there may be a complex interplay of factors, e.g. the growth that happens during the work done for a PhD, contributing to their changing views and skill sets.IntroductionGeneral: It would be useful if authors could explicitly define early on in the manuscript what is meant by ‘preclinical’, as the term is also sometimes used to refer to pre-hospital and emergency medicine contexts.Line 41: Suggest changing to ‘Keeping up to date with health/medical literature…’Line 46: Please remove errant ‘-‘ from the sentence ending on this line.Line 56: Consider changing ‘throw light’ to ‘cast light’.Line 74: This introductory line should be presented as such. In its present state it is confusing and appears as another question, instead of the overarching aim. I would suggest amending this sentence to ‘This work aims to assess the impact of conducting preclinical SRs on researchers, their research and their field through the following objectives:’Lines 115-116: While it is acknowledged that these would be outside the scope of the current manuscript, which already describes a great deal of work, it would be interesting to assess the policy impacts and societal impacts (areas 2 and 4 in the manuscript) of the intervention in question using the modified Kuruvilla et al. 2007 checklist.Materials and MethodsLine 142: It is not clear what is meant by ‘thrive’ in this context, but it is assumed to be an incorrect translation. Consider changing to ‘drive’ or ‘development’, depending on the original intent.Lines 157-158: This sentence contradicts methodology in following sections, which describes participants who had not finished their SRs also included in the sample. Please correct this factual inaccuracy, or elaborate on why non-completers were included post hoc – if that was the case.Line 163: It may be useful to include here whether participants consented to be contacted by ZonMw for the purposes of intervention assessment when providing details, if this was indeed the case.Line 177: It would be useful, though not essential, to state at the start of this section whether informed consent was sought to use the responses provided in questionnaires.Line 186: As it is not possible for researchers still conducting their SRs to evaluate post-SR skills and experience, it is assumed that this should read ‘For the latter, only points 1, 2 and 3 were evaluated.’ Please correct if this is the case, or elaborate on how post-intervention experiences would be appraised before the end of the intervention – if the manuscript is correct in its current format.IMPORTANT Lines 205-208: It is a pity that more demographic/explanatory variables were not collected on participants, as some measures of association could have been calculated. This would have provided some insights into differences between researchers and how this may, or may not, have shaped their responses – specifically given the outstanding uncertainty presented by the lack of a control group of questionnaire participants. It would be useful if the authors could include these considerations in their manuscript, perhaps as a next step in the evolution of understanding the role of SRs in preclinical work.Lines 235-236: Please provide a breakdown of the number of inputs per background/career level, i.e. ‘…Thorough piloting was performed for both questionnaires and semi-structured interviews by seven researchers knowledgeable in SRs with varying backgrounds and career levels, namely research assistant (n=x), PhD student (n=y) and professor (n=z).’ I don’t think it’s necessary to identify a person at a professorial level as ‘associate’ (or not).ResultsLines 243-244: It is very possible that response and completion rates may be defined differently by different sources. A leading survey software defines response rate as ‘the number of people who completed the survey divided by the total sample’ (i.e. 45/95=47.4%), and completion rate as ‘the number of surveys filled out and submitted divided by the number of surveys started’ (i.e. 45/61=73.8%). Please revisit these concepts and calculations, and provide a reference for the definitions used to avoid confusion among readers.Line 248: Please revise the start of the sentence ‘And the 7 remaining…’. It is recommended that this sentence be combined with the previous sentence.Line 265-266: I am not convinced that reporting an ‘average median’ is wise; it is also a meaningless value in terms of agreement without information on the statements asked and the direction of the question. I wouldn’t risk readers skimming over this without referring to the figures, where the median breakdown per question provides much richer information.Line 317: Steer clear of statements such as ‘the results are quite positive’ as this indicates inherent bias for a result.IMPORTANT Lines 317-321: As stated before, it is very difficult to know whether the equivalence in the response of SR completers and non-completers can be ascribed to the SR, given the lack of a control comparator. It is possible that these insights may have evolved in respondents in a similar field, and with comparable education and training, in the absence of conducting a preclinical SR. This evolution might even be suggested by the identical medians in the two groups, with further SR experience (in completers) leading to no increase in critical appraisal compared to non-completers. It is accepted that this uncertainty is part of the case study design, but please make this clear to the reader.IMPORTANT Line 338: It would be interesting to know whether respondents participating in the scheme during 2020 had a different experience to those pre-pandemic in terms of time taken/delays with their SRs. If these data are available this might be an interesting aspect to include, as the time required to conduct an SR often presents a considerable barrier to their initiation and completion.Lines 470 and 484: Please steer clear from informal language such as ‘no easy business’ and ‘budge one bit’. Furthermore, ‘(yet)’ [line 484] again demonstrates an inherent bias by the author for a certain outcome, and should be avoided at all costs.DiscussionIMPORTANT General: As stated before, it is very difficult to know whether the reported experiences and skills of respondents can be ascribed to conducting the preclinical SR, given the lack of a control comparator or association with explanatory variables. It is possible that the reported insights would have developed in respondents in a similar field and with comparable education and training. The authors do address the subjective nature of their findings, but could suggest next steps in this important work. As a starting point, it might be useful to suggest (and consider) a pre-post test with a new intake of researchers, measuring their attitudes and skills at baseline and again following the completion of their SR. An alternative, but a weaker test of causality, would be to measure explanatory variables and test associations with reported outcomes in future work.General: It was surprising not to see any exploration of the effect of the pandemic on the findings presented in this paper, given both the global impact of the event as well as several mentions of COVID-19 in the questionnaire responses – specifically related to delays in SR completion. It might be useful for the authors to include a paragraph on how this context may have shaped their findings, both in terms of practicalities for SRs as well as responses; particularly due to shifts in mental health and outlook for many during this time.IMPORTANT Lines 568-569: Statements attributing changes in mindset and behaviour to the conduct of preclinical SRs without acknowledging the complex myriad of circumstances that may contribute to these are problematic. Please review this statement (and others like it) to accurately reflect that preclinical SRs *appear to contribute* to a shift in mindset and behaviour, though comparative data, and more work to identify other explanatory factors, is needed.Conflicts of interestThe authors declare their association with the invention provider upfront, and no serious conflicts were identified. I would like to caution the authors, however, around the language they use and how it unintentionally reflects a preferred direction of their findings.With thanks again for the opportunity, and best wishes.**********what does this mean?). If published, this will include your full peer review and any attached files.6. PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool, AttachmentPONE-D-21-16684_reviewer.pdfSubmitted filename: Click here for additional data file. 10 Sep 2021Dear Editors,We would like to thank the editor and the two reviewers for their comments and suggestions and the opportunity to revise our submitted manuscript. We have addressed all issues that were raised and address detailed answers below. The comments were very insightful and have helped to improve our manuscript. We genuinely hope that our revisions will be satisfactory and that our manuscript will be now suitable for publication. Thank you again for considering our manuscript for publication in PLoS One. Looking forward to hearing from you again,Sincerely,On behalf of all co-authors,Julia MenonEditor’s comments:1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file namingAuthors’ response: we made a thorough check and made the necessary changes to comply with PLOS ONE’s style requirements. We genuinely apologise that several mistakes passed through and thank the editor for their keen eye and recommendation to modify our manuscript accordingly. Here is a detailed list of what was modified: -Page 1: Affiliations are numbered, the postal address of the corresponding author was removed, and we added initials to the corresponding author. -line 130: we change the title of Fig 1 to bold-All headings and subheadings were verified and now comply with sentence case-Appendices captions were remodelled for appendix S2, S11, and S12. The appendices files were all renamed as “SX_Appendix.pdf” for example “S1_Appendix.pdf”. -Appendices are now formatted as level 1 headings and their caption were verified for compliance. Appendices were changed from .docx to .pdf format. Therefore, we also aim to reupload all appendices to be consistent.2. We note that you received funding from a commercial source: [Name of Company]Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.”  Experimentation (SyRCLE) team.Pandora Pound declared no competing interest.Erica van Oort, who supervised this study, is program manager for ZonMw and is in charge of the “knowledge infrastructure” module, as part of the ZonMw MKMD programme.ZonMw funded this project, as declared in our financial statement, as well as the employment of Erica van Oort. ZonMw is a Public Benefit Organisation, which is registered with the Chamber of Commerce The Hague under number 27365263, tax number: 0028.76.528. This does not alter our adherence to PLOS ONE policies on sharing data and materials.However, we all sincerely declare that we did our utmost to remain impartial when conducting, analysing and supervising this study.”3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.Authors’ response: It seems our ethics statement only appears in the Methods section . 4. Please include a copy of Table 1 which you refer to in your text on page 33, and Table 12 on page 34Authors’ response: Table 1 and Table 12 are provided in Appendix S1 and S12. Consequently, we deleted their caption and only kept the captions of Appendix S1 and S12. For consistency reasons, we would rather have all supplementary materials be referred to as “Appendix”, than have a mix of tables, figures, supplementary materials etc, in the captions. We also modified the mention of Table 12 to “S12 Appendix” in line 335.Reviewer #1 comments: 1. Line 1-2: Please revise it: "The impact of conducting preclinical systematic reviews on researchers and their research: a mixed method case study" will sound good. Authors’ response: We agree that this title is more straightforward. We modified it accordingly. 2. Line 15: Delete "a"Authors’ response: the “a” was removed accordingly, and a “s” was added to “cornerstone”. Thank you for noticing this typo. 3. Line 46: Delete "-"Authors’ response: It was deleted accordingly. 4. Line 49: Remove "a"Authors’ response: the “a” was removed accordingly and a “s” was added to “cornerstone”. Thank you for noticing this typo. 5. Line 50: Include also the number of SR being conducted.Authors’ response: Thank you for this great suggestion. We added numbers of SR being conducted as of 2014 (this is the most recent update we could find on that matter) and added the reference to the relevant article. The text now reads: “Today, they are cornerstones of evidence-based medicine, with 30,000 SRs protocols being registered as of 2017 and a 2014 estimate putting the number of published SRs at over one million ”. 6. Line 70: Remove "preclinical"Authors’ response: we removed “preclinical” as you suggested. 7. Line 161: Replace "contacted" with "recruited"Authors’ response: we changed the sentence as suggested. 8. Line 222-224: Even though this study did not pose any risk to participants, ethical clearance was needed since it involved human beings. This study should have applied for ethical clearence.Authors’ response: we realise that our statement may have been unclear. So, we provide more information here and in the text of the manuscript. https://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-not and https://english.ccmo.nl/investigators/additional-requirements-for-certain-types-of-research/non-wmo-research ). According to Dutch law, studies including humans can be subjected to the Medical Research Involving Human Subjects Act (WMO) and must be reviewed by the Central Committee on Research Involving Human Subject (CCMO) or a Medical Research Ethics Committees (MREC). However, some studies in which humans are participating are not subject to the WMO, and hence do not require ethical review . Considering the nature of our questions and the short amount of time needed to fill the questionnaire, our research does not fall within that description. Questionnaire research does not follow the WMO and hence does not require ethical review. The exception is if questions are, for instance, burdensome, intimate, or if the questionnaire takes a lot of time to fill in . The topics addressed in both questionnaires and interview guides posed no risks to the participants, and in particular no risk of physical or mental harm. For these reasons, we did not seek approval from an Institutional Review Board.”9. Line 242-248: Please add demographic characteristics of the participants; gender, age and their research levels (if possible).Authors’ response: We did not collect demographic characteristics systematically. Therefore, we cannot add them to the text. However, we agree that demographic characteristics could have influenced the results and discuss their potential influence in our discussion. We also suggest that future studies use demographic groups (e.g. young researchers vs senior researchers)Here is the text we added to our discussion: “In this case study, we focussed primarily on our intervention, and it should be noted that the improvements observed cannot be only attributed to preclinical SRs, as complex factors are also involved and may have influenced participants’ answers . Further assessments may help to highlight or understand the complex factors contributing to the beneficial effects.” And also: “A wider population would also allow analysis by demographic group, e.g., the comparison of senior vs younger researchers, comparison between research field.”10. Line 242-248: Please rephrase the paragraph it is confusing (not clear) in the current form, in terms of the numbers.Authors’ response: We reverified all numbers and identified that an error got introduced, which may be the reason for the unclearness. It was not 7 but 5 participants who terminated the questionnaire. It was modified appropriately.Moreover, we provided an additional sentence to clarify the numbers in the text and hope it will be suited to your expectations. At the end of the explanation, we state: “Therefore the 61 participants were divided as such: SR completed (n=36), SR ongoing (n=18), SR stopped (n=2), and NR (n=5).” The whole paragraph was also modified for more clarity:Of 99 potential participants, we were able to contact 95. Sixty-one participants started our questionnaire, and 45 completed it , giving a response rate of 47.4% and a completion rate of 73.8% (definitions of response and completion rate can be found here [40]). An overview of participants per phase is available in Fig 2.Of the 61 participants, 36 belonged to the “SR completed” group . In comparison, 18 participants belonged to the “SR ongoing” group . Two participants did not complete their SRs due to time constraints, while the 5 remaining participants terminated the questionnaire before answering the question about the state of their review. Therefore the 61 participants were divided as such: SR completed (n=36), SR ongoing (n=18), SR stopped (n=2), and NR (n=5). Ten participants agreed to participate in interviews but only eight interviews were eventually conducted due to the unavailability of two researchers.”11. Line 260-262: What happened to the 11 participants in the SR completed group? 36 - 5 - 6 = 25. Please talk about the other missing 11 participants.Authors’ response: We precise the number by adding the following sentences: “Within the SR completed group (n=36), 14 participants went on to perform primary animal studies after completion of their SRs, 5 performed preclinical research using alternatives to animals, 6 performed clinical studies, and 6 moved to meta-research or ceased research. The remaining five participants did not complete this part of the questionnaire.”Reviewer #2 comments: 1. Lines 17-18: It is not clear from this statement whether the focus of the ZonMw grant scheme is preclinical SRs specifically, or SRs in general.Authors’ response: Indeed, the sentence could be confusing. Thank you for spotting it. We added “preclinical” in front of “SRs” to specify that the ZonMw grant scheme focuses on preclinical SRs specifically. The sentence now reads: “Since 2011 the Dutch health funding organisation (ZonMw) has run a grant scheme dedicated to promoting the training, coaching and conduct of preclinical SRs.”2. Lines 22-23: This sentence does not accurately reflect that recruited researchers could be those who had finished a preclinical SR as well as those still busy conducting a preclinical SR .Authors’ response: To bring more clarity, we rephrased as follow: “We recruited researchers who attended funded preclinical SR workshops and who conducted, are still conducting, or prematurely stopped a SR with funded coaching.”s3. Lines 36-39: Though the authors do couch their language here, a word of caution: in the absence of measurements in a control group which did not receive the funded interventions and conducted a preclinical SR; the authors are requested to exercise caution in attributing the reported changes to the interventions alone. Even though researchers may themselves attribute these changes to the intervention, there may be a complex interplay of factors, e.g. the growth that happens during the work done for a PhD, contributing to their changing views and skill sets.Authors’ response: We understand and see your point for this part to be less “causal”. To dampen our conclusion statement, we modified the conclusion as follows: “Being trained and coached in the conduct of preclinical SRs appears to be a contributing factor to many beneficial changes which will impact the quality of preclinical research in the long-term. Our findings suggest that this ZonMw funding scheme is helpful in improving the quality and transparency of preclinical research.”4. General: It would be useful if authors could explicitly define early on in the manuscript what is meant by ‘preclinical’, as the term is also sometimes used to refer to pre-hospital and emergency medicine contexts.Authors’ response: To clarify what we meant, we added to line 56-57: “SRs are struggling to achieve similar status in the preclinical field ”5. Line 41: Suggest changing to ‘Keeping up to date with health/medical literature…’Authors’ response: Agreed, the sentence was modified as suggested. 6. Line 46: Please remove errant ‘-‘ from the sentence ending on this line.Authors’ response: Thank you for noticing this mistake. It was deleted accordingly. 7. Line 56: Consider changing ‘throw light’ to ‘cast light’.Authors’ response: this sentence fragment was modified as suggested. 8. Line 74: This introductory line should be presented as such. In its present state it is confusing and appears as another question, instead of the overarching aim. I would suggest amending this sentence to ‘This work aims to assess the impact of conducting preclinical SRs on researchers, their research and their field through the following objectives:’Authors’ response: Indeed, the sentence suggested provides a clearer introductory line. We modified it as suggested, with a small modification (we changed “This work” to “Our study”). 9. Lines 115-116: While it is acknowledged that these would be outside the scope of the current manuscript, which already describes a great deal of work, it would be interesting to assess the policy impacts and societal impacts (areas 2 and 4 in the manuscript) of the intervention in question using the modified Kuruvilla et al. 2007 checklist.Authors’ response: We agree that policy impacts and societal impacts would have been very interesting to assess. However, our entire set of questions in both survey and interviews are based to match the research impact category of the Kuruvilla et al. 2007 checklist and the behavioural wheel of change. Assessing policy and societal impacts would require other questions and hence to perform a whole new study – which we cannot do in the scope of this manuscript. We do recognise the relevance of the other areas and hence have added a sentence to the discussion in that regard. The sentence is: “Further studies are warranted to fully understand what hinders or facilitate the conduct of preclinical SRs. For instance, a more in-depth assessment of impacts would provide greater insight and could address other areas of the research impact framework, e.g., policy impacts, societal impacts”.10. Line 142: It is not clear what is meant by ‘thrive’ in this context, but it is assumed to be an incorrect translation. Consider changing to ‘drive’ or ‘development’, depending on the original intent.Authors’ response: As the reviewer mentioned, it is an incorrect translation. After review, we modified thrive to ‘motivation’. 11. Lines 157-158: This sentence contradicts methodology in following sections, which describes participants who had not finished their SRs also included in the sample. Please correct this factual inaccuracy, or elaborate on why non-completers were included post hoc – if that was the case.Authors’ response: Participants were recruited regardless if they completed, started (and are currently conducting) or started and stopped their systematic review. We can see how the second part can be confusing/unclear in line 157-158. Therefore, we modified it as follows: “The target population for the questionnaires comprised researchers who had participated in a ZonMw workshop and who had either started (currently conducting or prematurely stopped) or completed a preclinical SR with funded coaching.”12. Line 163: It may be useful to include here whether participants consented to be contacted by ZonMw for the purposes of intervention assessment when providing details, if this was indeed the case.Authors’ response: ZonMw was in contact with all researchers by being their funder. By default, researchers agreed to communicate with ZonMw on their work and progress. Besides, they were given the opportunity to refuse participation or withdraw at any time. We are not sure if this information is required in the manuscript, as “at worst”, researchers received 2 e-mails kindly inviting them to participate in a survey – without any pressure-, which is not burdensome or intimate.13. Line 177: It would be useful, though not essential, to state at the start of this section whether informed consent was sought to use the responses provided in questionnaires.Authors’ response: The introduction text of the questionnaires clearly stated the intentions of the questionnaire. Though it was not exactly stated as such, we have assumed that researchers agree for their data (anonymously) to be collected, otherwise they would not have agreed to fill out the questionnaire in the first place. 14. Line 186: As it is not possible for researchers still conducting their SRs to evaluate post-SR skills and experience, it is assumed that this should read ‘For the latter, only points 1, 2 and 3 were evaluated.’ Please correct if this is the case, or elaborate on how post-intervention experiences would be appraised before the end of the intervention – if the manuscript is correct in its current format.Authors’ response: We realise that this part does not convey the information we wanted. We meant “skills gained as a result of conducting (steps of) the SR”. Considering that systematic reviews follow structured steps, one researcher could learn new skills by doing some of the steps without actually completing the whole study, e.g., designing a comprehensive search. It is indeed points 3,4, and 5 for the “ongoing group”. We added the following:“4) skills gained as a result of conducting (steps of) the SR, and 5) experience with conducting (steps of) the SR )”. 15. IMPORTANT Lines 205-208: It is a pity that more demographic/explanatory variables were not collected on participants, as some measures of association could have been calculated. This would have provided some insights into differences between researchers and how this may, or may not, have shaped their responses – specifically given the outstanding uncertainty presented by the lack of a control group of questionnaire participants. It would be useful if the authors could include these considerations in their manuscript, perhaps as a next step in the evolution of understanding the role of SRs in preclinical work.Authors’ response: We understand your point of view and added this point to the discussion“ In this case study, we focussed primarily on our intervention, and it should be noted that the improvements observed cannot be only attributed to preclinical SRs as complex factors are also involved and may have influenced participants’ answers . Further assessments may help to highlight or understand the intricated factors contributing to the beneficial effects.”“in the future, when preclinical SRs become more established, it would be interesting to repeat this study on a broader scale […]. A wider population would also allow analysis by demographic group, e.g., the comparison of senior vs younger researchers, comparison between research field.”16. Lines 235-236: Please provide a breakdown of the number of inputs per background/career level, i.e. ‘…Thorough piloting was performed for both questionnaires and semi-structured interviews by seven researchers knowledgeable in SRs with varying backgrounds and career levels, namely research assistant (n=x), PhD student (n=y) and professor (n=z).’ I don’t think it’s necessary to identify a person at a professorial level as ‘associate’ (or not).Authors’ response: We modified this statement accordingly: “Thorough piloting was performed for both questionnaires and semi-structured interviews by seven researchers knowledgeable about SRs and from a variety of backgrounds and career levels, background and career levels, namely research assistants (n=2), PhD student (n=1), (associate), and professors (n=4).”17. Lines 243-244: It is very possible that response and completion rates may be defined differently by different sources. A leading survey software defines response rate as ‘the number of people who completed the survey divided by the total sample’ (i.e. 45/95=47.4%), and completion rate as ‘the number of surveys filled out and submitted divided by the number of surveys started’ (i.e. 45/61=73.8%). Please revisit these concepts and calculations, and provide a reference for the definitions used to avoid confusion among readers.Authors’ response: For clarity purposes, we decided to use the same definition you provided with a clear reference. Also, figure 2 was modified accordingly.18. Line 248: Please revise the start of the sentence ‘And the 7 remaining…’. It is recommended that this sentence be combined with the previous sentence.Authors’ response: we combined the two sentences as suggested. It now reads: “Two participants did not complete their SRs due to time constraints, while the 7 remaining participants terminated the questionnaire before answering the question about the state of their review”.To note: We reverified all numbers and identified that an error got introduced. It was not 7 but 5 participants who terminated the questionnaire . It was modified appropriately.19. Line 265-266: I am not convinced that reporting an ‘average median’ is wise; it is also a meaningless value in terms of agreement without information on the statements asked and the direction of the question. I wouldn’t risk readers skimming over this without referring to the figures, where the median breakdown per question provides much richer information.Authors’ response: We see your point and hence removed the sentences mentioning average median. To give you some context, our intention with the average median was to give an overall value for each part, while bringing new information that was not mentioned in the figure. We agree with the comment and wish people would directly look in detail at the figures.20. Line 317: Steer clear of statements such as ‘the results are quite positive’ as this indicates inherent bias for a result.Authors’ response: We deleted this part and read the result thoroughly to remove similar statements. 21. IMPORTANT Lines 317-321: As stated before, it is very difficult to know whether the equivalence in the response of SR completers and non-completers can be ascribed to the SR, given the lack of a control comparator. It is possible that these insights may have evolved in respondents in a similar field, and with comparable education and training, in the absence of conducting a preclinical SR. This evolution might even be suggested by the identical medians in the two groups, with further SR experience (in completers) leading to no increase in critical appraisal compared to non-completers. It is accepted that this uncertainty is part of the case study design, but please make this clear to the reader.Authors’ response: We added a sentence about this matter in the discussion: “Moreover, our study design does not include a control group nor a pre-post intervention assessment, which limits the certainty one can have on the causality of our intervention. An example of this uncertainty can be observed in Fig 4, where both groups answered similarly regardless of whether they completed their review. This highlights again that the impacts and benefits found in our study cannot be only attributed to the conduct of preclinical SR but may be influenced by the workshop. We could imagine that similar education, i.e., workshops with hands-on practice, could produce similar results without actually conducting a preclinical SR.”“Taken together, the intervention seems promising but our findings need to be interpreted within the narrow scope of the case study”22. IMPORTANT Line 338: It would be interesting to know whether respondents participating in the scheme during 2020 had a different experience to those pre-pandemic in terms of time taken/delays with their SRs. If these data are available this might be an interesting aspect to include, as the time required to conduct an SR often presents a considerable barrier to their initiation and completion.Authors’ response: It would be an interesting comparison to make, however we do not have the data to perform this analysis completely. Nevertheless, three participants (one from the completed group and two from the started group) mentioned delays in relation to the covid situation. To address delays, we added the following paragraph “Reviews (and reviews steps) often took longer than expected (26/30 participants for the completed group and 8/15 participants for the ongoing group). For instance, researchers often experienced the number of studies during screening, data extraction or data analysis as overwhelming, but also suffered in getting their protocols, manuscripts or revision for publication reviewed. Some had to wait for second or third assessors/reviewers and experienced delays when asking authors for further information. Also, three participants (both groups combined) mentioned the COVID-19 pandemic as a delaying factor.”23. Lines 470 and 484: Please steer clear from informal language such as ‘no easy business’ and ‘budge one bit’. Furthermore, ‘(yet)’ [line 484] again demonstrates an inherent bias by the author for a certain outcome, and should be avoided at all costs.Authors’ response: These instances were removed and modified as follow: “Interviewees’ experiences highlighted that getting preclinical SRs published can be challenging.” We also added: “discovering that the insights from their SRs appeared to have no impact on their field.” We verified the rest of the manuscript text to remove informal language. Regarding the ‘(yet)’, we meant that the participants wished for changes in their field (at least by the ones that were frustrated) but that this change did not yet occur. However, we see how it could be misleading and thus removed it according to your suggestion. 24. IMPORTANT General: As stated before, it is very difficult to know whether the reported experiences and skills of respondents can be ascribed to conducting the preclinical SR, given the lack of a control comparator or association with explanatory variables. It is possible that the reported insights would have developed in respondents in a similar field and with comparable education and training. The authors do address the subjective nature of their findings, but could suggest next steps in this important work. As a starting point, it might be useful to suggest (and consider) a pre-post test with a new intake of researchers, measuring their attitudes and skills at baseline and again following the completion of their SR. An alternative, but a weaker test of causality, would be to measure explanatory variables and test associations with reported outcomes in future work.Authors’ response: We added several statements to clarify this aspect: -line 596-600: In this case study, we focussed primarily on our intervention, and it should be noted that the improvements observed cannot be only attributed to preclinical SRs, as complex factors are also involved . Further assessments may help to highlight or understand the complex factors contributing to the beneficial effects.-line 647-648: In addition, a pre-post test study could be performed to evaluate changes in skills and behaviour before and after the intervention.25. General: It was surprising not to see any exploration of the effect of the pandemic on the findings presented in this paper, given both the global impact of the event as well as several mentions of COVID-19 in the questionnaire responses – specifically related to delays in SR completion. It might be useful for the authors to include a paragraph on how this context may have shaped their findings, both in terms of practicalities for SRs as well as responses; particularly due to shifts in mental health and outlook for many during this time.Authors’ response: It is indeed an important factor, we now mention it in the strengths and limitations, with the following paragraph: “Second, our data collection took place during the COVID-19 pandemic and the study design we used limits the generalisation of our findings. Our findings are reported impacts, thus they are subjective to the perceptions of the participants, in contrast to measurable, tangible impacts. As mentioned in the result, the fact that (part of the) participants conducted their review during the COVID-19 pandemic may have impacted their experience and hence influenced their answers. On the one hand, researchers might have had more time or opportunities to work on meta-research during the pandemic. On the other hand, the pandemic conditions seem to have created delays in conducting the SRs and could have impacted the researchers personally, including their capacity and opportunity for professional development.” 26. IMPORTANT Lines 568-569: Statements attributing changes in mindset and behaviour to the conduct of preclinical SRs without acknowledging the complex myriad of circumstances that may contribute to these are problematic. Please review this statement (and others like it) to accurately reflect that preclinical SRs *appear to contribute* to a shift in mindset and behaviour, though comparative data, and more work to identify other explanatory factors, is needed.Authors’ response: We overall reduced the strength of the language when talking about causality, for example saying “contribute to” or “seem to participate” instead of “trigger” or “enable”. In addition, as mentioned in an earlier comment, we added the following sentence: “it should be noted that the improvements observed cannot be only attributed to preclinical SRs, as complex factors are also involved . Further assessments may help to highlight or understand the complex factors contributing to the beneficial effects.”27. Conflicts of interest. The authors declare their association with the invention provider upfront, and no serious conflicts were identified. I would like to caution the authors, however, around the language they use and how it unintentionally reflects a preferred direction of their findings.Authors’ response: We performed a thorough check of the manuscript to ensure that no preferred directions were mentioned in the findings. When necessary, we rephrased/removed part of the text. Of course, positive language was not consciously included in the text. We acknowledge our enthusiasm for systematic reviews, which may have unintentionally been included in our writing process. We would like to thank you for your warning and have rewritten the text accordingly.To note: We ran checks for minor spelling, punctuation and grammatical errors as you suggested.AttachmentResponse to Reviewers.docxSubmitted filename: Click here for additional data file. 3 Nov 2021PONE-D-21-16684R1The impact of conducting preclinical systematic reviews on researchers and their research: a mixed method case studyPLOS ONEDear Julia Menon,Thank you for submitting your manuscript to PLOS ONE.  Your responses to the previous comments raised have been well received. We however request that you respond to two minor comments by reviewer 2. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please submit your revised manuscript by 18 Dec 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.Please include the following items when submitting your revised manuscript:We look forward to receiving your revised manuscript.Kind regards,Eleanor OchodoAcademic EditorPLOS ONEJournal Requirements:Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.Reviewers' comments:Reviewer's Responses to QuestionsComments to the Author1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.Reviewer #1: All comments have been addressedReviewer #2: (No Response)**********2. Is the manuscript technically sound, and do the data support the conclusions?The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: YesReviewer #2: Yes**********3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: YesReviewer #2: Yes**********4. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #1: YesReviewer #2: Yes**********5. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #1: YesReviewer #2: Yes**********6. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #1: (No Response)Reviewer #2: Thank you very much for your considered feedback and amendments to the manuscript - as I stated during the first round of review, this is a very important piece of work; I hope to see it in print (and cite it!).I have no further *material* comments, but two further suggestions to consider, based on your feedback:- Line 249: I think '(associate)' can be removed from the text entirely, i.e. "...namely research assistants (n=2), a PhD student (n=1) and professors (n=4)."- Line 523: I take your point on the intention behind "...had not (yet) made..." in the previous version, and that this reflects the wishes of the participant rather than you as an author team. While I do think it's better to steer clear from any wording that could result in perceived bias, you could consider changing the sentence to "...discovering that the insights from their SRs appeared to have had no impact on their field to date." This is entirely up to you, as the revised version you provided is also fine, but perhaps this provides a bit more nuance.Wishing you all the best,Amanda**********what does this mean?). If published, this will include your full peer review and any attached files.7. PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool,  11 Nov 2021Reviewer’s comments1. Line 249: I think '(associate)' can be removed from the text entirely, i.e. "...namely research assistants (n=2), a PhD student (n=1) and professors (n=4)."Authors’ response: We agree that this term makes the sentence “wordy” and does not per se add to the context. We have removed it as suggested. 2. Line 523: I take your point on the intention behind "...had not (yet) made..." in the previous version, and that this reflects the wishes of the participant rather than you as an author team. While I do think it's better to steer clear from any wording that could result in perceived bias, you could consider changing the sentence to "...discovering that the insights from their SRs appeared to have had no impact on their field to date." This is entirely up to you, as the revised version you provided is also fine, but perhaps this provides a bit more nuance.Authors’ response: Thank you for this suggestion! We feel it indeed brings more nuance and clarity to this part of the text. The sentence was modified as suggested.AttachmentReponse to reviewers.docxSubmitted filename: Click here for additional data file. 15 Nov 2021The impact of conducting preclinical systematic reviews on researchers and their research: a mixed method case studyPONE-D-21-16684R2Dear Julia Menon,We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at onepress@plos.org.If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact Kind regards,Eleanor OchodoAcademic EditorPLOS ONE 1 Dec 2021PONE-D-21-16684R2 The impact of conducting preclinical systematic reviews on researchers and their research: a mixed method case study Dear Dr. Menon:I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. onepress@plos.org.If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact plosone@plos.org. If we can help with anything else, please email us at Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staffon behalf ofProf Eleanor Ochodo Academic EditorPLOS ONE