The outcome of the undergraduate medical programme is to produce clinically competent health care providers relevant for the South African context. Educational institutions find it hard to ensure the quality of assessments where competency must be assessed. This study aimed to compile an assessment framework that can be used to benchmark current assessment practices in the clinical phase of the undergraduate medical programme where competency must be certified.In this observational, descriptive study, qualitative data were gathered using the steps described by the World Health Organization for rapid reviews. Literature was searched, screened and selected before data were analysed and a framework was constructed.Twenty-five official documents were included in the study. The framework addressed the three components of quality assessment, namely, accreditation, assessment and quality assurance. Assessors should attend to the principles of assessment, namely, validity, reliability, fairness, feasibility, educational effect and acceptability, but realise that no assessment meets all these criteria. The first step to ensure quality assessment is to identify a clear outcome. Assessment should be planned and aligned with this outcome.It is clear that clinical assessment is multidimensional and that no assessment is perfect. Programme accreditation, assessment practices and psychometrics can assist to improve the quality of assessment but cannot judge clinical competence. Using experienced assessors with a variety of assessment methods on a continuous basis is the proposed way to assess clinical competence. An assessment framework can assist to improve assessment, but it cannot guarantee quality assessment. Passing the final assessment of the MBChB programme enables a student to graduate and qualify as an entry-level medical practitioner.2 The main outcome of the MBChB programme is to produce clinically competent health care providers relevant for the South African context.3 This context is determined by the quadruple burden of disease ,4 which, in turn, determines the required competencies to manage these conditions. Clinical assessment is unique because of integration of competencies and more than one possible correct approach to a problem.5 If clinical competence is assessed on the ‘does’ level of Miller’s pyramid,6 there will always be a compromise on reliability.7 Locally, the educational institution finds it hard to defend the quality of high-stakes competency assessments against validity, reliability and fairness. This study forms part of a larger study addressing quality of assessment in the clinical phase of the undergraduate medical programme.In South Africa, undergraduate medical training programmes are offered at nine accredited universities.9 In this article, registration is added to accreditation, as the MBChB degree is a professional qualification that must be accredited and registered with various authorities. The term audit is replaced by quality assurance, as both these terms serve the purpose of improving quality; however, the term audit is usually used in the financial context, and quality assurance in education and other fields.10Three components of quality in assessment have been described in the literature, namely, accreditation, assessment and audit.11 Clinical assessment should be benchmarked against best-practice evidence to ensure global relevance.12 Pangaro and Ten Cate recommended an assessment framework to benchmark assessment and competence against.13 In developing a framework, it is essential to clarify and/or define the terms or concepts that form the basis of the framework.14 McCall states that a good definition contains previously defined words, classifies and quantifies and has no counterexamples.15 The following terms and processes are used in this article.To assess the quality of an assessment, it must be benchmarked against appropriate criteria. Benchmarking is described as the process of comparing standards with external criteria, with the aim of improvement.13 It is important to realise that not all aspects of a framework necessarily apply to all assessments.16 A synthetic framework that integrates the domains of knowledge, skills and attitudes to ensure competence in real-life situations13 has been proposed for this study. Furthermore, for a framework to be of practical value, it should be simple enough to understand, remember and implement, while training and monitoring should form part of the implementation process.13An assessment framework can be described as a ‘common language and mental model’ that guides assessors on what to look for in student assessment to maximise the reliability of the assessment. This framework also informs students and leadership on what to expect during assessment.17Accreditation entails certification, which confirms that a programme and/or training facility is capable of fulfilling required specifications for a specific period. For instance, the South African Qualifications Authority (SAQA) accredits the providers who offer outcomes-based learning programmes that are aligned with registered unit standards and qualifications of the National Qualifications Framework (NQF).18 The University of the Free State (UFS) describes assessment as ‘the process of determining the value, significance, or extent of what students know, understand and can do with their knowledge as a result of their educational experience’.19 Assessment is therefore a comprehensive process, includes a variety of measurements for judging performance. The content and standard of assessment, types of assessment, assessment methods and principles of assessment are included under this concept.South African Qualifications Authority defines assessment as ‘a process used to identify, gather and interpret information and evidence against required competencies’ in order to make a judgement about a learner’s achievement.9 As far back as 1980, Donabedian defined quality in order to measure it.20 He concluded that quality is not one-dimensional but includes various aspects that should be agreed upon before the measurement. By applying the criteria of validity, reliability and defined concepts, quality has been described as ‘a situation when a set of inherent characteristics consistently fulfil the requirements of the organization’s … stakeholders’.21Quality, standards and relevance are key elements of quality assurance.1 The MBChB programme at the UFS is a 5-year (10-semester) outcomes-based programme presented at the UFS and the accredited training platforms of the Free State Department of Health. The clinical phase of the MBChB programme comprises the second half of the third year, the fourth year and the fifth year (semesters 6–10) of undergraduate medical training. During this phase, students rotate through the different clinical disciplines and receive clinical exposure to patients, as well as theoretical training. Formative assessment takes place during rotations and summative assessment at the end of the academic year. To progress to the next year, a student must pass assessments in all disciplines, and both the clinical and theoretical components separately.22The duration of undergraduate medical training is between 5 and 6 years, and it is offered in three phases, namely, orientation, pre-clinical and clinical training.23 The Health Professions Council of South Africa (HPCSA) also prescribes core competencies (adapted from CanMEDS) for undergraduate medical students.24 Internationally, the guidelines of the Association for Medical Education in Europe (AMEE) describe the importance of frameworks for clinical assessment,13 and the World Federation of Medical Education (WFME) has published basic standards for assessment in undergraduate training.25 Although these guides provide valuable information and principles, they are not directly transferable to the undergraduate South African and UFS medical training context where major emphasis is placed on clinical skills and clinical competence in the workplace. These guidelines focus on general programmes or assessment practices, and they do not apply specifically to assessment in the clinical phase of an undergraduate medical programme.A preliminary literature review was performed to identify frameworks, policies and criteria on quality assessment in the clinical phase of an undergraduate medical programme. At the institutional level, the assessment policy of the UFS sets minimum criteria for assessment of undergraduate students, which requires alignment with national policies and acts.What are the current regulations and policies as well as best evidence practices that inform quality assessment in the clinical phase of an MBChB programme?To compile a framework that can be used to benchmark current assessment practices based on official regulations and policies, and supported by best evidence practices to ensure defendable assessment in the clinical phase of the MBChB programme in South Africa.In this article, a rapid review of the regulations and policy documents of the bodies that regulate the assessment and training of the MBChB programme at the UFS was used to formulate a framework for clinical assessment. This framework may be helpful to benchmark the quality of assessment in the current MBChB programmes in South Africa and beyond its borders.27 Rapid reviews are typically used to inform decisions and compile guidelines.29 The single research question, narrow time frame, limited data sources, use of a single reviewer and minimal data synthesis26 justify the use of a rapid review in this article.Qualitative data were gathered using a rapid review. No formal definition or uniform method is described to conduct a rapid review although a rapid review can be described as a simplified systematic review.29 These steps are:The World Health Organization (WHO) proposed seven steps to follow for a systematic review, which may be adjusted according to the specific needs for a rapid review.needs assessment and topic selectionprotocol development with or without registrationliterature searchscreening and selection of the literaturedata extractionrisk-of-bias and quality assessment of dataknowledge synthesis.26 The components most commonly adjusted are the use of one reviewer instead of two reviewers, not conducting quality assessments of included data and not using grey data.27In rapid reviews, some of these steps are commonly simplified or omitted, but the description of the method should not be compromised.The following steps were used in this rapid review.The difficulty to defend the quality of clinical assessment in an undergraduate medical programme was identified as an area to investigate.A protocol was developed before the study commenced. The protocol limited the inclusion of primary source documents to the following:official regulations and policy documents of the regulatory bodies responsible for assessment in undergraduate medical education at the UFS and South Africainternational guidelines on clinical assessment issued by the WFMEthe AMEE guidelines on frameworks for clinical assessment.The primary outcomes to investigate were:accreditation and registrationassessmentquality assurance.The following search strategies were followed:No date limitations were placed on the documents included in the review.The search process was conducted from May to July 2019.An Internet data search was conducted.Health Professions Act, 56 of 19742 was used as the original document source for national policies and the UFS assessment policy23 for institutional policies.The These databases identified from the literature study were consulted: UFS official website, HPCSA, SAQA, Council on Higher Education (CHE), WFME and AMEE websites.The following words and/or phrases were searched: accreditation, assessment policies, assessment guidelines, clinical assessment, quality assurance in assessment, principles of quality assessment and undergraduate assessment. Searches were conducted with single words and phrases and the inclusion and exclusion of ‘AND’ and ‘OR’.Backward searching was performed using references and cross-references to related policies and regulations of the identified regulatory bodies.Forward searches of the literature entailed the search for related and updated information from the same documents or topics to ensure that all relevant information was identified.Documents were screened for selection by a single review author. Documents initially found not to meet the outcome of the study were not included but saved separately. These documents were screened a second time to ensure that relevant data were not omitted. When in doubt, the study leader could assist with selection decisions.Documents were grouped according to the primary outcomes that were accreditation and registration, assessment and audit. The assessment category was subdivided into the following subcategories:assessment content and standardsassessment typesassessment methodsprinciples of quality assessment.A table displayed the specific outcomes that were addressed by each document included in the study.This was omitted in this review, although care was taken to include all relevant documents by following the prescribed protocol. Document quality was not assessed as only official policies and regulations were included.13 were then followed to display the results visually. Finally, recommendations were made for the implementation of the framework and research limitations were discussed.For each category, the results of the review were summarised and discussed. This was supplemented by a secondary literature search to clarify concepts. The guidelines for framework development described by Pangaro and Ten CateTo ensure the credibility of the data collected and to ensure that relevant documents were included in the document review, the protocol was strictly followed. National and international guidelines were added to enable the transferability of results to other institutions. The steps followed in the rapid review were described clearly to assess the dependability of the results.The study was registered and approved by the Health Sciences Research and Ethics Committee (HSREC) at the University of Free State (UFS-HSD 2019/0001/2304). As only documents in the public domain were used for this literature review and analysis, no permission was necessary.Health Professions Act, 56 of 19742 was used as the original document source for national policies, the UFS assessment policy23 for local policies and the AMEE (https://amee.org/home) and WFME (https://wfme.org/) websites to benchmark against international assessment principles. Twenty-five documents were included in the rapid review. The MBChB programme is offered under the legislation of the Department of Health and the Department of Education (previously the Ministry of Education). The Higher Education Act, the Ministry of Education must oversee and take responsibility for norms and standards in higher education.30 To assist with this task, the Minister of Education and Training established SAQA as a juristic person who must implement the objectives of the NQF.18 The NQF was established under the SAQA Act to classify, register, publish and articulate approved national qualifications.33 Medical training is addressed under the sub-framework for higher education. The CHE, as the Quality Committee for Higher Education as provided for in the Higher Education Act, oversees the quality of training and assessment in higher education. The quality committee must register appropriate professional bodies  to ensure that qualifications meet the requirements for professional registration. The quality committees make recommendations to SAQA to register higher education qualifications.38According to the Health Professions Act makes provision for appointing professional councils  to establish professional boards. The Medical and Dental Board is responsible for overseeing undergraduate medical training, as well as registering health professionals under this act.31 The HPCSA is responsible for accrediting universities and health care training in South Africa.38The 45Because of globalisation and the increased demand for accountability in health care, the WHO and the WFME worked together on documents for the accreditation of health training institutions worldwide. The WFME gives ‘recognition status’ to an accrediting agency that meets international standards.Four components of assessment were identified, namely, assessment content and standards, assessment types, assessment methods and principles of assessment.27 South African Qualifications Authority describes knowledge as foundational competence, skills as practical competence and values as reflective competence. It also emphasises the importance of assessing prior learning, and that assessment must include content to identify and stimulate further learning.34 Assessment of values, also described as core competencies, soft skills or critical cross-field competencies, has been prescribed as components of assessment in different documents.34 Critical cross-field competencies identified were problem-solving, critical thinking, teamwork, responsibility, data management, effective communication and effective use of resources.34 The core competencies for a health care practitioner include being a professional, a communicator, collaborator, leader and manager, health advocate and scholar.24 Assessment standards are the minimum criteria that must be achieved to pass an assessment. These standards include criteria for content and difficulty, and should be reasonable, defensible and fair. Students and assessors must know all the required standards before the assessment.35 The MBChB programme is registered on NQF level 8.35 Although there is no perfect passing score, the UFS sets the pass mark at 50%.23 In Clinical Medicine, students must pass both theoretical and clinical assessments separately in order to progress.22 None of the documents addressed specific standard setting methods or processes.An assessment to ensure a competent practitioner must include elements of knowledge, skills and values.39 Some of these types may overlap or be inclusive of each other; for example, integrated assessment may take place during formative and/or summative assessment.Different types of assessment applicable to medical assessment were identified from the document review, namely, formative assessment, integrated assessment and summative assessment.23 The purpose of formative assessment is to support learning, identify learning needs and accumulate marks.23Formative assessment is described as a series of assessments that occur during the learning and training process.23 Integrated assessment is described as ‘assessment that permits the learner to demonstrate applied competence’ using different methods of assessment.34 Integrated assessment may occur at any time during the learning process.Summative assessment is the assessment that takes place after learning. The aim of summative assessment is to award grades and to validate performance and competence.25Theoretical, practical and integrated assessment methods were described, and they relate to the aim or outcome of the assessment. Theoretical assessments include multiple-choice questions, modified essay questions or short-answer questions, as well as long questions. Oral examinations can be used to test knowledge or to combine knowledge with communication skills. Clinical assessments include unobserved long cases, mini clinical evaluation exercises (mini-CEX), objective structured clinical examinations (OSCEs) and direct observation of clinical practice (DOPS). Integrated assessment methods include portfolios, logbooks, elective reports and workplace-based assessments, as well as feedback from stakeholders.19 and assessment policies,23 which are aligned with the Higher Education Acts, the NQF Act and, by implication, the Health Act, the following principles were identified:From the UFS general rulesAssessment should be an integral part of curriculum planning and must be aligned with outcomes.Assessment should be performed on the appropriate NQF level in accordance with programme registration.All assessments should be planned to cover all assessment domains.Assessment takes place in a system and must be planned accordingly.In order to be a quality assessment, each of these assessments should fulfil criteria for validity, reliability, transparency, fairness and practicability.Moderation should form part of overall, as well as individual, assessments.There should be accountability for each assessment, with evidence that the assessment was moderated.36An assessment can be considered credible if the criteria for fairness, validity, reliability and practicability have been met.23 This article focusses on quality assurance applicable to assessment and addresses moderation, benchmarking and security of the assessment process. Moderation is guided by moderation policies.41 It is a process that involves a professional judgement of the validity, reliability and fairness of the assessment and involves students, assessors and external stakeholders. The WFME sets global standards for assessment that serve as benchmarks against which those responsible for medical education can evaluate their activities.25Quality assurance policies are essential for ensuring that specifications and standards are maintained.All the primary documents necessary for the rapid review were available in the public domain on the identified websites. Information in these documents was aligned with each other. Many cross-references to other documents were found in source documents. By comparing the information in the respective documents, it was found that there was no contradiction in the documents. The data included in the rapid review can therefore be considered representative and appropriate for the purpose of this study.12The three components of quality assessment, namely, accreditation and registration, assessment and quality assurance, should from part of an assessment framework to benchmark current assessment. The inclusion of best-practice evidence in the framework will make the framework globally relevant.46 For the MBChB programme, the following must be in place:Accreditation and registration is usually not a problem for training facilities in South Africa as the HPCSA conducts regular site visits and requires annual progress reports from training facilities to ensure compliance with accreditation and registration requirements.Accreditation of the training provider and the qualification by the HPCSA.Training may take place only at a university registered with the Department of Education.The qualification must be registered with SAQA.All students in the MBChB programme must be registered with the HPCSA.47 In response, the WHO and WFME developed international accreditation criteria. The WFME awarded ‘recognition status’ to the HPCSA as the accrediting body in South Africa; all training programmes accredited by the HPCSA will therefore have internationally accredited status.43A recommendation of the 2010 Ottawa Conference was to develop criteria for accreditation of international medical educational programmes.47 A competent health care practitioner who can integrate knowledge, skills and attitudes relevant to the South African context is the ultimate outcome of the outcomes-based medical curriculum. This competency must be observable and measurable to certify the student as competent. Competency is best assessed on the ‘Does’ level according to Miller’s pyramid.6 The overarching term of workplace-based assessment may be a solution to assess knowledge skills, behaviour, attitude and self-reflection in real-life situations.48 In spite of the advantages of workplace-based assessment, Miller states that, ‘no single assessment method can provide all the data required for judgement of anything as complex as the delivery of professional services by a successful physician’.6 This is echoed when researchers warned against the use of a single assessment when pass or fail decisions have serious implications, such as for registration or licencing.49 Assessment should be a continuous process with many data points that can be taken into consideration to make an informed judgement on competence.Assessment in medical education is complex and includes various stakeholders, each with their own expectations. These stakeholders include students, teachers, lecturers, educational institutions, health care systems, regulatory bodies and patients.The quality of clinical assessment can be improved if attention is given to the following assessment principles:Validity: Content validity can be improved with blueprinting of individual as well as overall assessments, and construct validity with appropriate assessment methods.Reliability: Reliability can be improved by training assessors, enhancing the quality of questions and mark sheets, and by increasing the number of stations or questions per assessment.Fairness: Although all assessment cannot be equal, there should be no discrimination against any student, assessor or patient. It is also important that assessment should be conducted according to the NQF level that the programme is registered for.Feasibility: All assessments should be realistic, practical and sensible in the context where they take place. This can be achieved by careful planning and consideration of all resources.Educational effect: Assessment should promote learning through study for assessments, or making use of workplace-based assessment and constructive feedback.Acceptability: All stakeholders, including patients, students and the educational institution, should be satisfied with the assessment. This can be achieved through transparency and keeping all stakeholders informed.50 Moderation should form part of the overall assessment in the MBChB programme, as well as of each assessment. Moderation can be conducted internally and/or externally, and should take place before and after assessments. An external moderator should moderate all high-stakes examinations.41 The aim of moderation is to check consistency and standards.12 Benchmarking is also part of the moderation process as the aim is quality improvement.Moderation is a quality assurance process that confirms that the assessment is valid and reliable and meets minimum standards.Limitations of the study: Although the rapid review was performed according to the protocol, the risk-for-bias and quality of documents were not evaluated by a second reviewer. These results may not be 100% transferable to all MBChB programmes as different universities have different assessment policies and methods.7The complexity of clinical assessment warrants that assessment be ‘evaluated on a programmatic level’ rather than on individual assessment level, as no individual assessment meets all the criteria of validity, reliability, educational impact, acceptability and cost.This rapid review attempted to develop a framework to benchmark the quality of assessment in the clinical phase of an undergraduate medical programme. As a first step, all stakeholders should be aware of the outcome of the programme. All assessment and training in the MBChB programme must be aligned with the outcome of the programme, namely, to produce a competent medical practitioner who can integrate knowledge, skills and attitudes relevant to the South African context.Open-mindedness is essential during the assessment process. Programme accreditation, assessment practices and psychometrics can assist to improve the quality of assessment but cannot judge clinical competence. Experienced assessors, using a variety of assessment methods on a continuous basis, is the proposed way to go. By implementing quality assurance processes, institutions can ensure that specifications and standards are maintained and improved, and that they are globally competitive. It is clear that clinical assessment is multidimensional and that no assessment is perfect. An assessment framework can assist to improve assessment, but it cannot guarantee quality assessment.


Multiple sclerosis (MS) is a neurological disease that typically affects young women of reproductive age. There are still many questions and heterogeneous clinical approaches partly due to the lack of consensus and guidelines. For many years, women with MS have been discouraged from getting pregnant for fears that the disease might negatively affect the fetus or increase their obstetric risk or for claims that the disease might decrease fertility. However, fertility does not seem to be impaired to a larger extent in women with MS. Therefore, all healthcare providers involved in the follow-up of multiple sclerosis patients must be prepared to discuss future fertility, pregnancy, and others matters, in addition to providing them with the best possible counseling. This study presents data based on updated evidence and discusses fertility and pregnancy in patients with MS with respect to the impacts of pregnancy on the risk and prognostic factors tied to MS, and the impact of MS on pregnancy outcomes and fertility treatments administered to females with MS. In conclusion, a clear relationship between infertility and MS has not been established. There seems to exist a link between disease aggressiveness and progression with several processes that might impair fertility. However, MS does not stand as a contraindication to assisted reproductive technology. From the several studies analyzed, it is possible to conclude that pregnancy is possible in women with MS. It is important to discuss and plan the ideal moment to start treatment and managing pregnancy and contraception aiming at better results. Multiple sclerosis (MS) is a neurological disease characterized by motor, sensory and/or cognitive impairment interspersed with episodes of remission, triggered by an abnormal demyelinating immune response in the central nervous system and spinal cord. It affects people aged 30-40 years on average. The prevalence ratio of multiple sclerosis from women to men has increased sharply in recent decades (2.3 to 3.5:1), which indicates a significant increase in multiple sclerosis among women , in association with environmental factors such as sun exposure, prolonged low vitamin D levels, smoking, and latent infection by some types of Epstein Barr virus. When associated, triggering factors generate a cross-autoimmune response with myelin proteins, triggering the destruction of oligodendrocytes, axonal damage, perivascular inflammation, and chemical changes in myelin lipid constituents. The lesions form atrophic plaques containing fibrosis, which are mainly found in the lateral and posterior columns of the spinal cord, brain white matter, optic nerves, and periventricular areas. The characteristics associated with the location of each plaque correlate with the seriousness and clinical presentation (subtype) of the disease . When stratified by age, a greater proportion of women with MS aged 18 to 34 and aged 42 years and older have been diagnosed with infertility compared to women without MS  AND (Pregnancy) AND (Fertility). The following were included in this review: cohort case studies; case reports; cross-sectional, experimental, and pilot studies; and randomized clinical trials  and its correspondents in the Descriptors in Health Sciences (DeCS):  treatments enrolled few patients and did not allow for significant data analysis. ART was associated with a 7-fold increase in the risk of MS exacerbation and with a 9-fold increase in the risk of enhanced disease activity as seen in MRI scans of lesions in patients prescribed the pituitary block protocol with a GnRH analog, and not when the block was performed with antagonists. There were no differences regarding injectable gonadotropins used to induce ovulation. Significant increases in relapsing disease were associated with the use of GnRH agonists and IVF failure. The proposed mechanisms included: cessation of DMT, stressful events associated with infertility, and immunological changes induced by GnRH, as well as augmented immune cell migration across the blood-brain barrier.There was no difference between levels of FSH, LH, inhibits, estrogen, progesterone, free testosterone, anti-Müllerian hormone (AMH), or ovarian reserve in women with controlled MS or without the disease. However, patients with uncontrolled disease and subjects with active MS had smaller ovaries with fewer follicles and decreased AMH levels. No relationship was observed between MS and the production of anti-ovarian antibodies.Despite the small sample size and the limited number of studies examining the subject, the data suggested that the use of assisted reproductive technology is not contraindicated in patients with MS; instead, it is recommended that ART be performed during a stable period of the disease and that treatment be maintained until pregnancy has been confirmed.Pregnancy in patients with MS and its implications in the eyes of physicians and individuals with the disease have been scarcely discussed, although three quarters of the patients with MS are women of childbearing age and at least a fifth of them will eventually have children after the onset of the disease. Proper management of patients in this situation requires multidisciplinary knowledge and study  conducted in Europe in 1998, which described a reduction in the rates of disease recurrence during pregnancy, especially in the third trimester , and progesterone trigger changes in the lymphocytic response profile. They suppress the T-Helper 1 (TH1) response, thereby reducing the release of interferon gamma and tumor necrosis factor alpha associated with MS episodes, in addition to increasing the TH2 response related to the circulation of interleukins 4 and 10, involved in maternal immune tolerance toward the presence of the fetus, thereby reducing the intensity and number of episodes of MS during pregnancy (It is well established in literature that pregnancy is safe in patients with controlled MS. Evidence suggests that minimizing the frequency of episodes before pregnancy correlates with better outcomes in the immediate postpartum . CurrentMost studies show no significant difference in fecundity and fertile capacity of women with early controlled MS. However, the use of assisted reproductive technology is five times greater among patients with advanced disease (The immune response of MS also increases the prevalence of endometriosis through shared mechanisms still unknown. In addition, some drugs used in treatment such as cyclophosphamide and mitoxantrone have deleterious effects on oocytes, decrease the ovarian reserve, and causing amenorrhea in up to 30% of the patients. These events might increase the indication of assisted reproductive technology for affected women (Sexual dysfunction affecting patients with MS related to depression and decreased quality of life should also be considered. Women with MS suffer from loss of sensitivity in the genital region, with ensuing changes in libido and primary impairment of sexual function affecting fertility .Overall, women with MS are less likely to have a live birth, more likely to have a diagnosis of infertility, and less likely to receive infertility treatment compared to women without MS (The low number of pregnancies among patients with MS might be explained by the need to postpone pregnancy as a result of the diagnosis of the disease and the prescription of therapies that include medication with potentially adverse pregnancy-related side effects. The burden of functional and cognitive impairment and their impacts on childcare, the fear of pregnancy triggering the progression of the disease, the impact of the disease on gestational risk and the fetus, and the risk of transmitting the disease to offspring have been described by MS patients as factors that discourage reproduction (It is essential to navigate through the prescription of therapy, disease management, and pregnancy timing while engaging in multidisciplinary discussions on family planning (From the several studies analyzed, it can be concluded that pregnancy is possible in women with MS. This matter should be discussed with patients with MS, since some avoid getting pregnant for fear of complications and for being unaware of the positive relationship between pregnancy and the disease and the possibility of using drug classes that are harmless for the mother and the fetus. In addition, the use of contraceptives, the ideal point in time to start treatment, and pregnancy preparations must be planned properly so that the best outcomes for the mother and fetus are achieved.The relationship between infertility and MS is not fully understood. There seems to exist a link between aggressiveness and progression of the disease and several processes that may impair fertility. However, MS does not contraindicate the use of assisted reproductive technology as needed. Further studies should be conducted to better define the interactions between the disease and infertility, along with the best procedures indicated for each stage or group of therapies in use.


The operating room nurses have encountered several challenges during the current COVID-19 Pandemic, which subsequently impact their clinical performance. The present study aimed to investigate the experiences of operating room health care professionals' regarding the COVID-19 pandemic challenges in southern Iran.This is a descriptive qualitative study. The required data were collected using in-depth and semi-structured interviews. Accordingly, 19 operating room health care professions were included in this study through purposive sampling. The data were collected from February 2020 to August 2021 and then analyzed using the Granheim and Lundman’s method.Three main themes and 9 sub-themes were presented in this study. They included: 1- Challenges of applying protocols and standards of care in the COVID-19 pandemic in the operating room . 2- Professional challenges . 3-Psychological crises .Based on the findings of the present study, it was indicated that lack of a specific guideline in the COVID-19 pandemic led to a sense of indecision and confusion among operating room staff. In this regard, performing tedious tasks, feelings of tiredness and exhaustion, lack of support by the authorities, and lack of access to adequate facilities and equipment were reported as the professional challenges of the healthcare staff in the current COVID-19 pandemic, which consequently affected the quality of care and patient’s safety. It seems that hospital officials can use the results reported in this study for efficient planning and effective management during the COVID-19 pandemic. The coronavirus pandemic with a highly increasing mortality rate is one of the main threats for the health status of human beings, especially professional health care personnel in the current century –3. The WIn Iran, the operating room staff includes surgeons, anesthesiologists, operating room nurses, and anesthesia nurses who are mostly experiencing lots of professional challenges and stresses in the current COVID-19 pandemic. Due to the fact that we are currently experiencing the fifth peak of the Corona pandemic in Iran, the health team is tolerating lots of professional and psychological pressures due to many challenges they are faced with. A number of operating room nurses and anesthesia nurses have been infected with Coronavirus and some have lost their lives. Awareness of the challenges and problems to which the operating room and anesthesia nurses are faced during COVID-19 crisis management is essential in designing management strategies to control the current infectious disease crisis. A previous review of the studies in this regard showed that few studies qualitatively explained the care challenges of operating room staff and physicians in the COVID-19 crisis. Therefore, to understand the concept and determine its dimensions deeply, performing a qualitative study is useful because such studies help to better explain the concept in the cultural context from the perspective of people who have a long-term involvement with that phenomenon .In addition, providing quality care in the operating room is not possible unless by considering the rich experiences of the staff and physicians in the field of care challenges. Without awareness about the experience of operating room staff and physicians regarding emotions and difficulties in managing patients during crises caused by infectious and contagious diseases, it would not be possible to identify and provide appropriate solutions, thereby increasing the quality of care. Therefore, the present study aimed to investigate the experiences of operating room nurses regarding the COVID-19 pandemic challenges in southern Iran.This is a qualitative research performed using a conventional content analysis approach. Conventional content analysis was also used when little information was available on the concept. Since the experiences of operating room staff and physicians regarding the challenges of COVID-19 crisis management in Iran are not fully understood yet, we made an attempt to use the mentioned approach. In reporting this qualitative study we have adhered to the COREQ  guidelines .23 operating room health care professions were invited to participate in the study. However, 4 operating room health care professionals refused to participate due to their busy schedule and intensive shifts. Therefore, 19 participants  from February 2020 and August 2021 were selected by purposive sampling. This is a technique widely used in qualitative research for the identification and selection of information-rich cases for the most effective use of limited resources. This involves identifying and selecting individuals or groups of individuals that are especially knowledgeable about or experienced in the phenomenon of interest. In addition to knowledge and experience, availability, willingness to participate, and the ability to communicate experiences are of importance.The inclusion criteria were willingness to participate in the study and at least one year of work experience. Sixteen individuals were enrolled, and then semi-structured interviews were done to collect the data. Each interview lasted based on the participants’ willingness.The participants were video called for interviews on WhatsApp at times which were convenient for them. The first author, who is an assistant professor of nursing and has carried out numerous qualitative studies performed and analyzed all the interviews. Interviews continued until data saturation. Saturation occurs when there is no new category emerging, and the categories are saturated based on their characteristics and dimensions .The interviews started with general questions, including “Describe your first work experience with the COVID-19 disease crisis” and then continued by asking specific questions such as “Based on your experience, what challenges or job stresses are operating room staff faced with in the current COVID-19 pandemic? and “What solutions do you suggest for managing the COVID-19 crisis in the operating room Moreover, to increase the clarity of information, some probing questions such as “Can you explain more?", "What do you mean?", and “Can you give an example?" were asked. The interview process was followed considering the main purpose of this research. The interviews lasted for about 35 to 60 min. At the end of each interview, the author firstly listened to the interviews several times and then transcribed them on paper. Afterward, data analysis was performed simultaneously with data collection, immediately after each interview, and then the next interview was planned based on the findings of the previously performed interviews.Immediately after the first interview and simultaneous data collection, the data were analyzed using Granheim and Lundman’s method in 5 steps. First, the recorded speeches were transcribed; then, by carefully studying the manuscripts, the researcher tried to find the external and internal elements of the speech. In other words, immersion in the data was done to gain the participants' understanding first. In the nest stage, by re-reading the manuscripts, the semantic units were identified as sentences or paragraphs from the statements and texts of the interview. Fourth, the initial codes or open codes were extracted from it. In open coding, the text of each interview was read several times and the main sentences were extracted and recorded as codes. The related codes extracted based on similarity were then placed in subcategories. Finally, subcategories were merged to create the final themes . For thiThe accuracy and trustworthiness of the data obtained were tested using the criteria suggested by Guba and Lincoln . In ordeThe present study was conducted in terms of the principles of the revised Declaration of Helsinki, a statement of ethical principles that directs physicians and other participants in medical research involving human subjects. Prior to performing the interviews, all the subjects were informed about the objectives of the study, voluntary nature of their participation, data collection methods, reason for recording the interviews, role of the interviewer and the participants, and confidentiality and anonymity of the information. Subsequently, they were asked to sign the informed consent form if they were willing to participate. The participants were notified that they would be free to withdraw from the study at any time.The means of the participants’ age and length of work experience were 35. 73 ± 5.42 years old and 9.52 ± 3.12 years, respectively. Other demographic characteristics of the participants are shown in Table COVID-19 crisis management challenges in the operating room led to the extraction of three themes and nine subthemes. Correspondingly, these three main themes included the challenges of applying the protocols and standards of care in the operating room, professional challenges, and psychological crises. Table The first theme was the challenges of applying protocols and standards of care in the Covid-19 pandemic in the operating room, which included lack of specific guidelines regarding the protocols and care standards in crisis management of COVID-19 in the operating room as well as the impossibility of social distance in the operating room."There is no single, comprehensive guideline for operating room staff care standards regarding COVID-19. They want to apply the same instructions that have been developed for the staff of other wards of the hospital for the operating room staff, but the environment and working conditions in the operating room are very different from those of other wards. We come across a large number of instructions every day that have no efficiency. It is better to develop a single and specific instruction to be used" (Participant 8).One of the challenges of the operating room staff in managing COVID-19 in the operating room was found to be the lack of specific protocols and standards in the COVID-19 pandemic from the participants' point of view. Accordingly, from their perspectives, the operating room personnel are faced with a new care instruction every day, which leads to both uncertainty and confusion among the staff. In this regard, they do not really know which instructions to follow."Operating room supervisors and clinical supervisors emphasize that you must keep your social distance, but they offer no solution. Is it possible for a team of staff and surgeons to work closely with the patient? I say stay away from each other and get away from the patient's bed. A moment of negligence and being away from the patient during surgery may endanger the patient's life." (Participant 17)The second issue in the field of challenges of applying protocols and standards of care in the current COVID-19 pandemic in the operating room from the participants' point of view was found to be the impossibility of keeping social distance in the operating room.The second major theme in this study was professional challenges, which includes tedious work, professional burnout, lack of adequate support, and lack of sufficient equipment."We really don't have energy left for us anymore. Working in the operating room is full of stress and in the COVID-19, This stress has increased a lot. In some situations, an operation lasts 8 to 10 hours, working in special clothes in COVID-19 condition in the operating room is very difficult and tedious, and the staff endures lots of stress and work pressures.” From the participants' point of view, burnout is another professional challenge for operating room staff in the current COVID-19 pandemic“Staff in these difficult and complex situations do not receive support from the directors of the institution. Managers do not take any effective action to motivate and boost the staff’s morale. They even fail to pay their wages under this COVID-19 condition. They did not even give a letter of appreciation to the staff. They hold no meeting with the staff to listen to their problems in this difficult and exhausting situation.” (Participant 19)One of the most important professional challenges in the COVID-19 pandemic from the perspective of operating room staff is the lack of adequate support by the managers. From these participants' point of view, during the period of the current COVID-19 pandemic, despite the fact that the staff are working hard while enduring a lot of stress, the managers do not provide enough support for them and even fail to provide their personal protective equipment."Operating room staff have mostly faced a shortage of personal protective equipment. Moreover, in some situations, a simple mask is not found in the ward. Face shields and special clothing are also scarce. Besides, the ventilation system in the operating room does not work well, so equipment providing better ventilation should be used. Many patients do not take the COVID test at all, but they are sent to the operating room. After the operation, we will find out that the patient's COVID test was positive. There must be facilities to perform a rapid COVID test, which is not available in our country unfortunately.” (Participant 10)Lack of personal protective equipment was another professional challenge for operating room staff during the COVID-19 pandemic.The third major theme was psychological crisis, including indescribable anxiety and worry, parenting stress, and impaired resilience. From the perspective of the included participants, the operating room staff experiences various psychological crises during the current COVID-19 pandemic.“Operating room staff are experiencing numerous and severe psychological crises during the COVID-19 pandemic. Currently, in the fifth wave of the disease, these stresses and crises have doubled. We see our colleagues suffer from this disease, and in some cases our colleague has lost his/her life. These moments are very sad and have caused operation room staff to develop depression and anxiety disorders.” (Participant 18)Indescribable anxiety and worriedness were one of the psychological crises from the participants' points of view. Psychological stress due to having a direct contact with the infected patients and fear and anxiety about the high probability of this disease led to indescribable anxiety and worry among the staff.“The COVID- 19 pandemic has had a profound effect on family functioning. We really do not know how to support and take care of our children in this COVID-19 pandemic, and we are all stressed and worried that we might pass this disease to our children. I have to take care of my child on my own. When I have to stay in the hospital for a few days due to the COVID-19 condition, I am always worried about my child who is at home.” (Participant 2)Another psychological crisis from the perspectives of the participants was parenting stress.“We have no more patience and endurance; we have gone through five terrible and exhausting waves of the COVID-19 pandemic and endured lots of stress, so we are really tired and cannot continue tolerating this condition. All hospital managers tell us to be patient and tolerant; how can we tolerate this situation? How much really? How long?” (Participant 13)Another psychological crisis described by the participants was the weakened resilience.This study aimed to explore the experiences of operating room staff regarding the challenges during the COVID-19 pandemic in southern Iran. Based on the obtained results, the challenges of operating room staff have emerged in the form of the following three main themes: challenges of using protocols and standards of care in the operating room, professional challenges, and job stress.One of the main challenges for operating room staff was the application of the protocols and standards of care during the COVID-19 crisis. In this study, operating room staff reported that they experienced lack of specific guidelines on care protocols and standards for the management of COVID-19 crisis and the impossibility of keeping social distance in the operating room. However, in the study conducted by Nasiri (2020), the operating room staff kept their distance and had minimum contact with other staff and patients to maintain the safety of both staff and patients. Moreover, medical services were provided in the shortest time when the patient was in the operating room . In the The professional challenges experienced by operating room staff were among the other findings of the present study, which included tedious work, burnout, lack of adequate support, and lack of adequate equipment. In a study conducted in Iran, stressful work environment, doubts on the occurrence of unexpected conditions, lack of support received from officials, and lack of adequate equipment were among the challenges experienced by the nurses who provided healthcare to the patients infected with COVID-19. In addition, staff shortage and lack of equipment were indicated to affect the quality of nursing care .The results of some other studies also showed that most of the problems experienced by nurses during caring for and dealing with COVID- 19 patients were as follows: staff shortage, fear of getting infection, and excessive fatigue due to long working hours without any proper nutrition , 22. LacTherefore, to improve the quality of nursing care and health services in the current COVID-19 pandemic, the hospital officials are recommended to pay more attention to the identification of the staff’s needs and providing adequate resources and facilities. In this regard, it is important to provide adequate manpower and support to officials through paying attention to the staff’s nutrition, organizing their working hours, providing adequate protective equipment, holding stress management training sessions, and providing psychological and medical support to them.Another finding of the present study was the psychological crises of the operating room staff, which included their indescribable anxiety and worry, parenting stress, and impaired resilience in the COVID-19 crisis. In the carried out study by Nasiri (2020), the operating room staff reported that they experienced several mental conflicts during dealing with patients infected with COVID-19. All the participants expressed fear and concern about their infection and that of their family and relatives. Having small children and elderly parents with underlying diseases in the family added to their stress and anxiety. Among the main causes proposed for having the fear of COVID-19, unknown and complete lack of knowledge about coronavirus, rapid spread of lethality, and lack of vaccine and definitive treatment for this disease were noted as well .The results of previous studies performed in Iran showed that nurses in COVID ward mostly experienced high anxiety. Based on the participants' experiences, Corona anxiety leads to fatigue, decreased concentration, sleep disturbance, decreased resilience, and burnout; also, it threatens the nurses' mental safety –20. AlsoAccording to the results of a study conducted during the COVID-19 pandemic, being a parent adds to the individuals’ stress. In this regard, workplace stress, fear of transmitting the infection to their children, necessity of staying in the hospital to stay away from their children, and inability to monitor the upbringing of children were the mentioned issues that have caused the nurses to experience excessive stress and consequently fatigue and burnout 7 .In another study, nurses were overworked and worried that their children and family members would be infected with this disease. Therefore, besides feeling tired due to a great deal of work as well as fear and worry about endangering their health and families, they have suffered from high anxiety and stress levels . TherefoOne of the limitations of the present study was that the interviews were individually conducted only with operating room and anesthesia staff. Therefore, performing more focused interviews and also interviewing with other members of the operating room team, such as surgeons and anesthesiologists, can provide richer information in this regard.To the best of our knowledge, this is the only study performed with a qualitative approach in Iran, aiming to explore the experiences of the operating room staff regarding the challenge of COVID-19 in the operating room, which is considered as a type of innovation. In addition, this study has not been done in other countries with a qualitative approach, or if done, it is very limited.Based on the findings of the present study, it was indicated that lack of a specific guideline in the COVID-19 pandemic led to a sense of indecision and confusion among operating room staff. In this regard, performing tedious tasks, feelings of tiredness and exhaustion, lack of support by the authorities, and lack of access to adequate facilities and equipment were reported as the professional challenges of the healthcare staff in the current COVID-19 pandemic, which consequently affected the quality of care and patient’s safety that have severely affected their professional performance and mental health. Therefore, health system managers and policy- makers can use the results of this study for efficient planning and effective management of operating room conditions during the COVID-19 pandemic and take key steps to reduce the challenges and needs of the staff in the COVID-19 crisis. Accordingly one of the priorities of the administrators of operating room centers under these conditions should be preserving the psychological health of the personnel who are dealing with COVID-19 patients. Development of evidence-based operating room care protocols and educational programs for raising the personnel’s awareness and facilitating their adaptation to the challenging conditions of the COVID-19 crisis will improve the quality of operating room care services.


SimilarTT_0615C  noted thConsequently, the research community would benefit from a consensus standard for tracking provenance of microbial stocks used in different applications. This need is especially critical for microbiologists involved in developing various health care applications, such as diagnostics, vaccines and therapeutics. Microbial reference materials used in these applications are obtained from different sources and are often not qualified/certified to the same set of standards, making it difficult for results to be confidently compared.One way to limit and monitor the genetic drift in laboratory handled strains, in addition to minimizing the passage and handling of properly stored lots, is by encouraging researchers and culture producers to carefully document the histories of bacterial cultures and routinely screen them for divergent genotypic or phenotypic signatures. This guideline establishes a framework for investigators to use in documenting the relationships among microbial cultures used in scientific studies.2These guidelines establish the roles and responsibilities for sponsors, performers, and culture producers with respect to the verification of relatedness among test and index cultures used in an extensible study. While not broadly enforceable, the guidelines are intended to create a framework and a set of expectations for properly qualifying and documenting the provenance of microbial cultures used in scientific studies.3extensible study is a research program whose results and conclusions are expected to apply equally to test and index cultures.An test culture is the microbial culture derived from the index culture that is being evaluated. The index culture is the reference culture to which the assay results are to be applied.In an extensible study, the traceable cultures, meaning that each has a unique identifier  and well-documented propagation history.Both the test and index cultures must be Extensible studies are generally supported by:Sponsors.—Establish the experimental objectivesCulture producers.—Manufacture and characterize the study’s traceable culturesPerformers.—Conduct the studyThese roles can be filled by the same or different organizations. However, each role has specific responsibilities with respect to the culture verification process.Culture verification is the process by which the species in a test culture is shown to be sufficiently related to that in an index culture to allow the meaningful extension of experimental results from one culture to the other. The relationship between the test and index cultures should be established via propagation history and orthogonal testing. It may also be desirable to use application-oriented testing to ensure study-specific similarities between the cultures.Propagation history describes a test culture’s step-by-step derivation from the index culture via a series of propagation events. These data are an essential part of the culture verification process because a culture’s propagation history is impossible to recover through empirical means. Furthermore, production and handling details provide important clues to the health and disposition of the culture that may not be evident through empirical observations, including potential changes in the genetic makeup.Orthogonal testing is the use of functionally independent assays to verify the genotypic and phenotypic relatedness of test and index cultures. Orthogonal testing is important for identifying genetic and physical changes that might have resulted from laboratory handling and could impact the validity of an extensible study.Application-oriented testing is designed to assess the relationship between the test and index cultures with respect to the specific genotypic or phenotypic phenomena being evaluated in the extensible study. For example, if the study relates to microbial virulence, some effort should be made to show that the virulence of the test strain resembles that of the index strain.Culture verification statements provide a convenient mechanism for documenting the relatedness of microbial cultures used in extensible studies.4An extensible study’s sponsor, culture producer, and performer have distinct roles and responsibilities with respect to the culture verification process. Each of these roles and their associated responsibilities are described below and can be filled by the same or different organizations.4.1The sponsor is the entity that defines the study’s objectives. The sponsor’s principal responsibilities with respect to culture verification are to approve the index cultures to which each of the test cultures must relate and to determine the acceptable level of relatedness. These responsibilities are generally accomplished by specifying the culture producer and lot number  of each of the study’s index cultures and ruling on the adequacy of performer-supplied culture verification statements.When selecting index cultures and approving culture verification statements, the sponsor should be aware of how their decision will impact project costs. Limited availability of singled-sourced cultures or capable culture providers can drive verification costs upwards. Furthermore, the paucity of producer-supplied provenance records and test data could make it costly, if not impossible, for the performer to adequately demonstrate the relationship between the test and index cultures.4.2Under these guidelines, it is the responsibility of the culture producers, who manufacture the test and index cultures, to provide the performer with any nonproprietary information that can help the performer demonstrate the relatedness of the test and index cultures. It is understood that the culture producer may not have or be able to release all of the information that the performer needs to complete a culture verification statement. However, the lack of supporting data may drive the sponsor or performer to select different sources for their index and test cultures. Internal proprietary information and HIPAA-related materials are examples of content that may not be sharable with the performer.4.3Organization(s) that are tasked with conducting the scientific study fill the role of performer. It is the performer’s responsibility to demonstrate to the sponsor’s satisfaction that they are using test cultures that are sufficiently related (as determined by the sponsor) to the index cultures to support meaningful comparisons and conclusions within the scope of the extensible study. The performer meets this obligation by providing the sponsor with a culture verification statement for each of their test cultures. The verification statement summarizes and references enough data to convincingly demonstrate the provenance and empirical equivalence of the test and index cultures.Most, if not all, of the information contained in the culture verification statement should be available from the culture producer, who likely generates provenance and test data as part of their production and quality control processes. However, in some cases the performer may be required to complete additional orthogonal or application-based testing on the test culture, as dictated by the specific study.5The performer is expected to demonstrate, to the sponsor’s satisfaction, that their test cultures are sufficiently related to the study’s index cultures by documenting passage/subculture history, orthogonal test results, and application-specific assay outcomes in a culture verification statement. Typically, the performer will not have independent access to the records or resources necessary to fully demonstrate culture relatedness at the level recommended by this guide. Rather, they will often rely on information available from the culture producer to verify the relatedness of the test and index cultures. The culture producer will generally transmit this information to the performer via certificates of analysis, product information sheets, and direct communications.5.1Appendix A, a well-constructed culture verification statement should relate the test culture to the index culture via the culture’s propagation history, orthogonal test results, and optionally application-oriented test results.As illustrated in 5.1.1The culture verification statement should contain a propagation history, which describes the production and handling of the test and index cultures as well as any cultures that constitute the intervening lineage . Each of these cultures should be linked to its predecessor via a documented production method. The performer should specifically identify the starting cultures, culture producer, lot number, production date, and production method of each culture in the passage history. With respect to the production method, the performer should describe the materials and methods used to derive the referenced culture from its predecessor in the lineage declaration. If the current culture is the first in the chain, the production method should clearly describe the clinical, environmental, or laboratory origin of the culture and the method used to propagate the traceable culture from that origin.5.1.2It is also important that the culture verification statement describe how available orthogonal test results address the relatedness of the test and index cultures. Orthogonal testing relies on multiple analytic techniques to compare one culture to another. Cultures can be compared with respect to morphology; genotypic and phenotypic properties; metabolic, immunological, and molecular features; molecular functions; and virulence. However, the quality of microbial verification and confidence associated with it ultimately depends on the number, type, and diversity of applied assays. Culture producers should strive for more comprehensive approaches to orthogonal testing. Minimally, orthogonal testing should include a mix of genotypic and phenotypic assays. 5.1.3While orthogonal testing is intended to uncover unexpected changes that might occur in cultured bacteria during laboratory passage and handling, application-oriented tests are used to confirm that laboratory propagation did not adversely affect properties that relate directly to the planned extensible study. Although application-oriented test results do not need to be included in the culture verification statement, incorporating such data adds scientific credibility and demonstrates that the test and index samples perform comparably on assays that relate to the specific extensible study. AOAC Stakeholder Panel on Agent Detection Assays (SPADA) Working Group for Bacterial Strain Verification:Co-Chair), Joint Research and Development (JRAD) supporting Joint Program Executive Office (JPEO) Joint Project Leads (JPL) Chemical, Biological, Radiological, and Nuclear Defense (CBRND) Enabling Biotechnologies (EB); Deputy Under Secretary of the Army, Test and Evaluation (DUSA TE)Linda Beck (Co-Chair), Logistics Management Institute supporting Defense Biological Product Assurance Office (DBPAO), JPL CBRND EBShanmuga Sozhamannan  Chemical Biological Center (CBC); Dugway Proving GroundCory Bernhards, CCDC CBCThomas Blank, National Biodefense Analysis and Countermeasures Center (NBACC)Trevor Brown, Venesco, LLC, supporting JPEO-CBRND Joint Project Management Office for CBRN MedicalMatthew Davenport, U.S. Department of Homeland Security (DHS)Bruce Goodwin, U.S. Department of Defense (DoD) JPEO JPL CBRND EBRandy Hofmann, CCDC CBCPaul Jackson, Lawrence Livermore National Laboratory (LLNL) and Los Alamos National Laboratory ; Middlebury Institute of International Studies (Adjunct Professor)Scott Jackson, National Institute of Standards and Technology (NIST)Katalin Kiss, ATCCNancy Lin, NISTTimothy Minogue, United States Army Medical Research Institute of Infectious Diseases (USAMRIID)David Rozak, Unified Culture Collection, USAMRIIDMark Scheckelhoff, Armed Forces Health Surveillance BranchSanjiv Shah, U.S. Environmental Protection Agency (EPA)Charles Young, Johns Hopkins University Applied Physics Lab (JHU/APL)Sharon Brunelle , AOAC INTERNATIONALDeborah McKenzie (Staff Liaison), AOAC INTERNATIONALThe guidelines were reviewed by SPADA and approved on October 15, 2019.Certain commercial equipment, instruments, or materials are identified in this paper only to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.


A digital tool, Multiple Sclerosis Progression Discussion Tool (MSProDiscuss), was developed to facilitate discussions between health care professionals (HCPs) and patients in evaluating early, subtle signs of multiple sclerosis (MS) disease progression.The aim of this study is to report the findings on the usability and usefulness of MSProDiscuss in a real-world clinical setting.initial individual questionnaire  and a final questionnaire . The responses were provided on a 5-point Likert scale. All analyses were descriptive, and no statistical comparisons were made.In this cross-sectional, web-based survey, HCPs across 34 countries completed an initial) to 98.2%  of the cases. In 93.54%  to 97.1%  of the cases, the HCPs agreed  that patients were able to comprehend the questions in the tool. The HCPs were willing to use the tool again in the same patient, 90.47%  of the cases. The HCPs reported MSProDiscuss to be useful in discussing MS symptoms and their impact on daily activities  and cognitive function , as well as in discussing progression in general . While completing the final questionnaire, 94.9% (260/274) of the HCPs agreed that the questions were similar to those asked in regular consultation, and the tool helped to better understand the impact of MS symptoms on daily activities  and cognitive function . Overall, 92% (252/274) of the HCPs reported that they would recommend MSProDiscuss to a colleague, and 85.8% (235/274) were willing to integrate it into their clinical practice.In total, 301 HCPs tested the tool in 6974 people with MS, of whom 77% (5370/6974) had relapsing-remitting MS, including those suspected to be transitioning to secondary progressive MS. The time taken to complete MSProDiscuss was reported to be in the range of 1 to 4 minutes in 97.3% (6786/6974; MSProDiscuss is a usable and useful tool to facilitate a physician-patient discussion on MS disease progression in daily clinical practice. Most of the HCPs agreed that the tool is easy to use and were willing to integrate MSProDiscuss into their daily clinical practice. Multiple sclerosis (MS) is a chronic debilitating disease of the central nervous system that primarily affects young adults . In mostThe Multiple Sclerosis Progression Discussion Tool (MSProDiscuss) is a digital tool for use by health care professionals (HCPs) in clinical practice to raise awareness of the risk of progression from RRMS to SPMS through a structured interaction between HCPs and patients. MSProDiscuss aims to help physicians, in dialog with patients, to evaluate early, subtle signs suggestive of MS disease progression . The tooWe conducted a separate usability testing study to assess the performance of MSProDiscuss on a larger scale, involving HCPs from different geographies and health care systems, with the aims to further test the (1) usability of tool in daily clinical practice, and comprehensibility of items included: (2) usefulness of the tool to assist patient-physician discussion on MS disease progression, (3) feasibility, ease, and willingness of HCPs to integrate MSProDiscuss into their routine clinical practice, and (4) insights gathered from HCPs on areas of improvement on the tool’s usefulness. In this paper, we report the findings from physicians on the usability and usefulness of MSProDiscuss while discussing disease progression with patients in a real-world setting.This was a multinational, cross-sectional, and noninterventional study surveying HCPs, including MS specialists, general neurologists, or others as indicated by the self-identification choices provided to the participants. The HCPs responsible for the diagnosis, management, and care, or those in charge of symptom evaluation and control, of at least five patients with MS per week in their daily clinical practice were included in the survey. The HCPs were invited by local representatives of the study sponsor, Novartis, to participate in a web-based survey conducted between July 2019 and December 2019 in 34 countries across North America, Europe, Asia, South America, Africa, and Australia. No ethics approval was required because this was a survey. Nevertheless, all respondents were offered detailed information about the survey method, and they provided written consent before participating.The survey was carried out by Real World Evidence Solutions (IQVIA AG). A web-based survey link was sent to the HCPs who consented to participate in the study. The survey consisted of 2 questionnaires to gather feedback on the understanding, usefulness, usability, integration, and adoption of MSProDiscuss in daily clinical practice. The questionnaires were developed based on feedback from HCPs during the tool development phase and according to the principles of implementation science, and inputs provided by the sponsor’s medical team and by researchers from Real World Evidence Solutions were included. The initial translations of the questionnaires were provided by IQVIA, which were further evaluated and modified as necessary by the local Novartis medical teams. The HCPs used MSProDiscuss to assess what they felt was a broad range of people with MS, excluding those with clinically isolated syndrome and primary progressive MS. The survey methodology is illustrated in Each HCP was asked to fill two types of questionnaires , with thThe HCP responses to the 2 usability questionnaires were analyzed separately with regard to the individual and final questionnaires and descriptively on a question-by-question basis and reported as a proportion of the total responses. The differences in the responses to the questionnaires by country were summarized by region and visualized as a heat map. No statistical comparisons were performed. To ensure that the survey sample was representative of the true population, both weighted and unweighted percentages were calculated. The weighted analysis ensured that the results were adjusted to reflect the underlying sample distribution. Therefore, weighting was performed at the country level with regard to the survey sample size. For the individual questionnaire, this refers to the number of patients for whom MSProDiscuss was used, whereas for the final questionnaire, the number of HCPs who participated in the survey was relevant . The feeOf the 390 HCPs who were invited, 301 provided feedback on at least one questionnaire. Most of the participants were MS specialists , but also included general neurologists, MS nurses and nurse practitioners, and physician assistants . In a suOf the 301 HCPs who participated, 232 completed the expected number of questionnaires. During the MSProDiscuss usability test on a total of 6974 patients, the HCPs identified 5370 (77%) patients with RRMS, which also included those who may be transitioning to SPMS. The number of patients per HCP and the total number of individual consultations in the 5 European countries are shown in The usability and usefulness of the tool were assessed using both the initial and final questionnaires . The HCPThe final questionnaire assessed additional usability aspects from the perspective of the HCPs. There was general agreement  that the questions in the tool were similar to those asked by an HCP during a regular consultation. MSProDiscuss was also found to be helpful in understanding the impact of MS symptoms on daily activities  and cognitive function . The findings based on the weighted analysis were similar to those based on the unweighted results  and 5. OOn the individual questionnaires, the responses from the individual countries and regions were in line with the overall results, with the exception of Belgium, as shown in The final questionnaire included items to assess the adaptability of MSProDiscuss and the integration of the tool into routine clinical practice. The responses were very positive. Overall, 91.6% (251/274) of the HCPs thought that adaptability and integration are feasible, 87.2% (239/274) thought that adaptability and integration would be easy, and 85.8% (235/274) were willing to integrate MSProDiscuss into their clinical practice . FinallyThe findings at the country and regional levels were reflective of the overall results, with HCPs having high levels of agreement on the ease of implementation of the tool . An averThroughout the development of MSProDiscuss, qualitative feedback was gathered, and improvements were implemented iteratively. A meaningful impact of this approach may be seen in the overall positive experience expressed by the HCPs while using MSProDiscuss. Of the 301 HCPs who returned the final questionnaire, 162 made at least one recommendation on improvements, such as including additional variables or expanding the existing variables. real-time evaluation of early signs of MS progression in routine clinical practice [The highly variable disease course in individual patients and the lack of consensus on the diagnostic criteria often result in considerable delays in SPMS diagnosis ,14. In 2practice . Tools bpractice -19. MSPrThe results of this survey indicate that MSProDiscuss is a useful tool to aid the discussion of disease progression with patients. The findings were consistent between the individual and final questionnaires. Most of the HCPs agreed or strongly agreed that MSProDiscuss was beneficial in their practice. The time taken to complete the tool during routine consultation was considered satisfactory (1-4 minutes). In most instances, the questions in MSProDiscuss were found to be comprehensible by the patients and were similar to those asked by an HCP during a regular consultation. Thus, MSProDiscuss facilitates patient-physician discussion by capturing a structured disease history without imposing an additional time burden on the HCP. Furthermore, these positive results indicate that MSProDiscuss is easy to use and universally helpful, regardless of region, professional background, and practice setting.More than 90.48% (6310/6974) of instances the HCPs indicated that they would use MSProDiscuss again with the same patients, including both patients with RRMS  and patients with SPMS . In particular, the HCPs indicated that they would use the tool again with more than 97% (5209/5370) of the patients with RRMS and with 94.01% (1508/1604) of the patients with SPMS. This indicates a potential role of MSProDiscuss as a complementary disease-monitoring tool for longitudinal follow-up to be added to current empirical measures such as clinical relapses or magnetic resonance imaging lesion counts. Qualitative insight into the symptoms and their impact on patients as captured by MSProDiscuss will help identify early signs of disease progression. In patients with RRMS, the tool might complement clinical assessments and help in treatment decision-making during the earliest stages of SPMS. For patients already diagnosed with SPMS, the tool will be helpful in identifying functional domains that are most affected by progression and in choosing the appropriate modalities for symptom management.In this survey, 80.3% 220/274) or more of the HCPs indicated that MSProDiscuss is a useful aid in discussing MS symptoms and their impact on daily activities and cognitive function, as well as in discussing progression in general. The fact that 91.9% (252/274) of the HCPs would recommend MSProDiscuss to their colleagues and think that it is feasible to integrate MSProDiscuss into their clinical practice is very encouraging because it suggests that the tool is beneficial to patients as well as HCPs themselves. Key recommendations have already been implemented, including longitudinal follow-up, an enhanced user interface, improved sensitivity at lower EDSS scores, correction for overlapping fatigue symptoms, and creation of a patient-completed version. The suggestion to expand cognitive assessments is a key point; cognitive impairment is an important yet underrecognized sign of disease progression /274 or m. How empAlong with the traffic light output, considering the patient’s responses to the individual questions provides additional insights, supporting the relevance of MSProDiscuss for holistic disease management in individual patients. MSProDiscuss is suitable for longitudinal follow-up and has already been included in large observational studies  and inteA patient version of the tool, the Your MS questionnaire, was developed to cover patient-derived information on MS symptoms also contained in MSProDiscuss . The YouOverall, the results were consistent at the individual country and region levels, with some minor variations reflective of either individual HCP preference of disease management or general practice guidelines followed in the country. The overall positive feedback from most of the countries and regions suggests that MSProDiscuss is already perceived as a valuable tool. In some countries such as Belgium where the feedback was less encouraging in terms of integration into clinical practice, a root cause analysis could not be performed because of the limitations of the study design. It can be speculated that, with limited treatment options in some geographies, it is likely that there is caution regarding discussing disease progression when no solution can be proposed. In other cases, there may be reluctance to discuss disease progression because of the potential emotional and psychological implications for the patient. It is likely that these factors could have influenced the respondents in our survey. However, this only re-emphasizes the need for clear and transparent discussions regarding the importance of identifying early signs of disease progression, allowing treatments and holistic modalities to be started sooner.Gathering feedback on the usability and usefulness of the tool while simultaneously using the tool might have resulted in a potential bias in the HCPs’ responses to certain components of the survey, such as integration into practice and time taken for completion. In general, the questionnaire methodology only highlights trends or attitudes and does not explain the underlying reasons for the responses .Overall, the positive findings from our usability and usefulness study of MSProDiscuss are very promising . AlthougThe findings from this real-world study suggest that MSProDiscuss is a usable and useful tool to facilitate a physician-patient discussion on disease progression in daily clinical practice. MSProDiscuss facilitates the dialog between the patient and the physician by capturing a structured disease history. Most of the survey participants indicated that MSProDiscuss was beneficial in the discussion of disease progression. Overall, the feedback from the HCPs was very positive regarding the integration of MSProDiscuss into their clinical practice. The tool was used by physicians, MS nurses and nurse practitioners, and pharmacists from very different practice settings and was found to be of value; MSProDiscuss is a tool that is acceptable to all the users involved in the care and management of patients with MS.