The use of machine learning techniques for the construction of predictive models of disease outcomes (based on omics and other types of molecular data) has gained enormous relevance in the last few years in the biomedical field. Nonetheless, the virtuosity of omics studies and machine learning tools are subject to the proper application of algorithms as well as the appropriate pre-processing and management of input omics and molecular data. Currently, many of the available approaches that use machine learning on omics data for predictive purposes make mistakes in several of the following key steps: experimental design, feature selection, data pre-processing, and algorithm selection. For this reason, we propose the current work as a guideline on how to confront the main challenges inherent to multi-omics human data. As such, a series of best practices and recommendations are also presented for each of the steps defined. In particular, the main particularities of each omics data layer, the most suitable preprocessing approaches for each source, and a compilation of best practices and tips for the study of disease development prediction using machine learning are described. Using examples of real data, we show how to address the key problems mentioned in multi-omics research . Finally, we define the proposals for model improvement based on the results found, which serve as the bases for future work. In recent years, the biomedical field has experienced a big data revolution. Since the appearance of the first microarray technologies, the competencies of generating data and extracting useful knowledge have increased exponentially. In fact, now we can perform a whole range of molecular analyses on a genome-wide scale, generally referred to as omics analyses. Omics technological advances have led to breakthroughs in our fundamental understanding of cell biology: from our ability to identify alterations in the DNA sequence via a genome-wide association study (GWAS), to the study of gene expression levels by means of RNAseq experiments, or the possibility of studying environmentally inducible chemical DNA modifications with an epigenome-wide association study (EWAS). Similarly, omics studies are generating very positive insights into improving our knowledge of clinical treatments and managing multi-factorial and complex diseases. One of the most promising clinical applications of omics technologies has been the generation of predictive biomarker panels for personalized estimations of disease risk and the consequent implementation of stratified clinical guidelines. In this regard, omics technologies have taken further advantage of the recent advances in the machine learning (ML) field. ML is a research branch of artificial intelligence that has experienced a notable boost due to its ability to automatically generate predictive and descriptive models from massive amounts of data. Within the context of predictive modeling, increasingly sophisticated ML algorithms have become available; highlighting ensemble modeling or the recent revolution of deep learning . We can Most of the challenges encountered are related to the implementation of accurate and reliable analytic pipelines, which are aggravated by the shortage of suitably trained professionals to perform such complex data analysis tasks. This is mainly due to the complex nature of omics data, with huge variations across platforms, different needs for pre-processing steps, intense heterogeneity within and between human subjects, and the ubiquitous problem of high-dimensionality and low sample size settings. Predictive modeling is severely affected by high dimensionality due to what is known as the curse of the dimensionality problem. Some advanced pre-processing methods for feature selection, such as ridge regression, lasso, and elastic-net, have been used to improve the performances of ML models. Research shows that these techniques, despite their great potential, are not recommended for use with low data sample sizes because they cause overfitting ,8. In thThe selection of the most suitable pre-processing pipeline for each omics layer and the choice of the most appropriate ML model are critical steps that must take place considering the particularities of human datasets and depending on the purpose of each predictive modeling tool. This problem increases if we take into account the need to create interpretable models. To address this need, the recent explainable artificial intelligence (XAI) revolution has emerged, which recommends the use of transparent models that are easily understood by human users or the use of post hoc mechanisms that provide comprehensibility to models that are not understandable by scientists, which is especially relevant for medical applications . In the The PUBMEP  project is a longitudinal research study in which children with and without obesity are followed from pre-puberty to puberty evaluating the prevalence of metabolic syndrome and the progression of related cardiometabolic risk factors. In this population, a series of multi-omics analyses were conducted with the aim of discovering new and promising blood molecular biomarkers of IR during the metabolically critical period of puberty see  11]..11].IR is one of the metabolic alterations derived from obesity that appears the earliest in patients. If not properly addressed, IR finally results in the development of more severe diseases, such as cardiovascular disease or type II diabetes. For this reason, IR has become a cornerstone in preventing obesity-associated morbimortality. In the PUBMEP study, 90 Spanish children  were allocated into two experimental groups according to their IR status (IR or non-IR) after the onset of puberty see . The numlsewhere .Genomic data were generated by the sequencing of blood samples using a bead chip called Infinium Global Screening-24-v3.0. This technology allowed us to measure 651,563 single nucleotide polymorphisms (SNPs) with a small percentage of missing values. Genotype imputation was performed using the Minimac 4 method from the haplotype reference consortium (HRC) reference (GRCh37/hg19 genomic annotation) panel using the Michigan imputation server . On thisAF<0.01) .The underlying bases of the inheritance of diseases were not always the same ; this has direct effects on the way we represent data and construct predictive ML models. In the case of obesity, as mentioned in the previous section, we are dealing with a complex trait with a strong polygenic and additive nature (the accumulation of many low-risk effects of SNPs is what constitutes a high-risk profile). Considering this, GWAS data were encoded according to the additive model in this paper. For this reason, we propose using a dosage format (raw) to perform the classification task. A dosage format indicates the presence or absence of a risk or reference allele in a SNP encoded with 0, 1, or 2. One advantage of its use is that it allows the use of numerical genetic variables, making it suitable for the algorithm’s learning process ,15.https://www.ebi.ac.uk/gwas/, https://www.pgscatalog.org/, [Regarding feature selection prior to ML application, we selected a subset of 151 SNPs from the entire array according to previous evidence in the literature. In particular, we collected SNPs highlighted in the meta-analysis, since they are considered to be the studies with the highest degrees of evidence, guaranteeing high statistical power to detect the small effects that each SNP could exert on the phenotype. For this purpose, we performed a literature and database search  and og.org/, ,19,20. normalization, which affects only biased type II probes, using the wateRmelon R package. The selection of this normalization method was based on the fact that all samples under study were obtained from the same tissue (blood) [EWAS data were generated using the Infinium MethylationEPIC 850K from blood samples. To remove any source of technical variability, low-performing probes were filtered out according to different criteria: probes with a detection  (blood) . Regardlquations ; as wellquations . A detailsewhere . Lastly,In this case, the feature-selection procedure consisted of the application of an agnostic selection, a type of feature selection in which differentially methylated CpG sites associated with IR were extracted genome-wide (hypothesis-free). This procedure was conducted in an independent population study that had the same origins as our study population, with some samples overlapping. The study population, which facilitated the agnostic selection, was part of a study of 139 children (76 girls), including longitudinal and cross-sectional approaches, and followed the same experimental design. More details about the selection of these CpG sites can be found in reference . The choThe last dataset is referenced as the biochemistry dataset; it involves the combination of data of diverse origins as mentioned previously. This dataset consists of 48 input variables related to the pubertal IR problem. The main problem with these data involved the presence of missing values. The structures of missing data in our cohort were checked (missing completely at random (MCAR), missing at random, missing not at random, and structurally missing). Then, 14 biochemical variables with more than 5 missing values were discarded in order to avoid introducing excessive noise into the data via imputation. We revised several imputation methods, such as mean/median imputation, knn imputation, bagged trees, Multiple imputations by chained equations (MICE) , and misAfter completing individual pre-processing procedures, three different datasets  were obtained. Each dataset had 1 response variable with 2 distinct classes (IR and non-IR) out of 90 children. The input feature numbers (per dataset) were 151, 267, and 34 for the GWAS, EWAS, and biochemistry data, respectively. A summary of the main characteristics of each dataset considered in this study, as well as the number of variables fulfilling quality filters can be seen in Although a promising approach would have involved the simultaneous modeling of several omics layers together with biochemistry data, merging so much information into a single model would also increase the problem of high dimensionality. Moreover, the different nature of each dataset makes it essential to take a first look at the models constructed separately, in order to understand the amount of valuable information available in each source. In this paper, as a preliminary approach, we propose generating independent ML predictive models for each layer of data, leaving multi-omics modeling as a pending task for future work. Our approach allowed us to extract predictive information from the different biological layers and validate the most important variables for the IR problem while avoiding overfitting .One of the most important practices in the ML field is to train the algorithms on a set of individuals differing from the set aimed to evaluate the model performance. If it is not possible to access an independent population, then the training and test sets must be selected iteratively from the same population through a process known as cross-validation (CV). There are several types of CV: leave one out (LOOCV), Montecarlo CV, Bootstrap, k-fold CV, and repeated k-fold CV. Generally, the default k-fold CV is preferred because it presents the average estimations with the least possible errors. Choosing the right validation methodology (according to the characteristics of the data) is the key to preventing erroneous conclusions from the models .Another important factor is that the learning process should be as homogeneous as possible in each iteration. That is, the distribution of the variables and the proportion of classes should be the same in the training and test sets for each iteration of the CV process. In this paper, a stratified 5-fold CV, repeated 5 times, was used to evaluate the model performance, adding up to a total of 25 executions. Research shows that this approach is one of the best CV procedures to reduce the variability of average classification metrics in low sample size designs. Although other CV methodologies such as LOOCV have also been commonly used in the context of low sample sizes, we continue to recommend the use of repeated k-fold cross-validation for similar studies where the sample size is low, as this methodology has the lowest estimation error, offers a good bias-variance ratio, and is a computationally affordable procedure ,28.As can be seen in Another point of debate when constructing a predictive model is the choice of the ML algorithm and the metrics to be used, which will be strongly conditioned by the objective to be pursued. For example, looking for a model with high predictive ability, neural networks, support vector machines, random forests, or boosted trees might be valuable options. However, if a model is to be used in clinical practice, clinicians must understand how the algorithm makes decisions due to the ethical issues underlying decision-making that may have an impact on the patients’ lives. In such cases, we may opt for more interpretable models, such as decision trees or other rule-based methods, avoiding the so-called black box models, whose predictions lack understandable explanations of their underlying internal mechanisms . The neeIn this study, we chose the well-known OneR , CART 3, and XGBThere are several metrics of interest when evaluating the performance of a model. Metrics that consider classes separately, such as sensitivity or specificity, provide valuable information about that class but should be complemented with other measures to obtain a complete picture of the model’s behavior. Some performance indicators combine several metrics in an effort to achieve a more comprehensive approach, such as G-mean—the geometric mean of sensitivity and specificity—which provides an assessment of how well-balanced the sensitivity and specificity values are. Balancing the accuracy between the majority and minority classes is the key to avoiding underfitting the minority class while overfitting the majority class. Of course, the metrics used to evaluate the model are strongly conditioned by the problem being addressed, e.g., in cases where correctly predicting the positive class is critical, we should focus more on sensitivity than specificity, or make use of metrics that especially penalize failures in the prediction of the positive class, such as G-mean and F1 .All trained models are included in the caret  package One of the drawbacks to using complex but powerful methods, such as XGBoost, lies in the fact that its behavior is difficult for humans to understand. SHAP helps to understand the mechanisms behind its decisions, using the concept of the Shapely value, which goes back to game theory and can help to understand every single decision made by the model by assigning an attribution to each variable, the sum of which equals the model output for that specific instance. We can think of our model as a game in which every variable plays a role to obtain the model’s output, which can be computed as the sum of the individual attribution (with sign and magnitude) of each variable.We also use SHAP to compute the overall importance of each variable considering the whole dataset, so that we can obtain a global idea of the role in the insulin resistance mechanism. Equation  shows hoIn this study, we illustrated how to face the main challenges related to ML prediction in the multi-omics analysis, using a case study on childhood obesity. Our multi-omics dataset is composed of data from GWAS, EWAS, and biochemistry. From the GWAS, we initially had 651,563 SNPs that were matched to HRC, subjected to the initial quality control, imputed in the Michigan Imputation Server, and subjected to a final quality control analysis. The remaining 5,894,726 SNPs fulfilled the eligibility criteria. Then, a feature selection of SNPs was performed before the ML application in order to select a subset of the total 151 SNPs, of which there was prior knowledge about their association with obesity. Consequently, the EWAS dataset was also subjected to quality control, resulting in the elimination of 56,478 low-performing probes and the remaining 834,371 CpG sites. Next, the feature-selection procedure was applied based on an agnostic selection and we selected a final number of 267 CpG sites.When we addressed the class imbalance, the quality of all classifiers improved, as shown in Given these results, our next step is to uncover the mechanisms behind the XGBoost model trained with the biochemistry dataset by undersampling the majority class. We used SHAP to uncover the mechanisms behind classifier predictions using the XGBoost algorithm, which performed slightly better in the distinct omics layer showing robust behavior. From the three possible models, we chose the one that showed the best average result, taking into account the accuracies in both classes . Next, we will move from a general vision of the dataset to a more specific one. Human research faces a range of difficulties , which directly result in studies with relatively low sample sizes. This issue is evident in the context of omics data, where there are millions of variables measured that massively increase the rate of false-positive discoveries . In the context of ML predictive modeling, low sample sizes and a huge search space have direct effects on the performances of the models built with omics data, leading to increases in computational burden and overfitting. For these reasons, it is essential to perform feature-selection steps prior to model training. There are several ways to perform feature selection, and choosing which features to use depends on the characteristics of the data and the research problems to be evaluated. Another common challenge in human research is the high presence of unbalanced designs in which one class is over-represented in relation to the other. This often occurs in a setting where the disease under study is not frequent and the recruitment of patients is complicated. As is often the case with low sample sizes and high-dimensionality problems, the class imbalance directly affects the ML predictive models by inducing overfitting. Multiple solutions have been proposed to face this problem  depending on the characteristics of the samples under consideration. In our case study, we demonstrated that the undersampling solution is one of the most recommended for most human contexts since it avoids introducing additional noise into the data. Another issue of importance when dealing with human data is the strong variability that exists between subjects. To address this, the development of accurate and valid experimental designs that minimize sources of bias  is of utmost importance. Moreover, the validation of findings in an external population to ensure the reliability of predictive models is essential. To this end, in cases where it is not possible to recruit additional patients, several iterative validation solutions based on cross-validation methodologies are available .Several particularities should be highlighted when focusing on the characteristics of omics data. This is due to the need to apply different preprocessing procedures for each molecular layer, which is inherent to each platform. In all omics analyses, there is some background noise or an unwanted source of variability that is associated with technical laboratory procedures. This heterogeneity is, therefore, not related to the biological issue being studied and must be removed from the analysis. Background noise due to technical procedures usually differs not only between different types of omics but also between the different technological platforms normally employed for the analysis of the omics .GWAS refers to any observational study of a genome-wide set of genetic variants or SNPs in different individuals to see if any variant is associated with a trait. GWASs are evaluated using microarrays and are subject to several problems: erroneous genotype call assignments due to poor quality DNA samples, poor DNA hybridization to the array, poorly performing genotype probes, and sample mix-ups or contamination. Although currently available GWAS platforms map many SNPs , there are still many unmeasured variants of interest for disease prediction that could be imputed using appropriate procedures . In ordeIn EWASs, the DNA methylation (DNAm) status across the whole genome is interrogated at the CpG level. For each molecule of DNA in a single cell, DNAm is a binary entity, in that at any cytosine it is either present or absent. However, as DNAm studies profile either bulk tissues—comprising multiple cell types—or a population of purified cells, DNAm measurements for CpGs are always reported as continuous values representing the proportion of methylated CpGs for the DNA position. Methylation levels are commonly reported as either quations  and 2);;β valuesquations . M-valuelsewhere .As with GWAS, EWAS data are also subject to many sources of unwanted variability, some of them are derived from the microarray nature of the analytical platforms, detection errors, the existence of cross-reactive probes, the need for special treatment of probes located on sex chromosomes, or the need to normalize raw fluorescence intensity signal data to address within-subject and between-subject variabilities. Regarding normalization processes, although there is no single method that is universally considered best, the functional normalization method is the most appropriate for datasets with overall differences in methylation between different tissue types , and theThe fact that EWASs analyze a mixture of cells in a tissue is an important issue as, in some cases, tissues are infiltrated by other cell types or are so heterogeneous that they might confound the findings. In blood sample types, which are most commonly analyzed in EWAS, there is an important part of variability that comes from the proportions of white cells present in each individual. Therefore, and especially when dealing with diseases with inflammatory components, as with obesity, it is extremely important to correct the findings for the proportion of white cell types presented by each subject, as it might affect the DNAm findings and, thereby, confound the effects of DNAm on the disease. This is usually resolved in EWAS using the Houseman procedure, which deduces the proportion of white cell types in each subject. Then, the estimated proportions can be included as confounding variables in the models .Beyond the aforementioned technical sources of variabilities associated with each technology, there are also other particularities that affect data pre-processing and that are of particular importance when one wants to predict an outcome. With regard to GWAS data, it is a fact that certain diseases present strong polygenetic architectures; many genes are involved in the development and progression of the disease. Associated genes often have small individual effects on the phenotype, so the accumulation of many small-effect variants constitutes a susceptibility profile. In EWASs, environmental confounders can strongly affect epigenetic patterns. For this reason, it is well-known that the findings from one study population cannot be easily extrapolated to another population .Models trained using the imbalanced datasets show better sensitivity at the expense of very poor specificity, while datasets balanced during the training stage provide more consistent values for both metrics and greater generalizability to unseen data of any kind.When the training dataset is balanced, the biochemistry dataset provides the best results in terms of F1, G-mean, accuracy, sensitivity, and specificity, followed by EWAS and GWAS; this leads us to conclude that combining biochemistry and EWAS datasets may be a promising strategy to improve these results. As In our study, the above considerations were extrapolated to a practical scenario aimed at analyzing the predictive ability of ML models in the development of insulin resistance in children. From the ML prediction models generated with the OneR, CART, and XGBoost algorithms, the following two conclusions can be drawn:When studying GWAS data, and looking at the accuracy, one might think that these models classify well, but this is only true for the majority class. This is relevant because the objective of our case study was the exact opposite—to predict the minority class correctly. Despite using undersampling methods to avoid overfitting in this case, it can be observed that the classifiers constructed on GWAS data were no better than randomly assigning individuals to one class or another (area under the Roc curve or AUC ≈ 0.65). In relation to the results obtained, GWAS data did not contain useful information patterns for predictive tasks. Among other reasons, this could be attributed to the complex genetic architecture of obesity traits and the additive effects of SNPs on disease risk since there are thousands of SNPs, with small risk effects on the phenotype, which constitute a high susceptibility profile. Finally, looking at The biochemical dataset, therefore, provided the patterns with the most useful predictive information, achieving the best values in most of the metrics. The results obtained for the biochemical dataset are not surprising since it included phenotypic, anthropometric, and clinical variables with direct implications in the development of obesity , which are currently used in daily clinical practices to estimate the risk of metabolic syndrome in children with obesity. Moreover, through a detailed study of attributes of the biochemistry-balanced ML model, we were able to demonstrate that those with the highest IR predictive abilities  were strongly related to the presence of obesity. In fact, the regulation of leptin levels is one of the key factors of the disease, given its implications in the development of obesity-associated comorbidities, such as non-alcoholic fatty liver disease (NAFLD) ,42.In future work, other ways of encoding omics data, such as genetic, methylation, or metabolic risk scores (polygenetic risk score) should be explored. In particular, the way we select the input SNPs, and how we pass that information to the ML model are very important in genetics. As we previously mentioned, obesity and other complex traits involve a complex polygenetic architecture and it has been demonstrated that directly using individual SNPs is not the best tool for predictive purposes. Otherwise, risk scores  are powerful tools to account for the complex structures of omics data and how to best predict long-term outcomes. With risk scores, we gather information for thousands of SNPs (or variables), thus reducing the problem of dimensionality while modeling the complex structures of omics. Similarly, performing appropriate feature selections on omics data that have small sample sizes is an unsolved task in the omics ML field. For this reason, some multivariate methods could be tested by checking their promising abilities in order to deal with omics data to reduce their high dimensionality. Another crucial issue to consider according to our results is the integration of multi-omics data with biochemical and clinical data in a single model. Despite this, such combination procedures are not always as straightforward as combining all data into the same model .In this paper, we illustrated how to face the main challenges encountered when constructing ML predictive models with multi-omics human data. The main topics covered in this paper were as follows: a description of the main particularities of the omics data layers, the most appropriate pre-processing approaches for each source, and a collection of the best practices and tips for applying ML to these kinds of data for predictive purposes. By exemplifying the generation of predictive models using real data, we showed some of the key issues that need to be addressed in this kind of research . This paper presents a collection of the best practices and guidelines that could be extrapolated to other human diseases with complex bases . We lay the groundwork for future work by incorporating some proposals to improve the models, advocating that they are necessary according to the insights encountered.


Background: Exponential increases in the development of medical knowledge, the expansion of areas where medicine develops its activities, the emergence of new pathologies , novel diagnostic methods and therapeutic strategies, together with the appearance of multiple communication and information technologies, determined that the education of future physicians required targeted training in scientific methodology.Methods: The design and execution of a course in scientific methodology in thecurriculum of Facultad de Medicina, Universidad de la República, Uruguay, is described. The course is carried out at an advanced stage of the medical studies for all the students, in which they develop a 10-month research project supervised by the medical school faculty. Students undergo all stages of a research endeavor: generation of hypothesis, elaboration of a research protocol, submission to the Research Ethics and Animal Welfare Committees, data recollection, analysis, interpretation and publication of the results.Results: The course is undertaken at the Facultad de Medicina, Universidad de la República, Uruguay, the main university of the country, with high numbers of students enrolled. The course involves the participation of 600 students and up to 300 professorsper year, which implies a huge institutional effortConclusions: The scientific methodology course resulted in one of the most important incorporations of the current 2008curriculum. Local students, faculty and international evaluators have qualified this activity as an educational breakthrough, being a gratifying and productive experience. The course represented the first exposure of medical students to the research methodology, scientific literature and publication rules, and emphasized the dynamic nature of medical knowledge within modern medical education. Moreover, for some students it constituted the onset of academic research careers. An additional positive outcome was the reactivation of some faculty research projects, in a way that largely exceeded the boundaries of the course. The exponential increase of information in molecular medicine, genetic engineering and biotechnology, and the continuous evolution of the paradigms of evidence-based and personalized medicine, fosters a focused training of medical students in scientific aspects of the medical sciences. Therefore, innovations in the medicalcurriculum require the provision of methodological tools and to stimulate a “mindset” for future physicians with the ultimate goal of improving medical practice. The organization of these innovations needs to be adapted, among other factors, to the specific institutional capacities and the number of students.There is a permanent demand in medical schools to provide a strong background on scientific methodology and research to the future physicians. The number of active students (those who have presented activity in the last two years),increased from 81,774 in 2008 to 139,830 students in 2019. In 2019, 18,549 students were incorporated toUniversidad de la República, of which 2,225 started the medical career at Facultad de Medicina. Every year, close to 500 medical students graduate . Thus, in this context, a specific course of “Scientific Methodology in the Medical Sciences” for advanced medical students was conceived and executed, taking into consideration the described conceptual framework and also the numerosity of students. Now, an initial assessment of the organization, evolution and impact of the course over the last eight years will be provided.Universidad de la República in Uruguay, being public and the largest university of the country, has seen a significant increase in students in the last decade, overall representing around 80% of university students in the whole country, in which he discussed the need to incorporate the basic sciences and a strong scientific component to the training of physicians, believing that this would result in better performance during their clinical practice. From the report arises the need to incorporate training of the scientific method into the medical career, through the formulation of problems, generation of hypothesis and the development of a series of well-designed studies to reject or confirm the hypothesis, with the idea that the skills for problem solving can be applied directly to patient care. One hundred years after the publication of the Flexner Report, several authors analyzed how it resulted in a transformative document that generated the foundations for the teaching of 20th century Medicine. At the end of the 90s, the need to generate a change in the study plans of medical careers was raised in the United States, as established in the report generated by the Council on Graduate Medical Education (COGME). In this context there is a current emphasis, in prestigious schools of medicine at the international level, on explicit scientific training in study plans. Therefore, educational strategies are in full development and are undergoing experimentation, varying in the different academic centers where they are applied.In 1910, Abraham Flexner published an extensive report analyzing the situation of 155 medical schools in the United States and Canadacurriculum provides students with fundamental skills needed for academic achievement, and can help them to establish academic careers. Another recent work published by Waaijeret al., evaluated the scientific activity of medical students, and its effect on scientific activity after graduation. The authors demonstrated that the students who published during their career were more likely to continue publishing after graduation, being more scientifically productive. There is a general consensus that training medical doctors in the 21st century requires the incorporation of different new skills, including biomedical informatics, information and communication technologies and scientific methodology in addition to clinical skills, in order to cope with exponential increase in medical knowledge.Although a causal relationship cannot be attributed, recent data suggest that successful early participation in research can influence the long-term scientific activities of clinicians. For instance, Huynh and co-workers described the incorporation of a surgical research program for medical students, and demonstrated that integrating research early in the medical schoolcurriculum for medical students had only a minor component of research methodology focusing on (basic principles of) biostatistics at the beginning of the career and an approximation of research design in the first year of clinical training. It is important to note that Uruguayan medicalcurriculum resembles that of European universities and therefore differs from the United States. Indeed, right after high school the medical students initiate a seven-year program   .the Asamblea del Claustroof Facultad de Medicina already referred specifically to the required scientific quality of medical professionals which must "maintain a critical attitude, based on good scientific training and practice that allows them to analyze, understand and contribute to the resolution of problems related to health in the field in which they operate”, it was not until eight years ago that this requirement could be met with specific course content and adequate strategies to effectively enable good scientific training during undergraduate studies. This incorporation was part of the accreditation process at the regional levelunder the MEXA systemwith the generation of a final report in March 2012 which allowedthe certification of the medical career in November 2012 by ARCU-SUR. One important issue mentioned in the report was the need of advancing scientific training of the medical students, indicating the necessity for improvement. It is with that objective that within the currentcurriculum of the Medical Doctor degree two courses, Scientific Methodology I and II (SMI and SMII), were established. Although this paper will focus on the SMII course, it is important to point out that the SMI course sets theoretical basis for the second course . A series of scientific conferences at the Facultad de Medicina were also formalized since 2013, with local and international speakers and the aim of communicating current biomedical research topics.In addition to the Scientific Methodology courses, a series of activities were incorporated into the 2008curriculum for the Facultad de Medicina was designed andapproved by the Council in 2008 and started to be progressively applied in March 2009  for the first-time presents theoretical elements related to the design, methodological and ethical aspects of research, medical literature searches and biostatistics  a two-day poster presentation in the Annual Meeting of the course, with the participation of the students and professors of Facultad de Medicina , with several projects in translational medicine.The project finishes with the generation of: a) a monographic work by each group, with a predefined format of an original paper or review article,edicina . Topics Anales de la Facultad de Medicina. The evaluation of all the projects is first performed using a guideline to standardize criteria, followed by a meeting of the whole committee to perform an extensive analysis of the different projects and select the best ones. The publication process allows the students to become familiar with the editorial process and represents an important aspect for the diffusion and promotion of some of the research performed in the different departments of the Facultad de Medicina.The completed research project carried out by the students is evaluated by an external evaluating committee that selects a set of works to be published in the official journal of Facultad de Medicina,per year) and a vivid collegial exchange between students and professors.The SMII course Annual Meeting allows the presentation of the results of all the participating groups  there was one professor allocated to each group, while now there is an average of three professorsper group, in many cases from different departments  which helps to foster interdisciplinarity.The Scientific Methodology II course implies the participation of a large number of faculty members either in their role of project directors or evaluators. Since the course was established in 2014, the number of faculty and departments involved has steadily increased, with the important participation of most of the basic, epidemiological and clinical services. In 2021, over 300 professors were involved in the direction of research projects, and around 60 departments in all basic and clinical areas, from a total of 80, participated 18. It.An important issue to mention is that initially there was the significant participation of professors from the basic areas, who had considerable expertise in research activities . This changed significantly over the years and actually there is a significant involvement of the general and specialized areas which is an important goal of these courses  18.If a closer analysis of the distribution of areas of Facultad de Medicina that are participating since 2014 is done it can be observed that, since 2015, general and specialized clinical departments increased the number of mentors and projects, notably for the surgical areas, which had no participation at all in 2014, and are now participating in several groups . Indeed,Each year a diversity of topics from all departments are presented which allow students to work in different areas of their interest, since they will choose their topics at the beginning of the year, and usually are related to their future specialization.Anales de la Facultad de Medicina were published, containing the selected publications of each year. Most of the monographs published each year are included at theColibrí Database fromUniversidad de la República, a repository which contains much of the graduate and doctoral thesis and projects from Uruguay.Since the SMII course was settled, special issues of the journal. In some cases, professors from schools other than Medicine were involved in co-direction of the projects, further promoting an interdisciplinary approach. Many of the selected topics have been closely related each year to the social and health situation of the country, and original knowledge has been created in topics which are relevant to our society, such as the current situation and prevalence of different diseases, clinical evaluation of the use of drugs, incorporation of new diagnosis tools or novel therapies, perception of the quality of health care in public hospitals, environmental conditions, mental health and medical education learning conditions, among others. Several of these research projects are the first step for more in-depth investigations, and this is an important contribution of the SMII course to the research of Facultad de Medicina  pandemic imposed an additional challenge on all the faculty staff and on the students themselves. Despite the fact that the health situation in Uruguay was for the most part of the evolution of the pandemic significantly better than other countries of the region and the worldLast year, at the end of the 2020 course and due to the pandemic, the Annual Meeting was restructured, and the presentation of posters could not be done as usual because the university was closed at that time of the year. Instead, the 100 groups presented their work orally through a virtual platform, constituting an unprecedented and unique experience in Facultad de Medicina, which will be incorporated in future editions of the course.GACH) that worked during 2020 and until July 2021 advising the national government in making decisions based on the best available scientific evidence. This group was assembled with recognized scientists and physicians, many of whom are faculty of the School of Medicine, who at the same time worked in SMII either in the mentoring of the groups or in the evaluation of the projects. For our medical students, this was a compelling and “live” example of how appropriate scientific training can directly impact in the management and the resolution of health problems in our society.In the pandemic context, it is noteworthy to note the creation of an honorary scientific advisory group , generating interdisciplinary projects around the university.The described Scientific Methodology course (SMII) has successfully evolved after eight years of execution, resulting in one of the most important incorporations of the current 2008In addition, Uruguayan scientific and medical societies have included presentations from the SMII course students in their meetings, and many research projects have resulted in original publications both in national and international journals.In spite of the great number of students of each generation, an innovative and successful program was instated in our institution. The course impacted positively on the scientific background of advanced medical students, and renovated the research activities in many clinical Departments.et al. at New South Wales University, Australia, however, the number of students involved is much lower than the one of Facultad de Medicina, Universidad de la República. They reported the implementation of an independent learning project (ILP), to promote research skills within the medical students in the whole cohort of the last year of the career where students performed a 34-week research project. Similar to what is reported in this paper, they conducted a long evaluation process (14 years), concluding that students gained valuable experience in research methodology.Recently, a similar experience was published by Uebeli.e., 600 students). This condition represents an additional challenge since the proposal intends to allow the students to undergo all the stages of a research project in a short period of time (10 months).Though similar experiences are incorporated in different medical schools around the world, it is remarkable that Facultad de Medicina has a quite large number of students at this stage of their medical studies  is now being conducted. These data will allow to provide further objective elements on the influence that incorporation of formal scientific training in thehttps://doi.org/10.5281/zenodo.6625343Zenodo: Scientific Methodology Course II-Facultad de Medicina, Universidad de la República, Uruguay (2014–2022).This project contains the following underlying data:Scientific Methodology Course-Facultad de Medicina-Universidad de la República-2014–2022.csv SMC-Research Topics-2014.csv (Research projects year 2014)SMC-Research Topics-2015.csv (Research projects year 2015)SMC-Research Topics-2016.csv (Research projects year 2016)SMC-Research Topics-2017.csv (Research projects year 2017)SMC-Research Topics-2018.csv (Research projects year 2018)SMC-Research Topics-2019.csv (Research projects year 2019)SMC-Research Topics-2020.csv (Research projects year 2020)SMC-Research Topics-2021.csv (Research projects year 2021)Creative Commons Attribution 4.0 International license (CC-BY 4.0).Data are available under the terms of the I have read, and enjoyed, the paper “A scientific methodology course for advanced medical students: an eight-year perspective”. First at all, let me apologize for my poor English. Anyway, the paper is easy to read. At least, to me. This paper describes nicely an interesting learning experience, without major statements that would require formal statistical inference. I do not have major concerns. Only minors, that the authors might address or not. Please note I call those minors, as ‘comments’, not suggestions. So, I think it deserves indexing. Minor comments. In the introduction you stated “… fosters a focused training of medical students in scientific aspects of the medical sciences…”. I’m not sure I agree with this “general” statement. Please, clarify what are you proposing. . In the next paragraph you refer to “advanced” students. Please clarify: how do you define ‘advanced’ in operational terms?  Furthermore, what proportion of publications have any value to improve clinical practice?] Lastly, you stated “early participation in research can influence the long-term scientific activities of clinicians”. It seems to me that your objective is to train practice medicine physicians in order to select the best ones to do research. In my opinion this is a spurious objective.  You use “fashion” terms such as personalized and translational Medicine. I wonder if they are needed to your report. [Personally, I think each intervention is already “personalized” with the definition of the eligibility criteria. Please, consider starting your results with a demographical description of your subjects, both students and trainers. Please, consider reporting frequencies including their numerator and denominator. For example, “X out of n (28%) departments from basic medicine” instead of “28% from basic medicine”. I also wonder if the statistical principles used in pharmacology (such as a constant effect) apply to surgery, because a different effect can be assumed to each interventionist - see the consort extension to non-pharmacological interventions. So, although I like the implication of surgeons and other non-pharmacological experts in this project, I wonder if they have to learn the same statistical methods. You start the conclusions section stating the “success” of your training. I think you can be proud, but I have been trained to strictly define the rules of a “success” before starting. Please, consider describing in the previous results section the indicators that would allow you this statement - specifying you have done this once you have viewed the results and your paper is not free of the risk of bias for selective reporting. Other comments I liked your proposal in footnote ‘a’ to call “Facultad de Medicina” your school. Please review you use this term throughout the text, figures and footnotes.Is the rationale for developing the new method (or application) clearly explained?YesIs the description of the method technically sound?YesAre the conclusions about the method and its performance adequately supported by the findings presented in the article?YesIf any results are presented, are all the source data underlying the results available to ensure full reproducibility?YesAre sufficient details provided to allow replication of the method development and its use by others?YesReviewer Expertise:Statistical methods, clinical trials, reporting guidelinesI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. This manuscript provides a detailed case study of how an a large, well-established medical school can design, implement and evaluate a scientific methodology course. Overall, the authors were able to demonstrate the success of the program through a eight year retrospective analysis. The authors provide an extensive background that not only includes of narrative of their local progression towards the development of the course, but also an international perspective as well. The authors also provide sufficient detail on the development of the course and its initial implementation. Through the data analysis, they are able to demonstrate the growth of the program and the success through successful dissemination, including publication, of student projects. Overall, the manuscript is well written and addresses an important topic in medical education.Is the rationale for developing the new method (or application) clearly explained?YesIs the description of the method technically sound?YesAre the conclusions about the method and its performance adequately supported by the findings presented in the article?YesIf any results are presented, are all the source data underlying the results available to ensure full reproducibility?YesAre sufficient details provided to allow replication of the method development and its use by others?YesReviewer Expertise:Medical Education Research, Program Evaluation, AccreditationI confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. I thank the insightful and detailed comments of Dr. Jason Booza and I am glad for the positive appreciation and approval of our manuscript. On behalf of my coauthors, Prof. Dr. Rafael Radi


This paper undertakes a literature review of psychological, Educational Learning Strategies, and Models during the COVID-19 Pandemic. It examines data from 359 publications relating to this subject, published on the Web of Science, Scopus, and ScienceDirect between 2020 and 2021 using bibliometric analysis adapted with VOSviewer software. The review discusses the following approaches , research work, countries, and research institutions). It concluded that bibliometric analysis is fundamental for detailing the theoretical literature and developing an integrated theoretical framework for psychological and Educational Learning Strategies. The psychological impact on students and potential stress needs to be closely monitored and evaluated, to plan effective policies while adopting these pedagogical approaches. Due to the significant movement to distance education that has occurred in response to the Coronavirus Disease 2019 (COVID-19) pandemic. As almost all teaching methods have rapidly moved to distance education, it is now imperative that consistent distance education approaches be developed using psychological and pedagogical learning methodologies e.g., .If, as Astley suggests, the wide variety of research projects is due to the many subfields of study that make up the educational sciences , the proSome databases organize research . As management sciences and researchers prepare increasing numbers of distinguished studies, there is a need to explore how to manage this increase in literature.Computer software helps manage data, enabling it to be organized, stored, published, and distributed. Through such resources, we are able to process increasing numbers of previous studies. For example, software such as Citespac and VOSviewer help us know the most influential researchers globally. Therefore, the field should focus on references, keywords, research cases, and organizations.This study undertook bibliometric analysis, examining and comparing previous methods , with a focus on psychological, Educational Learning Strategies during COVID-19. This paper analyzed the production of psychological educational learning strategies as indexed in Web of Science (WOS) and Scopus (2020–2021). According to RQ. What does bibliometric analysis contribute to the review and development of the theoretical literature on psychological and Educational Learning Strategies?The sub-questions are:RQ1. What role does bibliometric analysis play in establishing the PELS theoretical frameworks?RQ2. What structure is produced by the publications and citations that constitute PELS quality?RQ3. Which keywords do PELS authors use most frequently?RQ4. Who are the most frequently mentioned writers in the area of PELS?RQ5. What are the most frequently cited research publications in the area of PELS?RQ6. What are the most significant research institutes for producing research papers in PELS?RQ7. Which countries contribute the most to the development of research publications in PELS? (Q5 1 R2).This research will enable us to discover research trends in this field and help familiarize readers with the topic and become more knowledgeable about the development of Psychological and Educational Learning Strategies in the scientific community. Likewise, the justification and significance of the analysis in this study are based on seven research questions that guided the work. The major objective is to identify the source titles, institutions, authors, and countries with the largest scientific output on higher education, as well as to identify higher education trends in the scientific literature.The purpose of the study is to determine the results and process of bibliometric analysis, which will aid the researcher in educational sciences in finding distinctions for research, including identifying keywords, the most influential researchers in the field, the research work, reference sources, countries, and reference research institutions. Consequently, the research compares bibliometric analysis to conventional literature evaluations in the educational sciences, considering of bibliometric analysis and research techniques for bibliometric studies in the educational sciences. In addition to a bibliographic examination of the psychological problems faced by students during the COVID-19 pandemic, this study includes a discussion of its significance.The deductive hypothesis methodology  was adopted, which begins by examining the initial assumptions (hypotheses) and trying to support or test them according to what theories for bibliometric analysis have been developed previously concerning psychological and Educational Learning Strategies.Bibliometric analysis is a method of science mapping that analyzes data from previous research and studies, considering how articles, authors, areas and resources are related to one another , defining the critical areas of research, keywords, authors, countries, and organizations in a specific field of knowledge  such as via a topic search of the psychological and Educational Learning strategies and models used during the COVID-19 Pandemic using the WoS database on January 5, 2020. The following search terms were used: topic = , in title-abs-key from 2020 to 2021. There were 359 studies distributed over 2 years.To carry out the study, data were taken from three approved databases, namely Web of Science (ISI), Scopus, and Science Direct. We retrieved published research The 359 studies retrieved included articles, conferences, and book chapters, as shown in In total, we retrieved 290 studies on the topic of learning strategies and models in COVID-19 from the Scopus database, as shown in To prepare the bibliometric analysis, the number of publications and citations were used to find the relationship between keywords, based on outputs of two software KnowledgeMatrix Plus In terms of network and density, The next figure shows the results of the bibliometric analysis of keywords using the Scopus database.As This review used bibliometric network analysis to provide an overview of researchers worldwide’ studies on psychological and educational learning strategies and models used all over the world during the COVID-19 pandemic. The sample included 359 articles indexed in Web of Science (ISI) data, 290 articles indexed by Scopus Data, and 297 articles indexed by Science Direct data. It provides insights into the dissemination of knowledge among researchers in this field. According to the main review findings, several topics were highlighted and may be further reviewed in the future.The impact of the COVID-19 pandemic has been profound. It was necessary to radically change the style of teaching and learning depending on the current conditions imposed by the pandemic, including the tools and strategies used by institutions of higher education and the experiences of faculty members. This transition has had a psychological and educational impact on students, which has led to students’ anxiety and stress see .There are also many important messages for the post-coronavirus period. Among them, faculty members and universities were not exposed to many required teaching methods. Also, many of them had to go through a very steep learning curve to be able to deliver basic lessons. Many studies also covered the educational impact on students and the strategies used by higher education institutions in all countries of the world. The literature on the psychological impact and the strategies used is limited to a few countries, as discussed in a study by COVID-19 has had a profound impact on students, and its impact on student’s mental health and well-being cannot be fully understood. Many of the current psychological assessments in this area are short-term reviews. Perhaps well-studied longitudinal studies can shed light on this in the future.Colleges and universities have also had to deal with a lot of change during this time. However, the psychological impact on students and potential stress needs to be closely monitored and evaluated, to formulate effective policies while adopting these pedagogical approaches.The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.The author confirms being the sole contributor of this work and has approved it for publication.The author extends their appreciation to the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia for funding this research (under project number UB-36-1442).The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


World Professional Association for Transgender Health guidelines support the importance of a mental health assessment before providing medical treatment for Gender Dysphoria (GD). During this assessment, patients without GD but with mental health disorder and who request treatment for GD should be excluded. Dissociative Identity Disorder (DID) is a poorly known mental disorder which can be confused for GD.To provide a case report of a patient suffering for DID but asking for treatment for GD and to provide a review of the literature on GD and DID.A case report assessment and follow-up was described and a systematic review of the literature was performed in Pubmed, PsychInfo, and Embase databases.To provide all cases with assessment and follow-up of DID and GD.The case report describes a man suffering from DID and asking for hormonal treatment for GD. After assessment the patient was able to let go of his wish for treatment for GD and begin psychotherapy for DID. During the literature review 11 articles were included. 3 articles showed a prevalence of DID of 0%, 0.8% and 1,5% in GD samples. 5 articles were case reports of patients with both diagnoses of GD and DID which showed the complexity of the care of these patients. 2 articles were case reports, where a GD diagnosis was done first, and then counseling for GD was proposed. After a second session, the diagnosis was changed for DID. In 1 other case report and our case report there was a description of 2 persons suffering from DID and asking for treatment for GD.Our review shows the complexity of providing care to patients with a comorbidity of GD and DID, as well as the complexity of making the differential diagnosis between GD and DID.A systematic review was performed on these rare cases. Our study presents the results for a small group of patients.This article provides the first systematic review on GD and DID and shows that DID in a GD sample does not seem to be higher than in the general population. In addition, it allow clinicians to gain better knowledge about patients suffering from both DID and GD and patients suffering from DID who ask for GD treatment.Soldati L, Hasler R, Recordon N, et al. Gender Dysphoria and Dissociative Identity Disorder: A Case Report and Review of Literature. Sex Med 2022;10:100553. The DSM 5In the 11th edition of ICD, GD is classified in the category “Conditions related to sexual health” and renamed «Gender incongruence (GI)».Both assigned male at birth (AMAB) and assigned female at birth (AFAB) individuals suffering from GD may benefit from different treatments. There are hormonal therapies (HT) and gender affirming surgeries (GAS).Professionals involved in the care of persons with GD largely follow the standards of care developed by the World Professional Association for Transgender Health (WPATH) which are based on a somatic and a mental health assessment (MHA) before the initiation of a medical treatment for GD.It is not specifically mentioned in the current guidelines which stress the importance of MHA in GD treatmentPeople suffering from DID describe different alter personalities (AP), which have different mannerisms and moods, often different handwriting, food preferences, or preferences in clothes and also different gender identities, which may differ from the assigned gender at birth, as seen in GD.Despite the fact that clinicians often think that DID is a rare syndrome, a prevalence rate of DID of 1%–3% of the population has been described.We will next describe a case report of a person suffering from DID and asking for treatment for GD. We will then provide a review of the literature on GD and DID, and give some advice concerning the MHA and care of patients suffering from DID and asking for GD treatment.In the present article, we have decided to use a patient's assigned gender at birth when discussing case reports about patients with DID and without GD.The assessment and follow-up of a patient suffering for DID but asking for treatment for GD was described. The patient provided written informed consent.This review was carried out according to the guidelines of the Preferred Reporting items for systematic reviews and meta-analyses (PRISMA).Duplicates were eliminated.Two independent reviewers screened the abstracts, selected them using inclusion criteria, and analyzed the findings.Studies were included if they reported data on gender nonconforming identities and dissociative disorders together.We included all original studies regardless of their methodology, size of the research group, and the presence of the control group.Studies were excluded if there were no data on gender both nonconforming identities and DID, and if they were reviews, thesis and letters.Owing to the considerable heterogeneity of the studies and outcome measures, it was not possible to provide Forest plots. We used a narrative approach to synthesize the data from the studies included in this review.A 27-year-old person, AMAB, was referred by his psychiatrist to our sexology unit for an evaluation of his GD. The patient questioned his gender and wondered whether he should begin a transition including a feminisation of the body. The patient was already in care by his general practitioner for a somatoform disorder.During the first appointment of the MHA, the patient described a gender incongruence in relation to a stable female gender identity that he had felt since his adolescence. This made him feel the need, at times, to live in a female social role and in a female body. For this reason, he had been growing his hair for many years, and sometimes wore female clothes and behaved in a way to socially express his female gender in general attitudes and sexual behaviours. When his female gender identity was preponderant he would think about taking feminizing HT in order to have a body more in adequacy with his female gender. This wasn't a preoccupation when his female gender identity was not preponderant.The patient's explanations in the first appointment sounded like GD in a person with gender nonconforming identity. During the other appointments of the MHA, however, after asking for more precision, the patient described the different personalities he had within himself. He described 8 distinct personalities with different gender identities . Each of these personalities had different characters. The patient also described numerous memory lapses sometimes linked to traumatic events, and was experiencing difficulty to fit in socially and professionally.The patient had been adopted during childhood and had a leukemia as a child with long hospitalizations and treatments. Five years before the first appointment in our sexology unit, he had begun to suffer from a somatoform disorder, with tremors, pains and even paralysis of the legs for 6 months.Only the female personalities adhered to his request for HT for GD.After collecting all this information, the diagnosis of DID was first suspected and then confirmed through a clinical interview using DSM 5 criteria, the DESAlthough the patient described some criteria for GD , it appears here that the DID diagnosis takes precedence. The patient accepted extra sessions to discuss the possibilities of HT for his gender incongruence, but, as the sessions progressed, the different personalities understood that the wish for HT was actually a wish coming from his DID which needed psychotherapy. The patient was able to let go of his wish for HT. The patient is currently in psychotherapy for his DID.90 duplicates (38 in Psycinfo database and 52 in Embase database) were eliminated.2 articles were added to results through hand search.We found 459 results  using a systematic search in the databases.After selection using inclusion criteria, 32 abstracts were considered for full-text screen.After selection using exclusion criteria, 11 articles were included in the systematic review as shown in Three articles were within studies in GD population showing a prevalence of DID of respectively 0%Five articlesMun's case reportIn Schwartz's,Two older articles,Two further articlesBecker's case reportModestin's case reportIn Steinberg's book,First and foremost, our review shows that the literature in this field is very limited.Regarding the prevalence of DID in GD samples, 3 studies show a prevalence of 0%,In the 5 case reports which are concerned with the comorbidity between GD and DID,,In addition, the difficulty in making the differential diagnosis between GD and DID is evident in 4 case reports. In 2 of them, mistakes were made in establishing the diagnosisSteinberg's case report,Regarding the difficulties in making the differential diagnosis between GD and DID, some authors have stressed that people with DID may frequently experience bewilderment or confusion in their gender identity, sexual orientation or more specifically sex characteristic of their body, because their different personalities have different genders. Particular importance should be attributed to this disorder in considering the differential diagnosis for GD.Some authors suggest that patients with DID describe a global identity disturbance, unlike patients with GD who focus more on the gender incongruence.Furthermore, as stressed by the International Society for the Study of Trauma and Dissociation (ISSTD),Finally, almost all practitioners use standard diagnostic interviews and mental status examinations that do not include questions about dissociation or take a history of psychological trauma. Because DID patients rarely give information about their dissociative symptoms, the absence of a focused inquiry about dissociation prevents the clinician from diagnosing the disorder.Concerning the appropriateness of offering HT or GAS to patients with both diagnoses of GD and DID, some authors stress that it is highly questionable that these patients should receive GAS until the DID has been treated, and stress the importance, in the sessions prior to GAS, of providing psychotherapy for the dissociative symptoms.,,This review of the literature leads the authors to believe that the comorbidity of GD and DID cannot be considered an absolute contraindication for HT or GAS, since Mum's case report showed a benefit from HT and some case reportsRegarding HT, Mun's case reportThis article provides the first systematic review on GD and DID and shows the importance of alerting mental health professionals to the risk of confusing DID with GD and the possibility that these conditions may coexist.In addition, our review shows the complexity of following patients with a comorbidity of GD and DID, as well as the complexity of making the differential diagnosis between GD and DID.Mental health professionals working with GD patients should include DID in their differential diagnosis, and, if there is any evidence of memory lapses or dissociation, the use of diagnostic tests such as the DES,In patients seeking care for GD who have both GD and DID, it is important to offer a psychological follow-up in order to understand the experiences of the different AP. In this way, the patient may engage in a dialogue with their different AP in order to find the compromise that best suits the different AP before deciding on a treatment for GD.It is important to stress that misdiagnosis of DID for GD and counseling for GD in a patient suffering from DID may have had psychological iatrogenic effects, as shown in Becker's case report.If the diagnosis of DID is confirmed, the therapy of choice is psychotherapy according to the “Guidelines for treating dissociative identity disorder in adult” of the ISSTD.There are several limitations to the possible interpretations stemming from the present literature review. The limitations are due to the small number of articles and the fact that the majority are case reports. In addition, the assessment is often only clinical, without validated assessment instruments, which creates an important potential for bias.More studies are warranted to better understand the prevalence of coexistence of GD and DID, and the impact of GD counseling, HT and GAS on coexisting DID and, finally, the iatrogenic consequences of treating a patient mistakenly diagnosed with GD instead of DID.The authors of this article refer to the version 7 of the standards of care (SOC) of the WPATHConceptualization: Lorenzo Soldati; Methodology: Lorenzo Soldati; Data curation: Lorenzo Soldati, Roland Hasler, Nathalie Recordon, Marie Clément; Formal analysis: Lorenzo Soldati, Roland Hasler, Nathalie Recordon, Marie Clément; Writing – Original Draft: Lorenzo Soldati; Writing – Review & Editing: Lorenzo Soldati, John Kohl, Nader Perroud.


A critical aspect of designing and running online study programs is the identification of factors and elements that could potentially threaten the continuation of studies. In this study, we first identified a set of critical events that occurred in the running of a Finnish online doctoral study program over 16 years. Next, we analyzed the events using a four-pillar sustainability model, which consisted of the economic, social, environmental, and ethical pillars. We detected several contextually relevant and dynamic pivotal factors related to each of the pillars, which had effects on the sustainability of the program at the time of the critical events. The analysis revealed that positive pivotal factors in one sustainability pillar can be used to compensate for negative pivotal factors in the other pillars. Two aspects that were crucial for the sustainability of the online doctoral study program were the resilience and shared commitment of the community involved in its activities, which helped in overcoming any challenges encountered. Based on this study, we recommended that future research should design novel solutions that help online study programs to proactively identify potential critical events and related pivotal factors. Furthermore, studies should find creative approaches for constructively coping with critical events that have been identified. A critical aspect of ensuring a sustainable design for an online study program is to prepare for potential challenges, problems, and threats to the continuation of studies. In our experience, online learning initiatives start to diminish and eventually fade away when, for example, external funding ends, changes in the leadership of the host institution occur, key people cease to be involved in core activities, or the initiative fails to renew or no longer fits the strategy of the host institution. Research has identified that quality assurance lists, critical success factors, risk analysis tools, and good practices support the sustainable design of online study programs , such as high dropout rates  might reduce or limit sustainability to rigid design rules, strict regulations, or quality handbooks.In our previous study, we introduced a four-pillar model for analyzing the sustainability of online study programs. The model consists of four sustainability pillars: economic, social, environmental, and ethical , whereas a highly demanding or even threatening work community can endanger sustainability . A constructive tension between the pillars supports sustainability, whereas conflicts between them could scar and damage even a successful online learning initiative. Thus, it is crucial to find a mutually constructive interplay between the four pillars, since relying too heavily on a single pillar or even two could be potentially problematic for the sustainability of the program, as illustrated in Fig. However, two aspects affect the importance and relevance of the pillars. The first is the scale of a study program, which can vary from a set of individual courses to certificate programs and full degree programs. Completely online universities already exist, as do consortia of online universities. The scale of the online program is visualized by the size of the roof in Fig. We proposed two constructs for analyzing the sustainability of online study programs using the four-pillar model: CEs and pivotal factors. Mertova & Webster  defined IMPDET  is an international online doctoral study program designed and hosted in Finland. The program has been running since 2004, and it was originally designed to provide doctoral studies to applicants from emerging and transitional countries. In the IMPDET studies, doctoral degrees can be completed either in computer science or education. The decisions regarding the discipline and eventually the faculty where the doctoral studies are pursued are primarily made based on the applicant’s previous academic degree. The application procedures, study requirements, research cultures, and supervision arrangements vary between the two faculties, which means that the study experiences of the doctoral students also differ depending on the faculty in which the studies are pursued. However, some students have supervisors from both disciplines, which supports the multidisciplinary nature of their research work.All doctoral degrees in Finland are research-based, which differs for instance from the USA, where PhD and Ed.D degrees clearly serve different purposes  and an article-based dissertation. At the end of their studies, the student writes a monograph but refers to the scientific publications in the dissertation’s main body of text instead of including the publications as part of the dissertation. In the program, the main reasons for choosing a monograph dissertation are that the scientific publications related to the dissertation do not form a single coherent research study, there is no clear connection between the individual publications, or the dissertation itself includes an independent scientific contribution supported by the publications.The pedagogical components of the program are designed to mentor the students through the process of becoming a doctoral-level scholar . The publishing requirement is aimed at ensuring the quality of the doctoral students’ research work, since they can ultimately use the publications as part of their dissertation.The doctoral students in the program can freely use any suitable research approaches and data collection methods for their dissertation study. However, in recent years, action research  credits during their studies. At the host university, 1 ECTS credit equals approximately 26 h of study or required work time. At the beginning of the students’ studies, they together with their main supervisor create an individual postgraduate study plan based on the requirements of the faculty in which they have enrolled.Three different course work components can be included in the postgraduate study plan. First, the students can study online, mixed-method, and campus-based courses offered by the host university or organized by another university. The topics covered by the courses offered specifically to the students on the study program have included the following: (1) contemporary technologies in education, (2) research methods in computing, (3) the design of smart learning environments, (4) learning analytics, and (5) ICT for development. Furthermore, the graduate school of the host university offers online, mixed-model, and campus-based courses on transferable skill studies, such as scientific writing, scientific presentation, research project management, research ethics, and English for doctoral studies. The courses are available to all doctoral students at the host university, so they are not specifically targeted at IMPDET students. The only obligatory course for all doctoral students in the program is the research ethics course. The students can also complete relevant courses  outside of the host university, which provides additional flexibility for the studies. The students’ main supervisor or the coordinator of the study program accepts the inclusion of the courses outside of the host university.The second course work component is individual learning tasks, which provide flexibility and additional opportunities to complete the credit points beside the formal courses. The individual learning tasks are agreed with the main supervisor or the program coordinator. The tasks typically include the following types of activities: (1) orientation at the beginning of the studies; (2) different types of essays or other written assignments; (3) a literature review related to the students’ research topic; (4) project work activities, such as the implementation of a prototype or design tasks; and (5) presentations in scientific conferences. Furthermore, the main supervisor or the coordinator of the program can accept basically any activity or task if it is relevant for the students’ research work or supports their doctoral studies in a meaningful way.The third course work component is research seminars and workshops. The study program runs monthly online research seminars and students can gain credit points by participating in them and giving presentations. In addition, the program offers irregular intensive scientific workshops, such as those related to Academy of Finland- or EU-funded projects of the research groups involved in the program activities. Figure The completion time for the postgraduate course work and even the course plan varies between the students; therefore, there are no exact instructions for the students regarding when to complete their course work. The only binding requirement is that at the end of the studies the completed course work should fulfill the minimum requirements set by the students’ faculty. It is also highly common for the original individual study plan accepted at the beginning of the studies to be updated at the end of the studies, since some originally planned courses might no longer be available or new opportunities to complete the course work may arise.At the end of 2021, a total of 44 doctoral students representing 20 different nationalities were enrolled in the program. The first students were admitted in 2004, and most of the doctoral students (over 94% of those admitted) have studied computer science, which is because the program has mainly been run by an educational technology research group of the computing department at the host university. While the program was originally designed and implemented in close collaboration with educational scientists at the host university, this collaboration has diminished over the years.The first doctoral degree of the program was awarded in 2007, and the first doctoral student to have conducted most of his studies through distance learning graduated in 2008. A total of 32 doctoral students graduated from the program between 2004 and 2021. The average completion time for a doctoral degree is 5 years and 6 months, and the median completion time is 4 years and 11 months. The standard deviation is 2 years and 4 months, which indicates high variation among completion times. There are students who have completed their studies within 4 years , but several other students have required 9–10 years to complete their studies.Figure The persistence rate of the program has been relatively high compared with, for example, the high attrition rate in a related program in Sweden , both the financial resources and human capital for running the studies decreased considerably. Although the study program survived these difficult times, major changes to the practical arrangements were required, including the reduction of online courses and supervision arrangements. For several years, the program remained active and relatively stable in terms of student numbers, but the expansion of interest in the program led to challenges since the number of staff members and local supervisors did not increase. However, collaboration with the Tanzanian higher education institution provided additional financial resources and created opportunities to renew the studies. Another crucial aspect was outside supervisors who were committed to supervising the doctoral students without any financial compensation.CE5 and CE6 were less threatening to the sustainability of the study program compared with earlier CEs, but several aspects challenged the running of the studies. For example, due to a shortage of staff members, the admission of new doctoral candidates was suspended for long periods of time. The increase in the total number of students was moderate due to these suspension periods, but acceptance criteria were also tightened. On the other hand, for example, the maturity of learning technologies has allowed the full online implementation of the studies, which was crucial during the COVID-19 pandemic. Collaboration with two African universities has created new opportunities to supervise students and organize local support for those students who have been studying via the doctoral studies hubs.Based on our analyses, we developed the following guidelines for the design and implementation of online doctoral study programs. From an economic perspective, the most interesting observation was that a good economic situation can lead to the insufficient use of resources. On the other hand, declining funding can force designers to focus on the core aspects of the studies, which can lead to improvement. The social and ethical pillars were especially crucial when the funding decreased. Despite the economic and environmental challenges, the faculty members at the host university were resilient and decided to continue running the studies.The lessons learned from the social and ethical perspectives were the ability to redistribute responsibilities, mutual trust among faculty members, and personal ownership toward the studies, as identified previously by Russel . HoweverFrom the environmental and ethical perspectives, the role of external supervisors has been critical throughout the studies. Without their voluntary commitment, it would have been impossible to keep the student numbers as high as they have been, especially during 2016–2020. There have been two main incentives for the external supervisors: (1) an ethical motivation or internal obligation to find a doctoral training opportunity for candidates who require flexible study possibilities without study fees; and (2) the co-authored papers and eventually supervised doctoral dissertations are crucial scientific outputs that could potentially support the careers of voluntary supervisors. If a study program needs to find a voluntary work force, such as when economic resources are scarce, it is crucial to consider incentives that can motivate those volunteers. For some individuals, ethical aspects and internal motivation without financial compensation might be sufficient, but in the long run other incentives will be required.In this study, we analyzed the CEs of a Finnish online doctoral study program using the four-pillar sustainability model. We demonstrated that the model can be used to identify positive (constructive) and negative (destructive) pivotal factors related to the sustainability of online study programs from the four aspects of the economy, sociology, environment, and ethics. We claim that the proposed model can be used as a tool for the design, quality assurance, and improvement mechanisms of online study programs, since it can be used to foresee and detect potential strong and weak points. The model can also be used for identifying sustainability aspects of novel projects and initiatives; for example, it can be applied for the analysis of scalability or used for rationalizing activities.This work was based on analyzing the CEs retrospectively. The question for future research is as follows: How can weak signs of possible fragile structures and strong points in online study programs be identified? We can also view CEs as risks that have materialized for better or worse. Thus, the four-pillar model equipped with CE and pivotal factor analysis can be used for risk management. For example, the proposed model can be used to create stress tests for an online program in the form of simulations or scenarios. These stress tests could also be supported by a formal risk analysis conducted to identify potential future CEs (Vesper et al.,