Since 2012, the citizen science project ‘Mückenatlas’ has been supplementing the German mosquito monitoring programme with over 28,000 submissions of physical insect samples. As the factors triggering people to catch mosquitoes for science are still unknown, we analysed the influence of mass media reports on mosquito submission numbers. Based on a theoretical framework of how mass media affect citizen responsiveness, we identified five possible influencing factors related to citizen science: (i) project awareness and knowledge, (ii) attention (economy), (iii) individual characteristics of citizen scientists and targeted communication, (iv) spatial differences and varying affectedness, and (v) media landscape. Hypotheses based on these influencing factors were quantitatively and qualitatively tested with two datasets: clipping data of mass media reports  referring to or focussing on the ‘Mückenatlas’, and corresponding data of ‘Mückenatlas’ submissions between 2014 and 2017. In general, the number of media reports positively affected the number of mosquito submissions on a temporal and spatial scale, i.e. many media reports provoke many mosquito submissions. We found that an already heightened public and media awareness of mosquito-relevant topics combined with a direct call-to-action in a media report title led to a maximum participation. Differences on federal state level, however, suggest that factors additional to quantitative media coverage trigger participation in the ‘Mückenatlas’, in particular the mosquito affectedness of the resident population. Lastly, media types appear to differ in their effects on the number of submissions. Our results show under which circumstances the media presence of the ’Mückenatlas’ is most effective in activating people to submit mosquito samples, and thus provide advice for designing communication strategies for citizen science projects. With continuing outbreaks of mosquito-borne diseases in Mediterranean countries  as well www.mueckenatlas.com) or available at the project’s administrative office  and the Friedrich-Loeffler-Institut (FLI), with up to four people working on the sample identification, reference collection, participant support and database maintenance  and their potential health risk through the transmission of pathogens, mosquitoes are highly appropriate subjects for citizen science. Indeed, mosquito-related projects are finished, running or emerging across the globe –27. Yet,While knowledge about mosquito phenology and distribution in Germany has vastly increased due to the citizens’ involvement , 36, theWith regard to the latter, social and traditional media play a key role for both initial and sustaining participation in citizen science , but onlDeutsche Presseagentur  as a content provider that triggers print and online media reports. Hecker et al. [Active contacts to the media have been established since the beginning of the ‘Mückenatlas’, especially with the r et al.  already Communication via traditional mass media is a frequently used approach to attract participants and draw attention to citizen science projects , 44. HowFor our analysis, we had to reduce this complexity and focus on the most important variables and their relationships in terms of convincing citizen scientists to participate in our project. Media effects research mainly examines the relationships between input variables  and output variables . In this context, the communication matrix model of persuasion developed by McGuire (1989)  has becoBased on this persuasion matrix model and a literature review of research on mass media effects, we identified relevant factors for the responsiveness/resonance to mass communication events in citizen science projects: (i) project awareness and knowledge, (ii) attention (economy), (iii) individual characteristics of citizen scientists and targeted communication, (iv) spatial differences and affectedness and (v) media landscape. ‘Project awareness and knowledge’ and ‘attention (economy)’ represent the first two steps of persuasion in McGuire’s matrix model. They are thus preconditions for the response rate of citizen scientists. While they constitute output variables of the communication process, the other three categories include input variables related to the individual recipients’ situation , the message design (‘targeted communication’) and the source of messages (‘media landscape’).Project awareness and knowledge: Mass media raise awareness, communicate knowledge and can play a mediating role between citizen engagement and attention to socially relevant issues (e.g. ). Thus, Attention (economy): The next step of persuasion in McGuire’s matrix model is ‘attention’. The capacities of media recipients to receive and process information are limited  and massIndividual characteristics of citizen scientists and targeted communication: There is a number of personal characteristics of citizens that affect the motivation to participate –56. WhilSpatial differences and varying affectedness: Building on the considerations about individual characteristics of citizen scientists, we assume regional differences in the response rate to media coverage . On the Media landscape: With regard to the media landscape, there are different types of media  with different reaches  or target groups . They therefore filter information about the project and put a message into different contexts, which influences the efficacy of a message in different ways , 61. We The aim of this study is to gain insights on the connection between media coverage and the activation of citizens to participate in citizen science projects. We draw on aggregations of a media monitoring service between 2014 and 2017 to test our hypotheses built on the five influencing factors stated above. Based on real data, we explored how effective the ‘Mückenatlas’ mass media approach turned out and thus provide insights for the targeted design of communication strategies for other citizen science projects.The study is exploratory and based on data that was not originally collected for the purposes of this study, but to check the effectiveness of the public relations strategy in the context of corporate communications. This is why the dataset has some deficiencies, such as missing distribution data, limited information about the reach or impact area of publishers or outdated links to online media reports. In addition, the media type ‘print’ is only present in 2017, while the media type ‘online’ is lacking for this year. Nevertheless, the data present a rare opportunity to look for initial clues about the relationship between media coverage and participation.The raw data of media reports originated from Argus Data Insights, a media monitoring service that provides information on the media presence of companies, also called media clipping. The company was assigned from 2014 until the end of 2017 by the ZALF to find and consolidate media content for monthly reporting. The service’s engine searched the media types ‘print’, ‘radio’, ‘television’, ‘online’ and ‘news agencies’ (dpa) on a national level by using the keywords ‘Mückenatlas’, alone or in combination with ‘Leibniz-Zentrum für Agrarlandschaftsforschung’. Due to the exit of Argus Data Insights from the online business in 2017, the composition of ‘television’, ‘radio’, ‘print’ and ‘online’ varies greatly from year to year, so any temporal evaluation of media types would be meaningless.The media clipping dataset contained 1072 observations of media reports from 2014 to 2017, with 9 automatically generated covariates, such as date, title and media type , ‘xvalue’ and ‘yvalue’ for geo-references and ‘reach’. For the ‘municipality’ variable, media reports that originated from a certain municipality and targeted a regional audience were assigned the exact municipality name. In all other cases, the media reports were categorised as ‘federal’ or ‘national’ depending on their reach. For those media reports where a specific municipality could be specified, the corresponding geo-coordinates of the community centre were recorded in ‘xvalue’ and ‘yvalue’. For the variable ‘reach’, entries with a geo-reference were assigned to ‘regional’. Media reports that targeted an audience of one or more federal states were categorised as ‘single federal’ or ‘cross federal’, respectively. For entries with national reach, the categories for ‘reach’ were set to ‘national’  independently of specimen counts per sample site, hereafter referred to as ‘submission’. The resulting dataset comprised 16,610 submissions.Data of mosquito submissions to the ‘Mückenatlas’ were extracted from CULBASE on July 31To investigate our hypothesis of a positive correlation between the number of media reports and the number of submissions, we first examined the temporal relationship. The monthly frequencies of both datasets were plotted across all years and tested for a possible time lag between the maxima of media reports and submission numbers using Spearman cross-correlation analysis.To examine a possible spatial association between number of media reports and number of submissions, we applied a spatial pattern analysis with two spatial point datasets. The first dataset consisted of the geo-locations of ‘Mückenatlas’ submissions. The second dataset contained those media reports for which we were able to verify a municipality and to which a geo-location could thus be assigned. Consequently, media reports with federal or national reach were excluded from this analysis.To visually inspect the geographical distribution of media reports and submissions across Germany, we produced density plots, which suggested spatial clustering for both submissions and media reports. To assess the level of clustering of both point pattern datasets, each was individually tested by comparing the geographical distribution of the points to a theoretical point pattern of complete spatial randomness, using the Kolmogorov-Smirnov test. These initial assessments led us to assume that the two point patterns are similarly clustered throughout the country, indicating that locations of clusters of media reports are in the same geographical locations as clusters of submissions.Kcross in the R package spatstat) was used, where j is the number of submissions within a certain distance of a media report i. A Monte Carlo simulation was applied to statistically test the distinction between the observed Cross K-function and a Cross K-function produced by random labelling, that is, all observations were labelled either as i = ‘media report’ or j = ‘submission’ while keeping the observed proportions (permutations = 100).To investigate this potential relative clustering, the bivariate version of Ripley’s K-function  and free remarks for collection-site description. In addition, an open question is raised on the submission form as to how the participants became aware of the ‘Mückenatlas’, with the examples ‘internet’, ‘radio’, ‘acquaintances’ and ‘television’ provided as possible answers . We seleggplot2 [tidyr [dplyr [scales [zoo [rstatix [tseries [rgdal [raster [spatstat [maptools [viridis [All analyses were carried out with R, version 3.6.3 . Datasetggplot2 , tidyr [2 [tidyr , dplyr [r [dplyr , scales  [scales  and zoo les [zoo . Statist[rstatix  and tser[tseries . For pois [rgdal , raster  [raster , spatstaspatstat  and maptmaptools , while t[viridis .Insect samples were provided voluntarily by citizen scientists. By signing the submission form that includes comprehensive information on data privacy, participants agreed with the processing of their personal data according to EU General Data Protection Regulation. For the storage and analysis of the mosquito data, the personal data of the participants (i.e. the contact information for each response) is separated from the information of the submitted samples, with the exception of the geo-reference of the catch locations, thereby further anonymising the collected data. These anonymised submissions were also used for this study outside the actual research purpose of the ‘Mückenatlas’ in order to protect the personal rights of the citizen scientists.s = 0.677 (p < 0.001) is at a value of h = -1, which means that most submissions occur about one month after a maximum of media coverage  or Hesse (HE), and a small number of media reports can as well result in high submission numbers, as observed in Baden-Wuerttemberg (BW) and Brandenburg (BB) .χ2  = 2.13, p > 0.05) or reach of media reports  = 0.253, p > 0.05)  . DiffereAccording to the media clipping dataset about the ‘Mückenatlas’ from 2014 to 2017, most media reports were published online , followed by radio broadcasts , printed articles  and television programmes . By comparison, the evaluation of the submission forms showed that most participants became aware of the citizen science project ‘Mückenatlas’ online, followed by television, newspaper, radio, magazines, personal communication and other sources  , Fig 5.The ‘Mückenatlas’ is both a science communication and a research tool: information about mosquitoes is distributed to a broad public, and at the same time large amounts of data are collected for research. As participation does not require any skills or training–except for catching a mosquito without smashing it–the project counts on submissions from people who heard about the ‘Mückenatlas’ and rather spontaneously decide to take part. Therefore, we use mass media to reach the largest possible share of the population, because the more new participants submit mosquitoes, the more detailed the map of collections and the better the geographical coverage of Germany becomes. By investigating five influencing factors for the responsiveness to mass media communication events, we estimated how effective the ‘Mückenatlas’ mass media strategy actually is to reach the project’s scientific goal.Our hypothesis of a positive correlation between the number of media reports and the number of submissions was confirmed for both a temporal and spatial scale. Temporal associations between media campaigns and participants’ activation have been attested before, e.g. a social media campaign and press release led to increased download and subsequent usage of an app for water body observation .short-term fluctuation change [The time lag of around one month between the peak in media report and mosquito submissions could be an adaptive form of a typical response to citizens’ exposure of a topic via mass media called n change : the medThe spatial analysis revealed that both point datasets are spatially clustered, with two hotspots of different sizes each in North Rhine-Westphalia (West Germany) and Brandenburg/Berlin (North-East Germany). Although the spatial distributions of the reports and submissions are not exactly the same, they are similar and locations of submissions are closer to the locations of media reports than would be expected by chance. Even if this connection seems logical, it has not yet been demonstrated in the context of citizen science. This finding highlights the potential of using the media to attract more records from underrepresented regions or areas of special interest through targeted communication of large-scale citizen science projects, such as approaching regional press or customising participation campaigns for local communities , 75.In addition to the positive correlation of numbers of media reports and submissions, In general, First, the news  that theSecond, the style of the title of this dpa-release and those of the following media reports for which title information was available, was very distinct to that of all other dpa-releases by a concrete call to the public to submit mosquitoes. In addition, the text of the dpa-release, which was probably copied by many of the following media articles, included a quote by the project leader that related the importance of participation directly to the Zika virus. Chu et al.  reportedThis triad of existing media attention towards mosquitoes, a direct appeal to citizens and relevance to health may have led to the maximum of submissions in June 2016. Continued strong coverage likely sustained public attention throughout 2016 and 2017, resulting in still high numbers of mosquito submissions. We assume that, additionally to a wider information reach also through regional media , the pubAs assumed, regional differences in the response rate to media coverage could be identified. In accordance of the findings on the spatial correlation of the numbers of submissions with media reports, they also correlate positively on the federal state level. Despite this general correlation, the results are somewhat contradictory when having a closer look . For exaAedes albopictus) and Asian bush (Aedes japonicus) mosquitoes were detected in Baden-Wuerttemberg already in 2007 and 2008, respectively [Ae. japonicus) was only recorded in 2015 [Over a length of 420 km, the river Rhine borders Baden-Wuerttemberg with a multitude of riparian and floodplain areas and oxbow lakes. In addition, there are 155 km of shore area of Lake Constance. Warm and humid conditions and fluctuating water levels lead to regular mass developments of mosquitoes in these regions , which hectively , 82. By  in 2015 .Aedes koreicus has been occurring in Wiesbaden, Hesse, as the only population in Germany [A reduced quality of life for many citizens due to nuisance, a long tradition of countermeasures and prolonged media exposure of the impacts of invasive species may therefore have led to a higher affectedness by mosquitoes in the Baden-Wuerttemberg population. Hence, the willingness to participate in mosquito-related initiatives such as the ‘Mückenatlas’ might already have been on a relatively high level there and less dependent on media coverage. However, since 2017 a population of the invasive species  Germany . Whetherheadquarter effect, meaning that local communities tend to support their local projects, also known as place-based effects [Another interesting pattern was that, similarly to Baden-Wuerttemberg, the federal states of Mecklenburg-Western Pomerania (MV) and Brandenburg (BB) showed high submission frequencies despite low numbers of media reports , although minor (7.5%) and not supported by marketing, still takes place shows parallels to recruitment processes known from the volunteering literature . Face-toThere are some shortcomings to this study, mainly based on the media clipping dataset. First, there is a lack of online data for 2017, as the institute changed the clipping service for online media. We decided against including a separate dataset from the new service, as it is based on different search algorithms and search terms, making comparisons unfeasible. In addition, print media data is missing for 2014 to 2016, which could skew the dataset towards an overrepresentation of regional reach in 2017.The spatial distribution of submissions is biased by human population density . HoweverThe qualitative title and text analysis is based on dpa-releases, as these trigger the majority of media reports with mostly the same or very similar wording. However, there are also a considerable number of media reports that are independent of these dpa-releases. This information was excluded in the qualitative analysis. In order to make a more reliable statement on spatial and temporal effectiveness in connection with text quality, the inclusion and retrieval of all articles would be necessary. Future studies could use experimental approaches with differently designed press or dpa-releases to test our findings and hypotheses.The clear positive correlation between numbers of media reports and numbers of mosquito submissions demonstrate that communication of the project via the mass media as multipliers is important to activate participants. However, the resulting temporal and spatial clustering of the submissions is also a potential source of bias in opportunistically collected data. This has been observed in other studies , 91, butIn times when public attention is a scarce commodity, the title of a media report can be a decisive hook. With a targeted call-to-action in the title, we achieved maximum participation and, rather unintentionally, took advantage of increased media and public attention due to a current human health risk situation caused by mosquitoes. Since other projects have also made this experience , it coulDespite the clear spatial association between media reports and submissions on a national level, differences in regional participation frequency are presumably due to multiple factors. For mosquitoes, we could demonstrate that human affectedness might play an important role. We commend to use the level of affectedness towards the subject of interest in a more targeted way, e.g. appeal to pet owners to collect ticks or to hobby gardeners to monitor invasive environmental weeds.We advise to involve the media and establish reliable relations with press representatives already when planning new or reinvigorate existing citizen science projects . For exaHowever, the media can only be the spark that triggers pre-existing intrinsic and extrinsic motivational factors of the citizens, which need to be investigated in further studies. Knowing more about the demographic background and motivations of the citizen scientists would allow for customised marketing campaigns that might lead to constantly higher numbers of submissions compared to an undifferentiated ‘scattergun approach’ via the mass media .S1 Filewww.mueckenatlas.com and in paper form on request at the project office (German only).This template is available for download at (PDF)Click here for additional data file.S2 File(PDF)Click here for additional data file.


N = 248) were mainly in Life Sciences, Kinesiology, and Physical Education programs. They answered RP questions using TopHat©, an online educational software platform. The results of this study indicated that student performance on the final exam was enhanced when students engaged in RP. It was concluded that the use of RP effectively enhances learning and long-term retention of semantic memory. In addition to the traditional testing ‘of’ learning, teachers are encouraged to implement testing, in the form of RP, in their classrooms ‘for’ learning.It is generally assumed by students that learning takes place during repeated episodes of rereading and rote memorization of course materials. Over the past few decades, however, research has increasingly indicated that the said notion can and should be enhanced with learning paradigms such as retrieval practice (RP). RP occurs when students practice retrieving their consolidated semantic memories by informally testing themselves. This strategy results in the re-encoding and re-consolidation of existing semantic memories, thus strengthening their schemas. The purpose of this quasi-experimental design was to assess the effects of the implementation of RP on student performance on the final exam in a large, undergraduate Gross Anatomy course. It was hypothesized that student participation in RP during class would improve their performance on the final exam in the course. The participants ( Learning of new, semantic  information is traditionally thought by students to typically occur during episodes of rote memorization . HistoriSemantic memory, or that of factual information, can be classified as one of four stages, where the transfer from one stage to the next is completed by one of three processes. The first of these stages is sensory memory (SM) , where tMemory from the WM is transferred to both the hippocampus (HPC) as recent memory (RM) and the association cortex as long-term memory (LTM) simultaneously. This process—the initial registration of information—is known as encoding . EncodinHence, taking advantage of the described neuroanatomical processes in the form of RP is especially important in education. Researchers e.g., –7 over tIt is imperative to point out that the provision of feedback when implementing RP is particularly important , especiaThis study employed a quasi-experimental design in a large, undergraduate Gross Anatomy course that was designed for students who wanted to learn about the human body’s visceral systems. This study intended to address the question, “Does student participation in RP during class improve their performance on the final exam in the course?”This quasi-experimental study examined the effectiveness of RP on student learning, and in turn, their performance, on the final exam in a large, undergraduate Gross Anatomy course at Queen’s University, Kingston, ON, Canada. The course ran throughout the semester for 12 weeks, with three lectures and a 2-h lab in the Anatomy Learning Centre per week. The course discussed the gross anatomical structures in the head, neck, and thorax, including detailed descriptions of the structures’ neurovasculature. The instructor typically employs instructional practices such as the use of the course textbook and the document camera to teach this course, where they highlight important textual information and label figures and images, as well as present anatomical specimens and models obtained from the Anatomy Learning Centre. In past years, when RP was not practiced in this class, the student average was between 65 and 70%. This study was approved by the Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (HSREB) at Queen’s University.N = 248) in this study were full-time students who were enrolled in a large, undergraduate Gross Anatomy course. For most students, this course was a requirement for credits towards their respective degrees, and therefore, most of them regularly attended all lectures. These students were enrolled in one of three programs: third-year Life Sciences (LISC) (N = 60), second-year Kinesiology (KIN) (N = 121), and third-year Physical Education (PHED) (N = 43). All other students, collectively classified as ‘Others’ (N = 24), enrolled in this course as an elective. Participation in this research was completely voluntary and participants had to provide informed consent.All participants . A total of 252 RP questions were posed during the semester and a total of 148 questions were posed on the final exam. RP was used to study student knowledge of the innervation of the visceral systems; thus, of all the questions that were asked on the final exam, only the questions that addressed innervation were congruent with the questions posed during RP. These questions, which amounted to a total of 67 final exam questions, were designated as completely congruent questions (CCQs).‘Congruency’ refers to the degree that a RP question is similar to a question that was posed on the final exam. This does not imply that the same question was asked during both the implementation of RP and on the final exam; rather, two questions were considered congruent if the same knowledge that pertains to one also pertains to the other. For example, “Which of the following nerves innervates the masseter muscle?” was asked as an RP question; a CCQ congruent with this question was “Which cranial nerve, if damaged, causes inability to chew?”All other questions, which did not address innervation and were posed during RP but not on the final exam, were designated as noncongruent questions (NCQs). An example of an NCQ was “Which of the following is not a bone of the face?” Of course, students were not made aware of these categorizations and were expected to study and learn all the information that was taught during the course.The students’ overall levels of participation in RP were obtained from the tabulated statistics on TopHat©. Furthermore, their grades for each of the CCQs were obtained from TopHat© and their final exam answer sheets. All data were tabulated, and data analysis was performed according to demographics. Students were categorized into one of two groups based on the level of their participation. Participation was characterized as the total number of RP questions the students answered during the semester, regardless of the correctness of those answers. The dichotomous cut-off for participation was set at 85% due to a natural gap in the data  and strength of the relationship between the dependent and independent variables.The Statistical Package for the Social Sciences (SPSS Statistics) was used to analyze the data. Statistical significance was set at N = 48) were designated as the low RP group. All participants had participated to some degree throughout the semester. Table A total of 248 students participated in this study, 200 of whom participated ≥ 85% of the time; thus, they were designated as the high RP group. The remainder (SD = 1.6%), while their average grade for all CCQs on the final exam was 75.8% (SD = 3.7%). Similarly, the average grades for the low RP group were 73.5% (SD = 2.1%) and 62.3% (SD = 1.6%), respectively. These results indicated that the high RP group performed better on the CCQs on the final exam, relative to the low RP group.The average grade for all CCQs in RP during class for the high RP group was 91.6% (p < 0.001), LISC (p = 0.031), PHED (p = 0.046), and Others (p < 0.001). These results are shown in Fig. When the data were analyzed according to demographics, it was demonstrated that the high RP groups performed better on their final exams compared to the low RP groups: KIN p < 0.00, LISC (pp > 0.05) was performed to test for normality, which illustrated that the variables were non-normally distributed. Kendall tau-b and Spearman’s rank-order correlations were performed to assess the relationship between student participation in RP and student performance on the final exam. There were moderate, positive correlations between student participation in RP and student performance on the final exam, which were statistically significant, τb = 0.334, p < 0.001, and rs = 0.420, p < 0.001.A Shapiro-Wilk’s test  = 54.62, p < 0.001, accounting for 30.80% of the variation in student performance with an adjusted R2 = 30.30%.Linear regression was run to understand the effects of the dichotomized student participation (set at 85.00%) in RP on student performance on the final exam. It was demonstrated that student participation on the CCQs in RP statistically significantly predicted their performances on the CCQs on the final exam, The purpose of this study was to understand the effectiveness of classroom-based RP on student performance on formal evaluations in a large, undergraduate Gross Anatomy course. RP was implemented throughout the semester. The CCQs during RP specifically addressed the innervation of the visceral systems of the head, neck, and torso.Data analyses demonstrated that the utilization of RP in the classroom was statistically significantly associated with improved student performance on formal evaluations. More particularly, students who were enrolled in PHED as well as those who took the course as an elective benefited the most from the implementation of RP, compared to the LISC and KIN students. In addition, it was demonstrated that the LISC students in both low and high RP groups performed better than the students from other programs. A possible explanation for these findings may be that the students who participated more frequently were more conscientious, and that conscientiousness may be the actual driving factor for increased final exam performance. Another explanation may be that the LISC students, compared to the KIN students, were upper-year students who were generally looking to applying to professional schools—especially medicine. Nonetheless, the implementation of RP was associated with improved student performance on the final exam, irrespective of program. The implications of this research illustrate that implementing RP enhances learning experiences and retention of semantic information in Anatomical Education. Although these findings are especially significant in disciplines such as anatomy, RP can successfully be implemented in any other discipline.Perhaps one of the greatest limitations was that students may have engaged in peer instructional pedagogical techniques when attempting to answer RP questions, despite being repeatedly discouraged from so doing. If a group of students answered RP questions together, it is plausible that only one of them may have successfully retrieved the memory from LTM, while the others may have merely recognized that the answer was correct.In addition, the implementation of a randomized experimental design under strict, laboratory-controlled conditions would have been unethical because students in the control group would have been put at a disadvantage due to the positive effects of RP on the experimental group. This made it difficult to account for several factors, such as previous knowledge and level of interest in anatomy, that may have influenced the results of this study.Lastly, this study did not assess the impact of the NCQs on student performance. To date, no study known to the authors has completed such an examination. If it were to be demonstrated that NCQs during RP exhibited negative implications on student performance on formal evaluations, future research should attempt to reduce these negative effects.This study investigated the implications of the implementation of RP in a large, undergraduate Gross Anatomy course. It was demonstrated that participation in RP was associated with improved student performance on formal evaluations. These findings showed that RP enhances long-term retention of semantic information, thereby improving learning, at least in Anatomical Education. This study has proven that the implementation of RP successfully provides positive implications in classroom settings, where strict experimental methods cannot ethically be used. In totality, teachers are encouraged to motivate their students to implement RP on their own. Furthermore, teachers are encouraged to implement testing, in the form of RP, in their classrooms ‘for’ learning in addition to the traditional testing ‘of’ learning in the form of formal evaluations.


Objectives: This study aims to explore in depth the meaning and meaning discrepancies among older Norwegian breast cancer survivors in light of the meaning making model by Design: We utilized a qualitative design collecting data using semi-structured interviews of 23 elderly breast cancer survivors 7–8 years after treatment. The interviews followed an interview guide structured along three main themes: “everyday life activities,” “follow-up-care experiences” and “health status and QoL.”Results: Several health problems were reported by the women in the aftermaths of the disease, such as sleeping problems, pain, and fatigue—including cognitive and emotional impairments. Meaning discrepancies were concentrated on six main themes: shifting perspectives and priorities, growing sense of autonomy, widening the limits of normality, dissociating oneself from the disease, embracing alternative health services, and feeling lucky. The women engaged in a wide range of coping techniques as efforts to change global meaning, and to develop a more positive view on the cancer experience. Common coping efforts across the six main themes were social comparison, denial, positive reappraisal, problem-focused coping, and revaluing ordinary events.Conclusion: Many cancer patients report on unmet needs for help with their meaning making, and the facilitation of meaning making processes is rarely included in the follow-up care of cancer survivors. The findings of the present study may help health care professionals provide care for women who have experienced breast cancer. The concrete knowledge of common coping efforts in the meaning making process may contribute to the development of future interventions and for gaining a deeper understanding for older survivors of breast cancer. As breast-cancer treatments have become more advanced, more than 90% of women experiencing breast cancer enters long-term survivorship (>5 years after diagnosis) . From 19global meaning, i.e., general understanding of how the universe works, self-identity, control, justice, God, goals, sense of meaning/purpose, and (2) situational meaning, i.e., meaning of specific events such as an illness  relates to increased psychological health and quality of life (QoL) among cancer patients controllable and (un)fair, and to what degree it violates the persons own goals  attribute their illness to be caused by increased levels of psychological stress , and loss of meaning after cancer  . In the l stress . Attribul stress .n = 78), the use of downward social comparison was common (>60% of the women)  . Others e women) . Patiente women) . AscribiThe process of accommodation is initiated when attempts to change the meaning of an event are unsuccessful in reducing incongruence to global meaning . People Positive reappraisal is a cognitive process in which people place greater focus on the good things that have happened to them . PositivProblem-focused coping involves strategies such as searching for information, problem solving, and the direct actions involved in solving a problem . ProblemFinally, a change in global meaning may also involve increasing the value of events that previously were considered more “ordinary.” This may apply to everyday events such as having dinner, getting together with friends and family, and enjoying nature and beautiful sceneries . DespiteMany cancer patients report on unmet needs for help with their meaning making processes . Since tWe utilized a qualitative  design involving semi-structured interviews of 23 elderly female breast cancer survivors 7–8 years after their treatment for breast cancer. The interviews followed an interview guide structured along three main themes: “everyday life activities,” “follow-up-care experiences,” and “health status and QoL.” This third theme included both physical, mental, and existential dimensions of QoL.As a part of a larger long-term follow-up study 6–8 years after breast cancer treatment at Trondheim University Hospital, the patients were consecutively invited (by their oncologist or nurse) to participate in an explorative interview-study about life beyond cancer. Patients who agreed to participate were contacted by the last author, and appointments were made for the interviews (place and time). None of the patients received any incentives for participating.The interviews were conducted either at the participant’s home or at an arranged meeting place, most often at a dedicated room at the University hospital. The interviews were performed face-to-face by the last and the second author, working within the research fields of psycho-oncology and medical sociology. The remaining authors were engaged in research in the fields of psychology, health informatics, and nursing. The interviews, which lasted for 90 min on average, were electronically recorded, and verbatim transcribed by two independent research assistants. Two researchers analyzed the transcripts, and all subcodes, themes and final categories, including the quotes, were translated into English and thoroughly discussed among all members of the team aiming to strengthen the results’ reliability.“life has become harder,” “I do not suffice,” “I am not the same as before,” and “I am tired” were merged into the joint “life has become completely different.” All coding in these two steps were conducted using the QSR NVivo 12 software.The data analysis followed a stepwise, iterative strategy including both inductive, open coding and theoretical discussions, similar to the stepwise-deductive induction approach introduced by “I am not the same as before,” and “I am trying to live as normal” reflected both a discrepancy between being healthy  and being tired , and a tendency of reducing that discrepancy by widening the notion of what includes living a “normal life.” Hence, the final theme “redefining the limits of normal life” was established. To resolve any disagreements in coding, and thereby strengthen the validity of the results, the research team met frequently for discussions.The third step of the analysis involved research team discussions of preliminary themes from step two, guided by the theoretical framework of meaning making . This stThe analyses generated a total of six themes which are presented in the section “Results” with illustrating quotes presented in italics. In order to give a trustworthy representation of the informants’ perspectives, the results are presented with rich descriptions of the categories and several quotes.The informants were between 60 and 86 years (mean age = 67 years) at the time of being interviewed. Most of the women were married/cohabited and old-age pensioners. Current or previous occupation was cleaning, administration, health (shorter education) or farming. The women who were still working held executive-, health worker- or management positions. Characteristics of the participants are displayed in jumping on a train in motion. The interviews also bore evidence of an insoluble conflict potentially arising from incompatible expectations encountered when the women came back to life—cured from cancer and full of desires, but not necessarily being at good enough health to live up to neither their own, nor to other’s expectations.In general, the informants experienced life as quite different from the life before being treated for breast cancer. Many told stories about sleeping problems, pain and fatigue including cognitive and emotional impairments in addition to a sense of emptiness, and a general apprehension of not contributing sufficiently to society. An apt metaphor for returning to ordinary life was that it felt like Despite the discernible contrasts between the past and the present, the women seemed to perceive high levels of QoL, focusing on the good things in life rather than getting caught up in trifles or fearing recurrence of cancer. The initial analysis revealed universal and global thematic issues that were perceived as meaningful, such as being healthy, having family and friends, sharing life with a partner, commitment to working, gardening and being outdoors. In addition to these general meaning-categories, descriptions that were more clearly related to having experienced breast cancer, having received treatment, and subsequently the long-term impact on daily life, revealed the more situational meaning categories.The following text presents the main themes that emerged from the theory-based analysis of the meaning making processes that occurred when the breast cancer survivors attempted to adjust their situational meaning to their overall global meaning system, or vice versa. These themes are more abstract in their construction than the general ones mentioned above, and relate to shifting perspectives and priorities, a strengthened autonomy, widening the limits of normal life, dissociating oneself from the disease, embracing alternative health services, and feeling lucky. The themes are presented with illustrative quotes identified by a pseudonym for each person. All quotes are translated from Norwegian to English.Many informants perceived the disease and the following new life situations as a turning point toward a novel awareness that life is not a matter of course, which, in turn, became a reason for living their lives differently and not put things on hold.All those things, buy, buy, buy, it’s not important anymore. Rather try to live in presence; it’s better spending your money on nice experiences (rather than things) (May).Some of the women even reflected on a possible purpose of the disease, and that the new situation (having experienced cancer) made them change direction in life:I believe there’s a meaning to everything that happens. Perhaps it was meant to be, so that I could learn to be more relaxed, blessing in disguise, in a way (Mary).I don’t attach importance to details anymore. When you realize how fragile life can be… Today I really think life is even more valuable. I don’t focus on details anymore (Linda).Likewise, a more conscious distinction between important and unimportant issues in life was prominent in the material, especially matters that were associated with energy capacity. For many informants, yesterday’s problems had become today’s trifles; As a result of such changing priorities, there seemed to be a growing awareness of things that previously were taken for granted. It was often referred to as being more aware of their supportive family and close friendships, and of being more present in the very moments. In addition, the women explicitly pronounced a determinate gratification of being able to convey values and important life contents from generation to generation:I’m happy for every birthday I celebrate. I’ve grown much more conscious—after having such a diagnosis. I am grateful for the children having managed so well, and I’ve got two grandchildren who will always remember their granny. If I disappear (die), they will always remember me (Elizabeth).In sum, the shift in priorities was closely linked both to new ways of thinking, acting and reacting, and in the wake of these alterations, new values were identified.The informants reported that their individual needs and preferences had become increasingly important after experiencing breast cancer. It seemed that the fragility of life had become more visible, resulting in a growing awareness that they must live their life “right now.” Some of the women described it as “an egoism without shame.” Previously in life, they often felt bad when prioritizing their own needs above others’. After having lived through cancer and all its associated experiences, these feelings had changed dramatically. Paying more attention to their own needs and desires and at the same time releasing themselves from bad conscience and shame was a typical reaction observed:You must pull yourself together and prioritize your own needs. You can’t sacrifice your own life for others. I try to protect my weekends, use the time on myself. It’s not always feasible, but I try: I feel I need it, need to do pleasurable things—to get my strength back (Linda).Because it has been so busy for so many years, I feel that it’s okay to cool down a bit. If I want to go away for a weekend, I’ll do it, and if I fancy a trip downtown, I’ll do that as well—without getting anyone’s permission… And yes, sometimes I have felt that it was nice having someone visiting me, now and then, but I can decide that myself, it’s my choice (Maria).The notion of grasping life events instantly, being more decisive and not wait until tomorrow etc., had become an unambiguously global value among the elderly breast cancer survivors in this study.I have become very stingy with my time. I don’t care about so many things… I hear people saying, “you must go there,” and “you should do that”… I live here, among my closest family… Why should I go to Mexico? I don’t see the point (Margaret).These shifts in global values did not happen rapidly. Rather, they required specific actions (from the women) to reach there. Many of the breast cancer survivors described their everyday life quite different from the life they lived before the cancer (and in their younger days), especially in terms of lack of energy. The breast cancer survivors felt they were not able to do things that they used to do before the illness. For example, many of them expressed that it was too demanding to invite family and friends for dinner, to join interest groups, or to make any long-term plans. The women told how they actively decided how and with whom they spent their time and how they were “forced” to reorganize their lives to redistribute their energy to live meaningful lives:I held positions in several organizations, and surely, I fulfilled my obligations. Now I have reduced such activities. You turn more egoistic and self-centered, and you think; No, I don’t want to bite off more than I can chew (Lisa).People (friends) call me, asking me to join them at different events, but I have become so stubborn and weird; I hide, and I only join things that suit me. My friends must humor me. In earlier days (before) I was amenable, but nowadays I’m not, I become easily irritated/annoyed (Karen).The discrepancy or experienced gap between the previous “shame over being selfish” and the current “proud of taking own needs seriously” could be interpreted as a growing sense of autonomy and be the outcome of this meaning making process.Redefining the notion of “a normal life” emerged as another weighty theme in our material. Many reported having a good health, even though they struggled with several health issues. Being healthy was likely a global desire and perceived important to be associated with. Being an ill and weak person was difficult to adapt to. The women seemed to be reducing the discrepancy between the self-perception of being a healthy human and the current feeling of being impaired, by widening the limits of what defines a normal life. Almost all informants explained their current health problems, such as pain, reduced cognition, or physical capacity (the situation) to be a consequence of growing older rather than resulting from late effects of treatment. Typical thoughts were:It happens a lot to your body when you get cancer, and all you’ve been through …It’s difficult to know whether they (the problems) are due to the disease, all you’ve been through, or only a result of aging. I don’t know (Mary).At the same time as they accepted the changes, some of the women emphasized that even managing everyday tasks, such as making dinner, gave evidence of living as normal:I try to live as normal as possible […] Yes, I do—I make dinner and all… I’m living a quite normal life now, so I have returned to the good old me…even if it’s…you know…you’ll never be the same. I just have to take my time…(May).Redefining the notion of normality became apparent likely because the breast cancer survivors strongly wished to appear as active members of society, even though many were both physically and cognitively impaired.An opposite strategy to widening the limits of normality was the denial, or at least striving to keep distance to the disease and anything that could be associated with it. Such strategy was often materialized by avoiding any participation in breast cancer supporting groups, as described by this woman:I don’t want to attend such a (BC-support) group. I don’t want to talk about disease. I don’t want to keep company with people with cancer. I want to lift me up—want to be with people who don’t talk about disease (Karen).I don’t think about it. I don’t worry or speculate. In fact, I pretend to be healthy (Linda).Another way of dissociating oneself from the cancer disease, was to make no place for breast cancer in their lives, as quoted: Dissociating oneself from illness and patient experiences, simply by defining oneself not belonging to the patient group, seemed decisive for many of the women in maintaining the important perception of being connected to society.Many of the survivors were struggling with various health problems, which were ascribed to either late effects of breast cancer treatment, or as a part of getting older. Even though most of the women accepted that their health and daily life were totally different compared to the situation prior to their breast cancer history, some of them praised different alternative and self-initiated ways of improving their health and thereby the quality of everyday life. For example, one of the informants reflected a deep interest in yoga and mindfulness and seemingly believed in the positive effect of these activities for improving sleeping quality and pain relief:I must admit, there have been some bodily alterations, both physically and mentally. I have attended some courses in mindfulness that I found very helpful. I easily fell asleep every evening afterward. I also decided to start with yoga—it is once a week. I found that I needed to stretch the muscles—I feel that it helps for the pain in my arm—I have a lymph node removed… (Mary).Another informant talked enthusiastically about the health-related impact of healing methods/techniques:I believe it is more to heaven and earth. Our travel guide practiced a Japanese treatment technique called Reiki. It is a kind of energy transference…how should I explain? You get more energy, better quality of life, better sleep, and such things. I had one “treatment,” and I’ve never felt so relaxed in all my life. It was very strange… Later she arranged courses, and I attended with several others, and it was useful/instructive (Maria).The positive effect of homeopathy and acupuncture were mentioned by some informants, especially the benefits related to better sleeping quality and increased physical surplus:One of my colleagues had good experience with acupuncture. So, I have spent 4000 (NOK), no matter whether it works or not, but I’ve got better. Although I’m still not completely healthy, I manage to sleep better at nights (Sandra).I had reached rock bottom when I returned to work. I went to a homeopath who offered acupuncture, and that got me on top. I visited him for a year. I think that was the reason why I manage to work after the cancer treatment. I had a great confidence in homeopathy, and my general practitioner agreed with me (Helen).Lastly, some of the breast cancer survivors also stated a firm belief that dietary supplements could be useful in improving the level of energy. One woman expressed the fortune of knowing someone who could provide these health promoting remedies, as follows:I have my own methods, you know, through acquaintances, I’ve got some tips. Last year I heard about something special, expensive though! Something named “Synergy”—a lot of vitamins and minerals and other ingredients. I have used it since July last year and I feel a great difference. It works very good for me. I‘ve got a quite different everyday life since I started (Ruth).Luck was a recurrent construct in our material. Even though the women had been through a breast cancer diagnosis, subsequent treatment accompanied by anxiety and worries, and still were experiencing bothersome late effects; almost all of them stated -in one way or another- how lucky they felt after all. The luck they explained was associated with various aspects in life; from the most fundamental aspect of having survived to the gratitude over different life events, which they previously had taken for granted. Many of the breast cancer survivors reported that all in all, they had been very lucky compared to others—whether it was someone they knew or other persons they had heard about. The consequences of the disease could have been so much worse, such as loss of life, worse adaption to the treatment, spread of cancer, or a myriad of late effects. The feeling of luck was extensively expressed by the breast cancer survivors:I’m praising every new day I get, and I don’t understand why I should be the one who survived…, because… they died like flies around me (Sarah).Afterward, when I reflected on what I’ve been through, I thought that I’ve really been lucky, because the cancer hadn’t spread to other organs (Elise).I’m lucky for being as healthy as I am. I know so many people who had a relapse of the cancer, and suddenly they’re gone. So, I feel lucky to be so content with my life (Lucy)!Although some of the women were striving with both late effects of treatment, other comorbidities, and even challenging family situations, they still adopted a perspective of luck. The women explained their luck or gratitude in several ways:My breast is deformed, and I find it disgusting, but I don’t care. That’s how it turned out, and I’m lucky who got cured (Evelyn).I’ve got the best treatment and I’ve been strong enough to tolerate it, so I‘ve been very lucky (Mary).Guided by the meaning making model developed by The general finding that the women perceived a shift from before the cancer illness were expressed as major turning points in life. Sudden and profound changes in global meaning was reported as a direct cause of the illness. Some came to see the cancer experience as a blessing—the meaning of cancer illness was to facilitate a stronger focus on what they considered important in life. Such positive appraisals of the cancer illness are linked to a reduction of incongruence to global meaning . For theThe shift toward greater levels of autonomy can be conceptualized as accommodations of global meaning . From thThe ability to feel a full sense of ownership to one’s behavior, regardless of traumatic life event also develops with age . AccordiDespite physical and cognitive impairments, being normal was important. The women defined normality as the ability to engage in the same everyday activities as before the cancer. Furthermore, a sense of normality was maintained by explaining physical and cognitive challenges to be a result of aging—a “normal” process—as opposed to a consequence of the disease, which is unexpected and generally less likely to occur. Maintaining age as an explanation of health problems may facilitate a feeling of being normal, and of having more control of the situation.n = 40) reported that upholding a more or less unchanged everyday life was important to them in life after cancer treatment , trying to ease or eliminate poor mental and physical health to improve their QoL such as practicing yoga and mindfulness, use of healing techniques, homeopathy, acupuncture and taking dietary supplements. A previous study reported that 59% of middle age- and older long-term breast cancer survivors use CAM, and these women had a higher prevalence of co-morbidities and poorer emotional functioning . The reaThe current study reported that feelings of luck and gratitude thinking about current life situation was a common theme among the breast cancer survivors. The women reflected on the increased risk for losing their lives to cancer, complications, and late effects of the cancer disease. Comparing themselves with others, including fictive persons, who were perceived as worse off, was common. Such coping methods of downward social comparison is common for assimilating new meaning to an adverse situation . The dicThis study provided an in-depth exploration of meaning and meaning discrepancies of older breast cancer survivors 7–8 years after diagnosis. The women struggled with several health problems in the aftermaths of the disease, such as sleeping problems, pain, and fatigue—including cognitive and emotional impairments. Meaning and meaning discrepancies was concentrated on six main themes: shifting perspectives and priorities, growing sense of autonomy, widening the limits of normal life, dissociating oneself from the disease, embracing alternative health services, and feeling lucky. In light of the meaning making model by n = 23) provided rich material, and the process of analyzing the data was comprehensive due to the introduction of the meaning making model halfway in the analysis, the interviewer could not at this point ask the informants to clarify or invite the women to further explore themes relevant specifically for meaning making. Future studies should therefore focus specifically on meaning and meaning making. Furthermore, some additional constructs evinced significant for coping and meaning making such as self-compassion and emotional intelligence should be included to get a deeper understanding of the phenomena of the meaning making process for older breast cancer survivors (The study’s strengths centered on the homogenous sample of older (60–87 years) breast cancer survivors, all being equal in time since cancer treatment (7–8 years) which allows for generalizing the findings to older, long-term survivors of breast cancer. On the other hand, the long timeline could result in recall bias and thereby influence the study’s reliability. The interviews were conducted to gain knowledge on life beyond cancer, and the semi-structured interviews were not specifically designed for gaining in depth information on meaning making. Although the number of interviews . The patients/participants provided their written informed consent to participate in this study.RS, KS, GH, MN, and RJR made substantial contributions to the study, including conceptualization, design, methodology, and analyses. RS, KS, and RJR were mainly performed the data analyses. MN did mainly perform the introduction, discussion, and conclusion. RS and RJR wrote the methods and results. All authors reviewed and edited the manuscript and approved the final version of the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


We showcase a methodology to discover topics via Latent Dirichlet Allocation, which can reliably produce human-interpretable topics over an open news corpus that continually grows with time. Our system generates topics, or distributions of keywords, for news articles on a monthly basis, to consistently detect key events and trends aligned with events in the real world. Findings from 2 years worth of news articles in mainstream English-language Canadian media indicate that certain topics feature either women or men more prominently and exhibit different types of language. Perhaps unsurprisingly, topics such as lifestyle, entertainment, and healthcare tend to be prominent in articles that quote more women than men. Topics such as sports, politics, and business are characteristic of articles that quote more men than women. The data shows a self-reinforcing gendered division of duties and representation in society. Quoting female sources more frequently in a caregiving role and quoting male sources more frequently in political and business roles enshrines women’s status as caregivers and men’s status as leaders and breadwinners. Our results can help journalists and policy makers better understand the unequal gender representation of those quoted in the news and facilitate news organizations’ efforts to achieve gender parity in their sources. The proposed methodology is robust, reproducible, and scalable to very large corpora, and can be used for similar studies involving unsupervised topic modelling and language analyses.We present a topic modelling and data visualization methodology to examine gender-based disparities in news articles by topic. Existing research in topic modelling is largely focused on the text mining of  Gender equality is one of the UN’s 17 Sustainable Development Goals . By all The underrepresentation of women in certain areas of the news such as politics, business, or sports is well documented . There iIn this paper, we tackle one measure of gender equality in the media: the number of times men and women are quoted in news articles. More specifically, we present a scalable topic modelling methodology to explore how many times men and women are quoted across news topics. Our unsupervised large-scale analyses are enhanced with corpus-based studies of the language used in male-prominent and female-prominent articles, thus revealing not only which topics are different, but exactly how they are different in their language.Our analyses rely on statistics provided by the Gender Gap Tracker,We apply topic modelling, an unsupervised machine learning technique , to thisOur results show that there are clear trends in the proportion of men and women quoted depending on the topic of the article they are quoted in. Women are consistently quoted more frequently than men in the topics “Lifestyle”, “Arts and Entertainment”, and “Healthcare”, whereas men are quoted more frequently than women in the topics “Business”, “Politics”, and “Sports”. This comes as no surprise, regrettably, as the frequency of quotes seems to reflect a gendered cultural division of duties, where women are placed into the role of caregiver and nurturer and are less represented in the spheres of politics and business. This gendered division of duties and representation in media is no doubt self-reinforcing; quoting female sources more frequently in a caregiving role and quoting male sources more frequently in political and business roles enshrines women’s status as caregivers and men’s status as leaders and breadwinners . Since tWe would like to acknowledge that the terms we use in this paper are simplifications of a complex reality. We use the terms “women”, “men”, “female sources”, and “male sources”, implying a binary opposition that we know is far from simple. We know, at the same time, that lack of gender representation in many aspects of society is a reality. The aim of this study is to quantify that gender gap by analyzing language and the traditional associations of names with men and women.Data was obtained for the Gender Gap Tracker and analyzed for this related project. The Gender Gap Tracker data is scraped daily from the websites of seven major Canadian news outlets in English . We builWe analyzed 24 months worth of data, a total of 612,343 articles. As we see in sources. For the latter, we first identify quotes  in the text and then associate a speaker with them, using coreference resolution. Finally, we look up the gender of the speakers  using a mix of sources: an in-house cache of commonly quoted public figures and an API that stores the gender of people based on their first or full name. Thanks to this NLP pipeline, we obtain gender statistics  for each article, whose aggregated form is shown in Using the Gender Gap Tracker’s existing Natural Language Processing pipeline , the datpeople mentioned refers to all unique individuals named in an article, regardless of whether or not they were quoted. These names are extracted and identified by our NLP pipeline via Named Entity Recognition. People quoted refers to the subset of people mentioned that were quoted one or more times in an article, identified via coreference resolution. The counts of people mentioned and people quoted are per article, as we do not keep track of the same individual across articles, that is, our unit of analysis is the article.At this point, we would like to elaborate on our nomenclature. In this work, To illustrate this more clearly, consider a “set” of two articles. If the politician Chrystia Freeland is mentioned in both articles and has three quotes in one article and none in the other, this means two women were mentioned (one woman in each article) and one woman was quoted (one woman quoted three times in one article) in the set of all articles. If, however, Chrystia Freeland was quoted once in one article and once in the other, and another politician, Christine Elliott, was mentioned in the same two articles (but not quoted), two women were mentioned and two women were quoted in that set, with Chrystia Freeland counting as one woman in each article that she was quoted.Numbers alone tell a compelling story about the lack of female voices in the media, as we show in more detailed analyses of the Gender Gap Tracker data . Our goatopics, or themes, in a collection of unstructured documents  is by far the most popular. Because of LDA’s inherently non-deterministic and statistical qualities, methodological decisions involving data preprocessing, choosing the number of topics, tuning the algorithm hyperparameters, and evaluating the model’s results become all the more important .Gibbs sampling, in which a Markov chain of random variables is constructed with each variable dependent on the previous ones—the limiting value of this distribution equals the true posterior. The algorithm is run on the Markov chain defined on the hidden variables for a particular corpus and a number of samples are drawn using a Markov Chain Monte Carlo algorithm, following which the approximate distribution is constructed from the collected samples. While sampling-based methods are guaranteed to be identical to the true posterior under limiting conditions and can produce less biased results overall, they are quite computationally expensive and do not scale as well as variational Bayes methods do, as the corpus grows in size .By contrast with verbs, we chose to retain a large portion of nouns during text preprocessing, due to the importance of nouns in topic interpretation. It has been shown that a combination of lemmatization and limiting the corpus to just nouns provides a coherence advantage and lower word intrusion in topic modelling . While wOur final stopword list, a sample of which is shown in Term lemmatization is performed using a lookup process during run time, where each token is checked for presence in a dictionary of lemmas provided by the user. To do this, we turned to the spaCy NLP library,In this section, we discuss the key LDA hyperparameters we tuned to achieve our best results. Some additional modelling experiments are discussed in the The online variational Bayes algorithm was observed to converge quite rapidly (within 100–150 iterations). By contrast, we found that expectation maximization can take upwards of 200 iterations to converge. This makes sense, since the online algorithm is known to speed up convergence by using stochastic optimization and is less prone to getting stuck in local optima. We did not notice much of a difference in terms of perplexity as well as topic interpretability when running the online algorithm for 150 iterations and above. Minor differences in topic keywords were noticed for specific months’ results when we used just 100 iterations. As a result, we fix the maximum number of iterations to 150 for all our training runs, regardless of the size of the data involved each month.Three relative pruning hyperparameters are used during data preprocessing: maximum document frequency, minimum document frequency, and minimum term frequency. Controlling these values prunes, i.e., removes, those terms that occur both very frequently and very rarely from the vocabulary. As a result, these hyperparameters must be chosen carefully, so as to not hinder topic interpretability or miss larger themes in the data.We found that a maximum document frequency of 80% The minimum document frequency hyperparameter , the minimum term frequency is specified as a positive integer. A minimum term frequency of 1 means that no terms are pruned from a document, whereas setting The minimum term frequency hyperparameter prunes terms that occur fewer than a specified number of times Setting a finite vocabulary size limits the dimensionality of the random matrices that are solved for during LDA , our scenario is well-suited to e press” . Corpus e press” .For the language analyses, we rely on two key concepts from corpus linguistics, keyness and dependency bigrams. Keyness aims to identify large differences between the frequency of word-forms in two or more corpora . KeynessThe playerkickedtheball, the main verb kicked is connected to the noun ball via a direct object relationship, wherein kicked is the head and ball is the dependent. The dependency bigram returned using this process is “ball_kick”.A relatively more advanced form of corpus analysis, dependency bigrams identify verb-direct object combinations and their frequencies within a corpus . To extrDependency bigrams reveal linguistic aspects of the male and female corpora, unlike keyness, which mainly highlights textual or lexical differences. We obtained much more useful results with dependency bigrams rather than the more standard corpus analysis methods such as collocation or n-gram frequencies . This ishttps://gendergaptracker.research.sfu.ca/apps/topicmodel), we obtain per-month, per-outlet mean topic gender prominence measures for each of these topics over the 2-year period. We define gender prominence as the difference in mean topic weights between the female and male corpus or dataset for a given topic. A topic is categorized as having male prominence if the mean topic weights from the male corpus are greater than those from the female corpus. See the last section in the We examined nine topics that feature regularly in most months between October 2018 and September 2020. The topics cover a broad range of domains typically seen in the news, as seen on the vertical axis of The values for prominence range from positive red in , indicatOther topics that display strong female prominence over time include “Lifestyle” , “Healthcare and medical research”, and “Legal and court cases”. The primary reason for the “Lifestyle” topic being female-prominent is that it regularly features mothers involved in childcare, women reliving their past experiences, and female experts offering personal care advice. The “Legal and court cases” topic tends to feature more women mainly due to cases that pertain to sexual assault or women’s rights, featuring both victims and experts who are women. The “Healthcare and medical research” topic is particularly interesting, because, in Canada, a sizable proportion of healthcare experts and Public Health Officers happen to be women . This leOn the other hand, the “Business and market events”, “Sports”, “Federal politics”, and “United States politics” topics are almost entirely male-prominent throughout the period studied. The “Business” topic typically includes keywords pertaining to investments, analysts’ predictions on the stock market, and in-depth discussions on company futures, interest rates, and profits. Historically, these areas tend to be dominated by men, partly explaining why the topic intensity for “Business” is always much lower for the female corpus. Similarly, the “Sports”, “Federal politics”, and “United States politics” topics feature discourse that generously quotes male athletes, coaches, and politicians, who are not only more in numbers in society, but are also quoted much more often than women.In general, the heat map in distribution over topics. Mathematically, this means that the topic modelling process returns a one-dimensional vector of length 15 (we only model a maximum of 15 topics each month) for each document, where each component of the vector, which we call a topic weight, represents how strongly or weakly that topic’s keywords are associated with that document. To begin the combined topic and language analyses, we identify a topic of interest that exhibits strongly male or female gender prominence for a particular month. We then query that month’s data from our database and sort in descending order of topic weights for that topic. Sorting the articles in this order places all articles that are strongly related to that topic’s keywords on top. The full corpus, in sorted order for a particular topic, is then split into two corpora, each with male-majority and female-majority sources . We chose 200 articles for empirical reasons—we observed that in most cases, the maximum topic weights for each article rapidly dropped to less than 0.5 after a few hundred samples. Following these steps, we perform keyness analysis and extract dependency bigrams on five topics: “Sports”, “Business and market events”, “Lifestyle”, “Healthcare and medical research”, and “Crimes and sexual assault”. Below we discuss our analyses of these five topics for certain months.The first topic whose language we study in detail is “Sports”, for the months of June, July, and August 2019. August 2019 was an outlier because it was the only month for which the “Sports” topic demonstrated an overall female prominence. August 2019 was also particularly interesting because “Sports” was the strongest topic in both the female and male corpora. To better understand these trends, we performed keyness and dependency bigram analyses for the top articles from each of these 3 months. The keyness results uncover lexical differences between the female and male corpora, whereas the dependency bigrams show syntactic  differences based on subject-direct object relationships, that is, events and relationships that the articles capture.Yankees, Red Sox) and teams from the Canadian Football League . The dependency bigram plots indicate that there are differences in language across the female and male corpora, based on frequency counts of the most common bigrams.  The male corpus contains much higher occurrences of general verbs describing events in baseball/football games, such as hit homer, score goal, allow run, and give lead. The female corpus seems much more focused on winning, celebrating, and breaking records .From the keyness index plot, we can see that the female corpus contains numerous terms pertaining to the FIFA women’s world cup held in France that month. The names of Canadian and American soccer players and those of their female coaches appear, as do other terms related to women’s soccer federations. The male corpus, on the other hand, contains terms largely related to baseball . The male corpus continues to highlight terms and player/coach names from baseball (Blue Jays) and the CFL (Redblacks). The dependency bigram plots also reveal comparable results, with the female corpus using more vocabulary related to winning cups/medals and setting/breaking records. On the other hand, the male corpus much more frequently highlights terms that describe specific aspects of games, such as giving the opponents the lead or allowing them a run.July 2019  follows Finally, in August 2019, shown in win cup and break record), whereas articles that feature men’s sports focus on the finer details of each game. Perhaps authors that write about women’s sport highlight women’s achievements and success stories because of the limited time and space they have, compared to the coverage given to men’s sports. After all, the baseball and football seasons in Canada are each several months long, whereas events like the FIFA world cup and the Wimbledon tournament occur in the span of just 1 month. It does seem that, when it comes to sports, men just need to play to feature in the news; women need to win.This analysis of the top “Sports” articles across three summer months in 2019 shows that articles that feature women’s voices tend to focus more heavily on achievements and records , we see considerable amounts of coverage in the female corpus during certain times of the year. Sustaining a high level of coverage, however, seems to require more women role models across a wider range of sports, as well as longer seasons over which women’s sports are regularly featured. Since the COVID-19 pandemic in early 2020, our topic model gender prominence results show that women’s sport has once again taken a back seat in the news.The underrepresentation of women in sports news has been amply documented, showing that coverage has not kept pace with the increase of women’s participation in sports . HoweverWe consider 2 months, October and December 2018, for which this topic exhibits a particularly strong male prominence. The top 200 articles with the strongest topic intensities in each month were chosen for comparison.milk, shave) and small/local businesses . The dependency analysis gives us expressions like have impact, cover cost, pay debt, and run business. The male corpus contains terms related to larger financial instruments, stocks, trading, and company performance metrics. The male corpus also frequently features verbs pertaining to high-level business policy and decision-making .In October 2018 , in the gender, workforce, child, and racialized. The male corpus, as always, contains terms very specific to financial and business operations . Inspecting the dependency bigrams, we can see that the female corpus highlights terms such as cut production and make decision—this was primarily due to then-Alberta Premier Rachel Notley’s numerous comments on the oil industry’s production cuts. Other terms such as close gap, spend money, and provide service clearly point to some level of gender disparity between the two corpora.December 2018, the month for which the “Business” topic exhibits the strongest male prominence in all our results, showcases a similar disparity in the kind of language used across both corpora, as seen in The Globe and Mail, which we know quotes, on average, the smallest proportion of women.We find that the “Business and market events” topic regularly exhibits the largest disparity between the male and female corpora, in terms of language used, for all months of data studied. It seems that women’s voices are rarely featured in articles that cover business, especially big business, finance, and the stock market. The kind of language used (as seen in the dependency bigrams) shows that the focus of articles that quote women tends to be primarily small/local businesses and the challenges of working at or running a  business. It is also important to note that a large portion of articles in this topic were published by The “Lifestyle” topic, which covers content related to families, personal experiences, holidays, and shopping, exhibits strong female prominence for all months in which it appears. tbsp, tsp) to clothing and fabrics. Interestingly, not as many women’s names are seen for this month’s keywords as for other topics in other months. The male corpus keywords include several names of people, including Suresh Doss, a Toronto-based food writer. The dependency bigrams for both the male and female corpora focus mainly on the festive season, as well as acts of gift-giving and donating to charities.Keywords from the female corpus for December 2019 highlight a range of lifestyle topics, from food recipes  emerge, mainly because multiple outlets tend to write about the same topic, typically quoting the same people.We find that the “Lifestyle” topic, unlike other topics, tends to name regular people who share their life experiences with the media—this can lead to the drowning out of other keywords such as common nouns or adjectives in the keyness results. As a result, keyness is slightly less useful in interpreting textual disparities across the corpora for this topic. On occasion, interesting terms , while the male corpus shows a similar focus toward terms pertaining to men’s health . Interestingly, the bigrams from the female corpus seem to point to an additional emphasis on the effects of women’s personal choices on fertility and pregnancy . The male corpus bigrams, in this case, do not seem to have a particular focus other than the availability and effects of various treatments in the healthcare domain .The female corpus for March 2019  containsOnce the COVID-19 pandemic began spreading in Canada in March 2020, there was a stark shift in healthcare coverage. Because of the seriousness of the pandemic, there were multiple COVID-related topics in March. We focus our attention on four topics that were clearly female dominant throughout this period . The text of the top 100 articles for each topic (ordered by topic intensity) was collected and separated into male and female corpora, just as before. This gave us 400 articles in either corpus to analyze for their language used during this month. Keywords and dependency bigram for this corpus are shown in confirm case is significantly higher (>200) in the female corpus than the male corpus. This makes sense, considering that, for the most part, the Provincial Health Officers confirming the numbers happened to be women.Some of the keywords in the female corpus are names of women who are medical experts (Dr. Vera Etches and Dr. Penny Sutcliffe) or place names related to the spread of COVID-19 in various regions of Canada and the world. Similarly, the male corpus also names prominent medical researchers and scientists (Dr. Srinivas Murthy and Dr. Brent Roussin). The dependency bigrams show a very similar distribution across both male and female corpora, with terms pertaining to symptoms, case counts, and public health guidelines. Notably, the frequency count of the bigram In general, we observe that Canadian media outlets tend to provide a good degree of coverage to women’s healthcare and health issues throughout the year. This is helped by the fact that a relatively large proportion of healthcare experts and medical officers in Canada are women, which explains the high female prominence of the topic in most months of the year. The most notable of these is Dr. Bonnie Henry, the Chief Provincial Health Officer of British Columbia, who is the top quoted woman by far throughout the spring/summer of 2020, due to the COVID-19 pandemic. This leads to the conclusion that having women in prominent positions increases their chances of being quoted by the media, though this is, naturally, beyond the control of news organizations and journalists.In exploring our results, we observed that topics pertaining to women’s rights and sexual assault emerged around February and March, in both 2019 and 2020. We explore these topics for these periods using corpus studies, as shown below.hijab, trans, hormone). The issue of the gender pay gap in sports was also a major one during this period, shown through keywords such as fifa and cup. The male corpus keywords primarily feature names, presumably of men involved in crimes and their victims. Some key dependency bigrams from the female corpus include protect athlete, involve victim, and take action, indicating that coverage was focused on the action taken toward women’s justice. The male corpus highlights bigrams related to the crimes themselves, convictions, and legal matters.In February 2020, terms pertaining to women’s and trans rights appear in the female corpus . The male corpus keywords once again consist of names of the perpetrators or victims involved. In general, the language used in the female corpus (as seen in the bigrams) tends to focus on the victim’s accusations and details of the crimes, and on the convictions, defense, and accounts of the victims in the male corpus.In February 2020, as seen in take stand, tell story, have right) are used quite frequently in the female corpora, highlighting the role the media plays in spreading awareness about women’s issues and rights in Canada.It is quite interesting that topics around women’s rights  tend to emerge at the same time each year, around February and March, which is the time when International Women’s Day is celebrated. Analyzing the language used, we also find it heartening that strong action verbs  is. Almost every sport has a men’s and women’s division. But it’s often the men’s that is more lucrative and/or prominent. In that case, setting the baseline to the number of prominent or professional men athletes, would, in our view, simply reinforce the existing bias. This is why we assume a 50% baseline, that is, the approximate proportion of women in the world, in order to highlight the existing gap in women’s representation in the media.α and β, reliably produces topics that align well with events from the real world, while also effectively scaling to large datasets. In contrast to prior work that recommends removing stopwords as a post-processing step . During feature transformation, a common approach is to tokenize the corpus into bigrams . Words tThe second potential limitation pertains to model evaluation. Although there exist clean, well-tested implementations of popular topic model quality metrics such as Pointwise Mutual Index (PMI) and UMass’ topic coherence , Spark’sFinally, with respect to reproducibility, a common practice to reproduce results in LDA is to fix the random seed across runs. This should in principle work well, since it ensures that the Dirichlet samples are drawn from the same random distribution across runs. However, because Spark is a distributed system, there is an additional layer of randomness in the methodology, i.e., that of the distributed system itself. Due to the way Spark distributes data, the order of samples passed to an executor cannot be guaranteed, nor can the same random number generator seed be guaranteed on each executor. As a result, this stage of the process is non-deterministic. This causes a very minor deviation in the topic keywords discovered across multiple runs, even with the same random seed on the exact same data. However, our numerous experiments over multiple months of news data confirm that this does not affect topic interpretability or the overall robustness of the process.A study of bias is not complete unless the full gender spectrum is considered, as well as other forms of diversity such as status as visible minority, disability, or age. Our language-based analyses rely on names to assign gender to individuals and are not well suited to study such forms of diversity. Additionally, potential harms from assigning specific labels to individuals need to be considered . NeverthOne area we are mindful about is the relative lack of work that studies topic modelling and gender disparities in languages other than English. The Gender Gap Tracker project has also been collecting news articles from Canadian French-language news outlets since October 2018, which we have not yet analyzed. Research on morphologically rich languages such as Russian or Swedish has shown that effective pre-processing techniques, such as lemmatization, can greatly improve topic interpretability . Studyinstated vs. claimed) or in the way that the speaker is introduced, as an expert or merely a source. Analyses of transitivity structure in clauses can yield insights about the type of roles women are portrayed in  is updated every month, providing numbers and visualizations that can help Canadian newsrooms and policy makers monitor whether progress is being made. We open-source our code and visualization framework, so that others can also apply this methodology.Our public dashboard (