Measuring psychological attributes, such as motivation, typically involves rating scales, assuming that an attribute can be ordered, and that ratings represent this order. Previously, only the first assumption had been tested, albeit limited. First, we checked the ordinal structure of motivation, looking at whether people can establish transitive relations between motivation levels in pairwise comparisons; and we found different ordering patterns: strict transitive, weak transitive, changing order, and intransitivity. The rate of intransitivity was similar to that found previously and was somewhat higher than we obtained when we asked participants to compare definitely quantitative attributes (such as weight). Second, we checked if specific ordering patterns were related to individual interpretations of the statements that deviated from expected motivation types. Indeed, about a third of participants miscategorized statements, and these deviant interpretations were related to intransitivity as well as weak transitivity. Third, we checked whether Likert ratings represent the order of motives obtained from pairwise comparisons. We found rather homomorphic representation: ratings correlated with the order, but they did not differentiate between different ordering patterns and hierarchies of motives. We conclude that the Likert rating scale provides less information about respondents than pairwise ordering. The findings question the mainstream practice of using rating scales without testing underlying assumptions. The question about the additive vs. ordinal structure of psychological attributes concerns the very basic assumptions of psychometrics. However, this assumption is rarely explored experimentally. The lack of such explorations can be mostly explained by fundamental difficulties with the experimental control of extraneous factors in psychological research, which is crucial for establishing an attribute structure . AnotherHere, we addressed the most basic assumption underlying measurement in psychology, namely, the ordinal structure of psychological attributes. The ordinal structure is confirmed if any three levels of an attribute meet the conditions of transitivity, asymmetricity, and connectivity. For example, to be transitive, any of the levels of an attribute a, b, and c must satisfy the following condition: if a > b and b > c, then a > c; otherwise, there are transitivity violations that prevent ordering levels of the attribute, and the attribute cannot be treated in the ordinal scale. Moreover, since the ordinal structure is a prerequisite for quantitative structure, the failure to meet the order conditions makes the attribute immeasurable on the interval scale .A few studies on the ordinal structure of psychological attributes have shown that most people generally are able to order attributes in triads using pairwise comparisons  without transitivity violations . It is iHowever, due to the small number of such studies little is known about the sources for intransitivity. Some researchers have found that transitivity is violated when stimuli require more cognitive effort   or embedTherefore, this study was driven by the need to replicate findings on transitivity, and in addition, to test effects of individual interpretations of statements on ordering and transitivity.There is also another, almost unexplored, question regarding the Likert scales commonly used to rate psychological attributes. Given that some people fail to preserve transitivity when ordering attributes, how does this intransitivity affect their Likert ratings? Or, if people judge attributes as equal in pairwise comparisons, how do Likert ratings reflect this? We found only one paper on the transitivity of work values where these questions were partially explored . The autIn the current research, we focus on motivation, a psychological attribute that is typically assessed with Likert scales. The main idea of this study, besides the exploration of motivation ordinal structure, is to explore if and how Likert ratings reflect possible patterns of ordering motivation. Specifically, the study aimed to answer the following questions: (1) To what extent will previous results on the intransitivity be reproducible on new data? (2) To what extent can intransitivity be attributed to individual misinterpretations of statements? (3) How do the patterns of ordering motives (including intransitivity) and individual misinterpretations correspond to Likert ratings of these motives?To our knowledge, two methods of testing transitivity have been used. One is the binary tree procedure based on the unfolding theory . It consThe binary tree procedure is more demanding since respondents deal not just with the semantics of the attribute  but also with the semantics of the descriptions of the attribute levels (extremely support vs. support but not encourage). The procedure of pairwise comparisons is a less laboring method, as respondents deal only with the semantics of the attribute. Moreover, since we were interested in how the Likert scale can identify intransitivity and other ordering patterns, it was more practical to take ready-made statements from a well-established Likert scale, than to construct new statements for the binary tree procedure. So, here the method of pairwise comparison of attributes was used.M = 20.9 years, SD = 3.5) participated in the study. They were university students who got extra points as a reward for participation.Two hundred and fifty-two participants :I am interested in learning (I1).I just like to learn and learn new things (I2).I really enjoy learning new materials during classes (I3).External motivation (E):I have no choice, otherwise I will not be able to have a comfortable livelihood in the future (E1).I have no choice, because class attendance is being marked (E2).To avoid problems with the study office and exams (E3).Amotivation (A):I have got used to going there, but honestly, I do not exactly know why I do it (A1).I once had good reasons for going to university; however, now I wonder whether I should continue (A2).Honestly, I do not know; I really feel that I am wasting my time at university (A3).Each statement from one motive was paired with each statement from two other motives, so, there were 27 pairs of statements. The order of statements in each pair was random.First, participants compared the statements in pairs and chose one which better explains why they go to study. The participants could tick off “equally important” if they could not choose between motives.For example, a pair of motives were presented as follows:I attended classes last week because…I am interested in learning.- To avoid problems with the study office and exams.- Equally .The participants could not change their answers or return to the previous pairs after they gave an answer.Second, participants were asked to evaluate the same nine statements using a 5-point Likert scale (from 1-completely disagree to 5-completely agree).Third, in order to check whether specific ordering patterns were related to individual interpretations of the statements that deviated from expected motivation types, we asked the participants to relate each statement to one of three types of motivation. Motivation types were described for the participants as follows:Internal motivation-a person aims to learn new things and enjoys the process of studying.External motivation-a person aims to avoid problems right now or in the future and get some benefits from the study.The absence of motivation, or amotivation-a person goes to study because he/she has got used to it, but does not see any sense in it.In addition, we attempted to establish the “lowest” rate of intransitivity which we could expect even for definitely quantitative attributes. Therefore, in addition to the main procedures, we have asked participants to order well-defined attributes, such as age, height, and weight . If someTo assess the relation of specific ordering patterns with individual interpretations of the statements that deviated from expected motivation types, we used a chi-square test with a 95% CI as a threshold.To estimate the relation between the Likert ratings and subjective similarity of motives, we first calculated the differences between the totals on each Likert subscale, thus receiving a subjective distance between motives. Then we summed the number of times the “equally important” option was selected for a respective pair, internal/external, etc., for each individual. Then we calculated Pearson’s correlations between the subjective distance and the number of the “equally important” using a 95% CI.To assess the effect of the ordering patterns on between-motives differences in Likert scores we used a between-group one-way analysis of variance with a 95% CI.Firstly, we conducted quality control to remove participants who might not take the task seriously. We checked how participants ordered people by age. There were four people who rated themselves as older than their mothers, and one participant assessed his/her and mother’s age equally. We excluded these participants from the analysis. As a result, 247 participants were left for further analysis.n = 167) showed only transitively ordered motives responses.The pairwise comparisons resulted in a triad, e.g., I1E2-E2A3-I1A3, in such a way that we could check transitivity, e.g., I ≥ E ≥ A, and I ≥ A. For each of the 252 participants, we analyzed 27 triads. Overall, there were 2,223 triads, amongst which 178 (8%) were intransitive. This 8% of intransitive triads were made by 80 participants (32.3% of the sample). Around 68 percent of participants . In other words, if we have 2–3% of transitivity violations with undeniably quantitative attributes, then 8% of violations obtained for the motivation attribute is not quite large.Participants demonstrated different ordering patterns: (1) “strict transitive” who did not use the “equally” option at all and did not change the order of motives across triads (11%); (2) “weak transitive” who resided on the “equally” option at least once (46%); (3) “changing order” who ordered motives differently across triads, still preserving strict transitivity in each triads (11%); (4) “intransitive” (32% with at least one violation).2 = 6.52, p < 0.01]. In other words, participants interpreting statements differently were more likely to violate transitivity or resort to weak order, than change order.What was the role of individual interpretations of the statements in different ordering patterns? Namely, we explored the effects of the deviation of individual interpretation of some statements from the commonly expected interpretation. When the participants were asked to relate each statement to one of three types of motivation, a total of 93 people (38%) misclassified the statements. These “deviant” interpretations were related to weak transitivity and intransitivity, but not to the “changing order” pattern [χF = 29.04, p = 0.001. Post hoc analyses using the Bonferroni adjustment indicated that the average Likert score difference was significantly lower in the intransitive group than in the other three groups (see p < 0.001) between the number of the “equally” options chosen in pairwise comparisons and the between-motive differences in ratings: the more often the two motives were considered equal, the less was the difference between these motives on the Likert scale.We could expect that the biggest between-motives differences in Likert scores might come from “strict transitive” individuals rather than from “weak” or intransitive ones. Indeed, the results were partly in line with this reasoning –3 as theoups see . AdditioHowever, when we checked whether there was a minimal difference between the ratings of two motives which could guarantee that the participant used the “equally” option or made an intransitive triad at least once, we found no such between-motives differences . As you For transitively ordered motives, it was possible to derive individual hierarchies, that is, to say that one motive is dominant and the another is subordinate (for strict transitivity) or that at least one motive is dominant or subordinate (for weak transitivity). The question was, how did Likert ratings correspond with these hierarchies?For each participant, we compared the hierarchy of the motives derived from triads with the hierarchy of the respective motives derived from the Likert scale. We found almost perfect correspondence: the ratings failed to reflect pairwise hierarchies only in 3.8% of cases. Thus, the Likert scale mostly represents the ordinal relations obtained in pairwise ordering. However, the inverse operation was not possible: the Likert scale provided no information about what exactly hierarchy was obtain in pairwise ordering. For example, Likert scores 15, 13, 11  could correspond with a full hierarchy I > E > A. But the same Likert scores could just as well correspond with a partial hierarchy where only a dominant I motive or only subordinate a motive was identified. This is because Likert ratings did not distinguish between strict and weak transitivity, as well as intransitivity. In other words, the Likert scale provided less information about motivation than pairwise comparisons did.p > 0.30). Also the correlation between the number of the “equally” option chosen and between-motives differences in Likert ratings was practically the same as for participants who misclassified and for those who classified correctly (ranging from r = −0.50 to r = −0.35 (p < 0.001 for each correlation for different pairs of motives)).We did not find any indicators in Likert ratings that could be associated with participants who misclassified the statements across motivation types. There was no statistically significant difference in Likert scores between those who misclassified and those who did not  or treating motives as similar. Usually, different ordering patterns, as well as intransitivity, are explained by subjective similarity of motives. For example, if the subjective difference between motives is large, then the individual can order them strictly. However, if the difference is small, then the individual can either equate the motives, change their order, or violate transitivity .Without refuting this explanation, we allow for another. Namely, the individual may consider statements in a pair as belonging to the same motive. This would force the individual to use the “equally” option or violate transitivity. But in this case, the subjective similarity would not concern the strength of motives, but their essence; they would be perceived as identical. At least insofar as the statements that represent the motives are perceived as identical.Trying to prevent this problem, we paired statements belonging to different motives and relied on their semantics and on empirical data about their factor structure . In spitNevertheless, the “changing order” pattern seems not to be affected by individual interpretations. We can only speculate that this pattern can be caused by the instability of the individual motivation hierarchy. A similar conclusion can be found in decision making research, where it has been shown that people change their minds during self-reporting and thus do not show stable preference, yet preserving transitivity . It shouRegarding the second of our main questions  we found that the Likert ratings did correlate with the hierarchies, but they provided significantly less information regarding an individual than pairwise comparisons did. In particular, the ratings did not distinguish (1) a complete hierarchy of motives, when all motives were strictly ordered, from (2) a partial hierarchy, when the position of only some motives could be determined, and from (3) intransitivity, when a respondent could not transitively order the motives.Technically, the scale’s poor distinguishability can come from its limited sensitivity to the level of subjective similarity between motives. Although between-motives differences in ratings corresponded with how often participants equated the particular motives in pairwise comparisons see also , we did The absence of thresholds for between-motives differences could be due to the length of the scale limited to only five points. If the scale was longer , this could perhaps lessen the problem, but it would require respondents to use the numerical scale consistently. It is also possible that a limited range of total scores for each motive, in our version including only three statements, caused the poor distinguishability of the scale. If so, increasing the number of statements could lead to a more nuanced total scores, which perhaps could help establish thresholds for between-motives differences at least for intransitivity cases.Furthermore, it is worth noting that we chose quite strict criteria to define patterns, e.g., one intransitive triad was enough to consider a participant intransitive. We did so based on the unambiguity of such criteria, and because there were no other established rules. But we recognize the need for more elaborated grounds for these kinds of decisions and further research in the area of ordinal judgments.All in all, this study should be viewed as a part of a critical discussion on the transition to quantitative scales as supposedly more informative . In factThe raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.YT conceived the presented idea, developed the research design, and wrote the manuscript. YS and AK developed the research design, obtained and analyzed the data, and wrote the manuscript. ML conducted data collection and wrote a part of the manuscript. All authors contributed to the article and approved the submitted version.This work is an output of a research project implemented as part of the Basic Research Program at the National Research University Higher School of Economics (HSE University).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Managing learning continuity is critical for successful MOOC learning. Thus, enabling learners to have learning persistence needs to be integrated into the MOOC learning design. Motivation effort is a critical component enabling students to maintain continuous MOOC learning. The expectancy–value theory explains why learners engage in learning: (1) they have a higher perceived ability for learning success, (2) place value on learning, and (3) avoid psychological costs. However, it is unclear how these factors affect MOOC learning persistence and how learners’ motivation is formed from this perspective. This experimental study explored how learners’ motivational variables affect their learning persistence, focusing on the expectancy–value theory. The results of this study indicated that academic self-efficacy and task value had significant positive effects on learning persistence. The structural relationship of antecedent, process, and outcome variables showed that teaching presence as an antecedent had a significantly positive effect on academic self-efficacy and task value. Among the three factors of the expectancy–value theory, only the task value influenced learning persistence through student engagement as a mediator. Based on the results, suggestions are provided for motivating MOOC environments that support learners’ continuous MOOC learning. Maintaining learning continuity significantly affects learners’ cognitive, emotional, and behavioral engagement and promotes successful learning. Particularly, in the massive open online courses (MOOCs)—where lectures are started and maintained solely by learners—continuity is a critical indicator in determining successful learning compared to ordinary school education, which is compulsory . In thisGiven that 90% of MOOC students experience a dropout, many efforts are being conducted to solve this problem . Early MIn addition, to enable continuous learning and learning success, studies have examined a variety of variables, such as lesson designs that apply learning analysis , learnerIn order to understand the learning choices and persistence of MOOC learners, the expectancy–value theory examines how learners comprehensively recognize the “expectancy” that they can successfully complete the learning even in an environment separate from the instructor and the “value” and “cost” of the task. For instance, However, most of the studies examining MOOCs from the perspective of expectancy–value theory emphasize only one factor—self-efficacy or task value . FurtherYo reveal the mechanism of learning motivation on learning persistence and to provide design implications that can be applied to instructors or platform managers, beyond the simple influence relationship between the variables, it is necessary to explore the structural relationship among the variables related to the expectancy–value theory. In this regard, From this perspective, the teacher, parents, fellow learners, and the cultural environment could be considered antecedents affecting learners’ motivation . In partMoreover, a process variable should be identified. Therefore, this study aims to systematically examine the structural relationship between variables that affect learning persistence in the MOOC environment, focusing on academic self-efficacy, task value, and cost, which are sub-factors of expectancy–value theory. To this end, we selected teaching presence as an environmental variable that affects the learners’ expected value, student engagement as a process variable when learners’ motivation leads to learning outcome, and learning persistence as an outcome variable.Learning persistence comprehensively represents learners’ motivation, emotion, cognition, and behavioral factors , and it The expectancy–value theory is a motivational theory in that individuals’ expectancy for success and value for tasks is critical to predicting future decisions, participation, continuity, and achievement, and it is primarily composed of two sub-factors: “expectancy” and “task value” . AcademiThe task value corresponds to the subjective value that the learner has on the task, and can be classified into four factors depending on its characteristics: attainment value, intrinsic value, utility value, and cost . SeveralCost is a perception of the negative aspects of the task , and hasTeaching presence is defined as “the teacher designs, promotes, and guides learners’ cognitive and social processes so that individual learners can realize teachers’ meaningful efforts in improving learning effects” ; TeachinStudent engagement refers to “a state in which learners actively concentrate and continuously participate in learning activities to achieve their goals in a learning environment” . In sevederived research questions by reviewing the theoretical background and previous studies and established a research model as shown in In this study, to understand the structural relationship among the sub-variables of the expectancy–value theory that affect learning persistence in the MOOC environment, we 1–1, H1–2), but the cost will have a significant negative effect on learning persistence (H1–3). (2) Teaching presence will affect academic self-efficacy, task value, cost, and learning persistence. Specifically, teaching presence will have a significant positive effect on academic self-efficacy (H2–1) and task value (H2–2), but have a significant negative effect on the cost (H2–3), and teaching presence will affect learning persistence through academic self-efficacy, task value, and cost as mediators (H2–4). (3) The student engagement mediates between academic self-efficacy, task value, cost, and learning persistence. Specifically, academic self-efficacy and task value will have a significant positive effect on student engagement , but the cost will have a significant negative effect on student engagement (H3–3), and the student engagement will multi-mediate between teaching persistence, academic self-efficacy, task value, cost, and learning persistence (H3–4).This study reviewed the theoretical background and previous studies to understand the structural relationship among the sub-variables of the expectancy–value theory that affect learning persistence in the MOOC environment and established a research model as shown in An online survey was conducted on students from two courses loaded on a Korean-MOOC (K-MOOC) in 2020. Among the 298 questionnaires retrieved, 277 responses were finally used for the analysis, excluding duplicate participation, non-response, and identical responses. The K-MOOC course sampled by the research context consisted of 14 weeks of lectures covering the field of education and was provided free of charge to the general public on the K-MOOC platform. The courses consist of 3–5 lecture videos of 10–20 min per week and learning activities, such as quizzes, discussions, or reports. To issue a certificate of completion, learners must achieve a certain level of scores through quizzes and participate in discussions or writing reports; however, this was not linked to their college credit.The courses are operated as a regular 14-week course  rather than a full-time system (regular), and the instructor and tutor interact with the learner, while the lecture is in operation. Specifically, when the weekly lecture starts, the instructor emails the learner a greeting and a guide about the class content for the week. In addition, the tutor participates in the discussion to promote it so that learners are encouraged to provide varied opinions. When learners ask a question, the instructor and tutor answer within 24 h, and feedback is provided for the submitted assignment.A questionnaire was constructed as a measurement tool for teaching presence, academic self-efficacy, task value, cost, student engagement, and learning persistence, verified through previous studies. The survey was modified and adapted to the MOOC environment based on the advice of experts in educational technology, and the reliability and validity of the tool were verified based on the results of confirmatory factor analysis and exploratory factor analysis.research context. This instrument consists of eight items on a six-point Likert-type scale, and the sample items are “I took notes while studying the MOOC” for behavioral engagement; “I was inspired to expand my knowledge in the MOOC” for emotional engagement; “When I had trouble understanding a concept or an example, I went over it again until I understood it” for cognitive engagement. For learning persistence, we modified and used the tool developed by To measure the teaching presence, we modified and used a tool developed by This study adopts a quantitative cross-sectional research design to examine the relationships between the variables that affect the learning persistence of MOOC learners . The maip < 0.05) to confirm the correlation between the variables.First, frequency analysis was performed to confirm the demographic characteristics of the survey respondents. To test the normality of the data, we conducted a descriptive statistical analysis and examined the mean, standard deviation ≥ 0.150, , skewnesThen, we conducted verification of the construct validity of the measurement tool through exploratory factor analysis , SRMR (≤ 0.08), RMSEA (≤ 0.08), reliability (Cronbach α ≥ 0.60) ].Next, we examined the fit indices, such as χ ≥ 0.60) , and con ≥ 0.60)  and discp < 0.05) for the direct, indirect, and total effect by setting bias-corrected 95% confidence interval for the estimates derived through a total of 5,000 bootstrapping, and the indirect effect of individual pathways in the dual mediation through the phantom model . Convergent validity can be verified through factor loading, average variance extracted (AVE), and overall reliability. The analysis showed that the standardized factor loading for each variable is 0.534–0.953 ≥ 0.50, , the AVEB = 0.044, β = 0.035, p > 0.5) and cost  did not have statistically significant effects on the student engagement. So, we modified the model to construct a simpler one by deleting two direct paths that are not statistically significant from the initial structural model , and by proving that the initial model and modified model do not show a statistically significant difference .Regarding the structural model designed in this study, most goodness of fit criteria were satisfied , so we decided that this structural model is plausible. However, according to the result of parameter estimation on the influential relationship between the latent variables, it was found that academic self-efficacy  was found to be significant, and that of teaching presence on student engagement  and learning persistence  were also found to be significant. The goodness of fit of the phantom model for the individual path analysis between teaching presence and learning persistence was consistent with that of the final structural model . Additionally, as shown in 2–4 while partially supporting H3–4.According to the results of mediating effect analysis, as shown in This study explored the structural relationships between variables that affect learners’ persistence for MOOC learning from expectancy–value theory. To this end, we selected teaching presence as an environmental variable that affects the learners’ expected value, student engagement as a process variable when learners’ motivation leads to learning outcome, and learning persistence as an outcome variable. Findings show the following points:First, the MOOC learning environment, learners’ academic self-efficacy, task value, and cost significantly affected learning persistence. The positive effect of academic self-efficacy on learning persistence means that the academic self-efficacy of learners can increase the intention to continue learning by enabling learners to gain self-confidence in the MOOC learning environment. Therefore, instructors should use instructional design strategies to promote learners’ academic self-efficacy, and self-regulated learning strategies can be one way of doing this . FurtherNext, the positive effect of task value on learning persistence is related to learners’ motivation to take the MOOCs . In otheInterestingly, unlike previous studies, this study focused on the cost perceived by the MOOCs and confirmed that the cost had a negative effect on the learning persistence, which was related to the dropout of the MOOC learners. This shows that the demand for excessive effort and time was the reasons for the dropout , which mSecond, teaching presence as an antecedent variable had a significant effect on learners’ academic self-efficacy, task value, and cost; through these, it also had a significant effect on learning persistence. It means that, when learners feel that they can receive appropriate help from the instructors, learners earn confidence to do well in the MOOC learning process, and feel less pressure from the amount of effort they need to put into the MOOC learning process. The significant effect of teaching presence can be explained by the high perception of teachers’ roles and preparation levels in this study. MOOC teachers in this study played the role of designers, developers, and guides. They also played fellow learners during discussions between learners. Both the instructor and tutors in this study have been teaching the course for the past 3 years. To promote a teaching presence, teachers’ active interaction and professional class preparation must be encouraged. In addition, their electronic teaching skills should be improved. Teachers often face difficulties in MOOC development due to a lack of teaching experience in an online environment or technical difficulties, such as filming and editing . FurtherThird, student engagement as a process variable mediated only the relationship between task value and learning persistence among academic self-efficacy, task value, and cost in the MOOC environment. It means that, the more the learners feel the MOOC learning process is important and useful, the more time and money they invest in the task, resulting in behavioral, emotional, and cognitive engagement . EspeciaTherefore, the establishment of interactive, inferential, integrative, and involving instructional design strategies that promote student engagement is required . MoreoveUnlike the previous studies , academiManaging learning continuity is critical for successful MOOC learning. Thus, motivation efforts that enable learners to have learning persistence need to be integrated into the MOOC learning design. In this study, to explore learners’ motivation for MOOC learning persistence, the environmental, motivation, and process variables of MOOC learners were selected from the expectancy–value theory. Previous studies on MOOC learning from the viewpoint of motivation mainly focused on learners’ psychological characteristics, such as self-determination theory  or self-In addition, previous studies that have mainly analyzed the relationship between the expectancy–value variable and the learning outcome variable were only fragmentary. This approach has limitations in understanding and facilitating learners’ motivational mechanisms, so it is necessary to look at the motivational development process from an integrative point of view . This stThrough this study, the importance of teaching presence was highlighted as an antecedent variable affecting the expectancy, value, and cost for MOOC learners. This provides implications for the theoretical foundation and development direction of AI tutor or conversational agent systems related to providing customized feedback on the learners’ responses , or the In addition, student engagement as a process variable mediated only the relationship between task value and learning persistence in this study. So, to maintain continuous MOOC learning in MOOC learning design, it is necessary to confirm if learners’ perceptions of task value sustain interest and if their engagement is provided in the MOOC learning environment.This study has great significance in that we derived practical implications for the development of prescriptive teaching and learning strategies for the formation and expression of the expectancy–value from a holistic view on the process of motivational development. This suggests that to increase the MOOC learners’ learning persistence, it is important to establish an instructional design, create a learning environment, and build a system that comprehensively considers the learners’ teaching presence, academic self-efficacy, task value, cost, and student engagement.Our study has limitations. First, in the case of MOOC courses operated in Korea (K-MOOC), there is a tendency for MOOC certificates not to be recognized as work experience, as most courses are provided for free. Conversely, in other countries, MOOC completion is recognized in careers when getting a job or promotion, despite MOOC courses charging a fee in some cases. Depending on these countries and cultures, the value and cost perceived by learners of MOOC courses may have different effects on student engagement and learning persistence.With respect to the scope for research variables, this study examined the effect of motivation variables on learning persistence from an integrated perspective. However, it is hard to understand the learners’ motivation and engagement in detail in this way. Therefore, in a smaller scope, it is necessary to select each of the sub-factors constituting an individual variable as one potential variable , examine the influential relationship between the variables, cluster according to the level of differences between the variables, and finally explore the characteristics of the group and the differences from other groups. Through this, it will be possible to provide basic data for the provision of customized learning diagnosis and instructional treatment for the learners by understanding learners’ motivation and the level of engagement in-depth and classifying the learning types based on the level of expectancy–value.Finally, with respect to measurement and research methods, the variables in this study were limited as they were measured through a self-reporting questionnaire. Considering that all the learning processes of learners are stored as big data on the platform due to the nature of the MOOC environment, learning analytics research such as log data analysis, text mining, and social networking analysis can be conducted. Through this, it can be possible to analyze the learning participation patterns of the MOOC learners and provide basic data for the development of a learning management system based on user interface and AI-applied learning tutors.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.


The emerging field of human–computer interaction (HCI) opens up more opportunities for technology integration in language learning. Technology creates more workforces in the Science, Technology, Engineering, Mathematics (STEM) field, yet the number of STEM pursuers is declining due to poor command of the English language. There is a gap in English vocabulary for STEM, which needs a novel solution from the perspective of HCI. This study aims to explore the needs of STEM learners in creating an English language competency mobile module. The methodology used is through a qualitative study. Seven STEM learners, 17 years old, participated in semi-structured interviews. The results from the interview are divided into four main themes: (1) the importance of learning English, (2) problems of learners, (3) strategies of English language learning, and (4) learners' readiness in using a mobile app. This study is significant for mobile app designers, English language teachers, and course designers as the input will provide an overview of the STEM learners' needs in the English language. Plus, designing a mobile app from the learners' perspectives gives more effectiveness to HCI, rendering success to second language acquisition. Future work can design and develop a mobile app to enhance STEM learners' English language competency based on the learners' perspectives. Technology creates more workforces in the Science, Technology, Engineering, Mathematics (STEM) field. The STEM field is vast, yet the number of STEM pursuers is declining due to poor command of the English language among non-native speakers  was pioneered by Professor Ben Shneiderman and has since evolved with the industrial revolution, providing more fresh ideas for its integration  importance of English, (2) problems in learning English, (3) learning strategies used in learning English, and (4) mobile readiness in learning English. Only one researcher took part in this interview session to ensure that learners did not feel pressured by the presence of many researchers. The researcher also took field notes to note on participants' non-verbal behavior. Researchers did not use any template for the field notes.Data from the interview transcripts were analyzed using thematic analysis. The transcripts were analyzed using the Atlas.ti software. The process included six steps by Braun and Clarke . The proFrom the analysis, eleven subthemes emerged, categorized into four main themes: (1) Importance of English, (2) Problems in learning English, (3) Strategies in learning English, and (4) Mobile readiness in learning English. Why do you think English is important for you as a secondary school student?Question: in university, everything is in English.” Participant 1 mentioned, “for me, learning English is easy if we start from young or maybe we keep learning and learning, like that.” These perceptions showed that the participants had been exposed to learning English since they were young, perceiving that English is important for education.The researchers asked participants why English is essential to identify the learners' view of learning English. Across the responses, education is the most crucial reason for learning English. English is used in many subjects of education, especially Science and Mathematics. Also, in tertiary education, most STEM courses are taught in English, making the participants aware of the importance of English. The response from Participant 6 mentioned, “, “we can talk with our idols” (Participant 5). Another response mentioned, “easy to communicate with other people, can speak easily” (Participant 4). These responses referred to the participants' perception of learning English to communicate with other countries. One participant said, “we will not be easily deceived by foreign people” (Participant 5). These responses showed that learners in high school might have the thought of going abroad; hence learning English seemed to be important to them for communication.Based on the responses, a few participants mentioned that learning English for communication is essential, which many learners value. Five participants said that English is vital for communication. When prompted to, one participant mentionedEnglish is the language people always use. It's wide and wherever we go, other than our own language, Bahasa Melayu, we have to have side language. The most common is English” (Participant 1). In enhancement, Participant 2 also quoted the word lingua franca in his response, “The English language is one of the lingua franca languages, and almost all country in the world know English as a communicating language… Even though English is not our mother tongue language, but we have to learn it because English is also one of the lingua franca languages” (Participant 2). Additionally, Participant 1 added, “and when we go anywhere, it's not only our language, but we have to learn English so that it's easy”. Looking into these responses, the notion that English is an international language becomes one reason for learning English.Another importance of learning English is because English is an international language. One of the participants mentioned, “What are the difficulties or challenges that you've faced in learning English?Question: they are weak in vocab and grammar. Vocab is important because we want to write essays and all that, we use vocabs” (Participant 7). Another response noted, “the problem of grammar, grammar. Lack of vocabulary” (Participant 3). Most of the responses revolved around grammar and vocabulary. The researcher followed up with prompt questions to know why they said grammar and vocabulary are difficult, and one response mentioned, “because if we don't understand the vocab, we won't understand” (Participant 1). Language related problems that learners face are related to grammar and vocabulary. They know that they would not acquire other language skills without vocabulary.The researcher then asked the second question to know the learners' problems in learning English. One of the problems was related to language difficulties. One response highlighted, “sometimes when they try to speak English, people will make fun of them, like judgmental society” (Participant 6). Another response highlighted that “didn't get support from friends, maybe family” (Participant 4). These participants felt anxious when they did not have the support from their surroundings to encourage them in using English. Another response gave an example of anxiety that he felt when he spoke English, “but for speaking, you need to have courage. It must come from your own self. No one can help in courage own effort. In speaking, especially in presentation, it is difficult to improve in a group speaking. Because sometimes, someone will speak more and some will not speak at all. It's not about group. Also, to build confidence, we present alone, but only four or five people watch. It's not like we present many members and many people watch also, the courage is difficult to build” (Participant 7).The next problem related to English language learning is the learners themselves. The barrier is often associated with learners' shyness and lack of confidence in using English, despite having learnt it since young. Also, they are afraid of being laughed at; hence they refrain from using English. One participant mentioned, “How do you learn English? What are the strategies that you use to learn English?Question: via texts emerged as the subthemes, only two participants mentioned it. Regardless of that, the researcher believed it is an exciting finding that should not be excluded from the report. One response mentioned, “use Thesaurus” (Participant 4). Hence, the interviewer probed the answer:Third, the researcher asked what strategies the participants used in learning English. Though learning Interviewer: Could you explain how you use Thesaurus to improve your English?Participant 4: I find similar words. [long pause]Interviewer: Could you elaborate? Why do you use Thesaurus instead of Google?Participant 4: Google sometimes not good. I have it at home, the Thesaurus. Thesaurus has like synonyms and antonyms and got sample sentences.Reading novels, I like them. Sometimes I read Webtoons. My favorite” (Participant 5). Webtoon is an online platform for comics, mainly from Korean authors, but the comics are officially translated to English. Though participant 4 preferred to use the hard copy version of Thesaurus, Participant 5 mentioned that she even read novels online because it's convenient.Another participant expressed, “Although we [referring to him and his family] watch English movies, subtitles also English” (Participant 6). Generally, participants learned English by watching and learning through the subtitles. Still, one response mentioned, “I always watch movies and anime by reading the subtitles, and if I don't understand it, I would gladly Google translate the word” (Participant 2).Audio–visual materials like videos encourage learners to learn English. Many participants mentioned they watched videos on YouTube and other platforms in the English language. One interesting response said, “, “from games also we can know more words like vocabulary game” (Participant 4). Another response also mentioned a similar answer, “I use game word search also. I get to know more words” (Participant 6). On another note, one response elaborated using video games instead of word games, “and I used to expose to English at a young age since my nephew and uncle play video games so much, I started to play it and started to understand English a little bit” (Participant 2).Another way of learning English, which various participants mentioned, was through games. One response saidvia materials, one participant specifically mentioned that learning English in class with a teacher's guidance helped him improve. He noted, “Other than that, the learning process in class is also important to make sure our grammar, pronounce and literature good and nice. If somehow students out there didn't seem like to give a full attention on classes, maybe they won't be able to improve their English” (Participant 2). Though only one participant mentioned this, it is worth noting as learning English with proper guidance helps.Though participants agreed that they learned How ready are you in mobile learning? What are the contents you require in a mobile application?Question: of course. Now no one wants to hold a heavy dictionary” (Participant 6). Aside from the easiness to use a mobile app, Participant 2 mentioned that people are more attached to gadgets in the current days, which will be beneficial if an app for English is created. He specifically said, “yes, because people nowadays are more attached to mobile phones, laptops, tablets, and PC more than they attached to books. So, it's really helpful for them to have a good app to improve their English” (Participant 2). Additionally, Participant 1 personally mentioned the current pandemic situation, which makes learning through an app more beneficial through a mobile app. She stated that “yes, of course. Because now during pandemic, students do nothing at home. So, for those who want to learn English, think English is difficult and want to improve their English, they can use the English apps” (Participant 1).Finally, the researcher asked participants' readiness in using a mobile app to learn English. Participants are positive toward using the mobile app. All the participants are ready to use a mobile app as a supplementary lesson to improve their English language competency. Some of the reasons are the mobility feature, where learners can learn anywhere, which is easy. One response mentioned, “it's important to put media like audios, videos, graphics, and animations” (Participant 5). Another response also said that using audio–visual materials, “video can, but the important thing is not too long, like a minute okay maybe”(Participant 5). Plus, one participant said, “if we watch video, we can improve vocab” (Participant 7). Aside from audio–visual materials, many participants also mentioned games like quizzes and mini-games. Participants 3 and 7 noted that they are agreed on integrating quizzes in the app.Learning activities are central to improving learners' English language competency through the mobile app. One participant mentioned, “for example, if we play the tap app, understand? When you want to go to the next question or anything next, they'll ask you to tap this and that. Ah, that sounds like fun because it makes us feel like we want to know more, want to know more, like that” (Participant 1). Additionally, Participant 2 also mentioned the fun aspects of a mobile app with mini-games, “…maybe mini-games, so that the app will not be very boring for everyone to use. A little fun would be okay” (Participant 2).Participants 1 and 2 mentioned including mini-games to create a more engaging app. The suggestion from Participant 1 is unique as, she said, “This study aimed to explore the needs of STEM learners in creating the English language competency mobile module, adhering to HCI. The freedom to learn STEM subjects in different languages might cause difficulties for learners in the future Ye, . With reAdditionally, this study showed that many STEM learners mentioned being anxious and shy to speak English. Supporting the findings, a study that looked into the factors affecting English language learners' shyness reported that they had limited knowledge of the language, resulting in anxiety (Ahsan et al., via HCI could embed the social constructivism theory by Vygotsky (Human–computer interaction prioritizes multiple interaction patterns and the openness of the design space. Participants in this study mentioned the use of audio–visual materials. However, one participant also specified the teacher's involvement. Although the focus is on technology in HCI, there is a need to look into human interaction. Therefore, findings from this study clearly showed the importance of HCI to ensure that teachers' presence is also an aspect to be considered when developing an app. In enhancement, app development Vygotsky  to incluVygotsky . HoweverVygotsky . This prvia a mobile app, which can be considered when designing and developing feasible mobile applications.This study was carried out to explore the needs of STEM learners in creating an English language competency mobile module, which looked into specific samples with specific backgrounds. There are limitations to our study. First, there is a limitation in transferability. Also, the number of participants who volunteered was from one class in a public school; hence the sample might not represent the whole population of STEM learners. Despite the limitations, this study has strengths. First, this study is one of the earliest studies to look into the needs of STEM learners in second language learning from the HCI perspective. Second, this study provides evidence that learners have their preferences in learning via HCI.One major significance of this study contributes to the language policy aspect. Many non-native countries teach English as either a second or foreign language, especially in higher education, because English is an international language and learning English is necessary for education, especially for learners to further their education (Wallace and Lee, This study proposed exploring the needs of STEM learners in an English language competency mobile module. The results from the interview implied that STEM and English language rely on each other because language ability is an important aspect. This study also implied the importance of introducing vocabulary to second language learners first before teaching other language skills. This is because vocabulary is the foundation of language learning. Furthermore, findings from this study implied that further research on HCI to assist vocabulary learning is feasible, opening up more opportunities for future research on HCI.This study is significant for mobile app designers, English language teachers and course designers as the findings provide an overview of the STEM learners' needs in the English language. This study looks into learners' perspectives in mobile learning, which gives more effectiveness to HCI, rendering success to second language acquisition. One notable finding from this study, which is the mobile readiness of learners, provides insight into future mobile learning research. This study is only a preliminary study, and future work could look into the HCI aspect in designing and developing a mobile app to enhance STEM learners' English language competency.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the patients/ participants or patients/participants legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.KRMR, HH, and MMY conceived the study, participated in its design and coordination, and performed final analyses and co-drafted the manuscript. KRMR collected field data, entered study data, assisted in data analysis, and interpretation of study results. All authors read, revised, and approved the final manuscript.Universiti Kebangsaan Malaysia funded this research with Grant No. GG-2021-003.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.