# Natural Language Processing Technologies in English as Second Language Learning Applications: A Review

Julio Christian Young ${ }^{1}$ Makoto Shishido ${ }^{2}$
Graduate School of Advanced Science and Technology, Tokyo Denki University
1julio.christian.young[at]_gmail.com ${ }^{2}$ shishido[at]mail.dendai.ac.jp
}

\begin{abstract}
Previous studies have shown how Artificial Intelligence (AI) technology can help the English as Second Language (ESL) learning process. Text to Speech, Speech to Text, and Chatbot are some of them. In this research, the research objective is to show the status, opportunities, and challenges associated with such technologies for ESL learning.
\end{abstract
## 1 Introduction

Having fluent English communication skills increase one's academic and career opportunities. With the popularization of deep neural networks in 2015 [1], artificial intelligence (AI) began to accomplish humanlike performance in various natural language processing tasks. Therefore, its implementation starts to revolutionize how one can learn a language effectively, inclusively English. Chatbot, text to speech (TTS), and speech to text (SST) are AI applicative aspects favored by researchers to achieve such objectives.

For Learning English as a Second Language (ESL), research in $[8,14,15]$ shows that STT technology can create a personalized learning system; therefore, students can practice and evaluate their speaking skills individually. The research found that the resulting system made students feel comfortable as it mitigates the fear of being judged when they practice in a classroom setting. On the other research in [11, 20], researchers found that TTS technology can mitigate the cost to hire a professional for producing listening materials. Moreover, using TTS, teachers can adjust the listening materials according to the class competency. Lastly, research in [17, 26, 27] demonstrated that a chatbot could be a perfect learning partner as it never grows tired and can be accessed according to students' needs.
However, there are still several challenges that have not been fully addressed mentioned by the previous research. For instance, in [16, 17], researchers state a Chatbot's unintelligent behavior might cause a communication breakdown that minimizes students' learning interest. While for STT-based learning applications, researchers argued it requires unusual preparations and only works in the research setting, not in the real-world situation [18]. Lastly, most students in [19] believed their teacher is a better pronunciation model even when they achieved a better score using TTS word dictation system.

Based on the presented benefits and challenges, the focus of this research is to do a literature review to gather all information regarding the opportunity, challenges, and research results related to the application of TTS, STT, and Chatbot technology.

## 2 Research Methodology

In this research, the sequence of activities carried out for the literature review process is described in Fig. 1.

Fig. 1. The Sequence of Activities Conducted in Our

$
\text { Study }
$

In the article gathering step, 61 items were gathered using one from the following keywords: "chatbot", "speech to text", and "text to speech, with two extra keywords of "English" and "learning". The gathering was done using Google Scholar Search Engine for research published from 2017 to 2020. Secondly, the abstract screening step was conducted to ensure that each gathered
article truly presented the potential benefits and challenges of such technologies in English learning. Based on the abstract screening step, there are only 26 remaining articles (5 in TTS, 6 in STT, and 15 in Chatbot domain). Finally, the remaining articles will be read to extract all information under these categories.

1. How they implement their technology of choice to help the learning process.

2. The opportunity assumed.

3. The research results.

4. The presented challenges in the implementation part.

The summarization from all points above will be presented in the next section. For readers' convenience, the summarization results for each technology will be presented in a separate subsection.

## 3 Previous Research

## 3.1 TTS for English Learning

Learning Strategy - Several learning strategies can be incorporated with TTS technology such as Listening comprehension [11, 13], phonological acquisition [12], words dictation [19], and word rhyme \& synonym [20].

Opportunity - Through various implementations, researchers believe there are advantages of TTS for ESL learning, which are.

1. TTS technology mitigates the cost and time to hire professionals for producing listening materials [11, 20].

2. Facilitator has total control of speech rate, loudness, tone, speaker's identity, and delay between a word in the produced materials $[11,12,13]$.

3. TTS service read reads the words at a constant rate and each word is rhythmically segmented [20].

4. Audio files produced by the TTS service have a native-like pronunciation which might enrich students' language input [13].

Results - Research in [12, 20] conclude that both $\theta f$ students who learn with human voices and audio materials from TTS have a similar learning performance; while research in $[11,19]$ conclude that students who learn with TTS assistance perform better learning performance compared to students that learn with the conventional method. On the other hand, qualitative research [13] on teacher perspective in incorporating the TTS technology for ESL learning also yielded a positive result. EFL teachers believed TTS might be useful to create listening materials and the produced material has a similar pronunciation to the native speaker so it might be beneficial to students.

Challenges - Despite its positive impact, there are several challenges for its application, such as.

1. Due to unnatural voice characteristics and emotionless intonation, most participants prefer ESL teachers to TTS technology [19].

2. TTS technology doesn't truly understand the concept of language, so there is no confirmation whether each word is spelled correctly or not [13].

3. Current TTS technology has no intonation to some punctuations (e.g., exclamation or question mark) resulting in wrong intonation to exclamatory and interrogative sentences [13].

## 3.2 STT for ESL Learning

Learning Strategy - In ESL Learning, researchers mainly incorporated STT technology for pronunciationrelated activities which include single vowels pronunciation [21], students' phonetic and fluency evaluation [15, 18], and utterance practice [8, 14, 15]. However, in [22], researchers utilize STT technology to provide real-time transcription of EFL teachers' speech in a classroom.

Opportunity - The previous research stated there are several advantages of STT for ESL learning which are.

1. Mitigate the resources needed for one-on-one practice that involves a native speaker $[8,18]$.

2. Provide a less intimidating and serene practice environment as students often feel uncomfortable, and fear to be judged to practice in a public [15].

3. STT implementation for speech transcription [22] provides dual-modality so it might be useful to fit students' learning styles.

Results - All previous quantitative research shows a positive result on students' perception regarding STTbased learning applications [8, 14, 15, 21, 22]. Most participants in $[8,14,15]$ felt less intimidated in their speaking practices using STT; it fits their learning preferences, and they're willing to use it for in- and outclass learning activities. For the real-time transcription use in [22], most participants highly appreciate STT implementation for improving their listening skills. On
the other end, quantitative research in [18] shows that students who use STT for ESL learning have a better speaking performance compared to others who do not.

Challenges - Regardless, there are several difficulties mentioned from STT implementation for ESL Learning.

1. Small group of students with high language ability finds the visual aspect of STT useless as they processed information directly from speech. [22].

2. STT might affect students' speaking performance as they might be slowing their speech on purpose to accentuate clarity [18].

3. Considering current STT trained by audio data from native speakers, the recognition rate for ESL students is often low due to their utterances being often mistranslated [8].

## 3.3 Chatbot for ESL Learning

For ESL Learning, research in the past utilized chatbots more frequently compared to the other two. Chatbots leverage text or voice modalities to engage in meaningful conversations with students. For chatbot application that interacts through users' utterances, STT and TTS technologies become the two underlying components to process the input and output voice in it. Therefore, the successful application of a voice-based chatbot for ESL learning simply reflects the success of the other two technologies within.

Learning Strategy - Researchers primarily implement chatbots as learning tools by making them learners' conversation partners. While researchers in $[2,9,26,5,4]$ implement a chatbot as a free conversational agent that engages in a diverse and broad conversation, others try to use it to aim at a specific learning objective. For example, researchers in [10] utilized a chatbot to assist students in a writing practice; while in [23], researchers created a chatbot that assists students in their writing activities by finding spelling and grammatical errors in their writing; in other research [24], the chatbot acts as students' virtual reading partner that recommends appropriate readings based on their level and evaluates their reading comprehension.

Opportunity - Researchers in the past considered the following advantages when they integrated a chatbot into ESL learning.
1. Students learn better when words are presented in spoken forms with a conversational style in which a chatbot is capable of [25].

2. Chatbot has a huge potential to help countries with an insufficient number of English native speakers [25, 9].

3. Interacting with a chatbot creates a pleasant learning experience for students $[3,6,26,27]$. A chatbot voluntarily engages in a conversation with students anytime and anywhere [25, 26, 17, 9, 6]. For ESL students with low language and speaking skills, in contrast to a human partner, a Chabot never grows tired and impatient even when a student keeps repeating the same mistake $[26,17,27]$.

4. Learning with a chatbot allows students to dictate the flow of the conversation to present learning materials according to their needs $[3,6]$.

5. Through multimodality implementation, a chatbot allows students to practice all skills in the English language (speaking, listening, reading, and writing). Moreover, it provides the opportunity for students to make use of diverse vocabulary and sentences structures that they scarcely use [26, 17, 27].

Results - The qualitative research result in the past shows a mix between the positive and negative impact of chatbot usage in ESL learning. For the positive part, most participants believed chatbot is beneficial for language learning [10, 16]. Most participants also agreed that a chatbot can hold a native-like conversation that feels authentic [10]. Moreover, in [9], after 10 minutes of weekly engagement with a chatbot in 8 weeks period, students felt more motivated, interested, and confident for their English skills. Participants from various research in $[24,3,27]$ perceived chatbot as a better learning partner (more patient, non-judgmental, and more relaxing) compared to the human study partner.

However, research in [4] shows that when students were engaged in a chat session for long enough, they began to lose interest in the chatbot thus leading to a significant decrease in their motivation. Secondly, the previous research [23] finds students feel a conversation with a chatbot was dull as its response are is unvaried. On top of that, chatbot inappropriate and unnatural responses could also lead to a bland conversation that decreases their learning motivation.

On the other hand, most quantitative research showed a positive result of a chatbot on improving students' English performance. In the previous research, after a short term (a week) [7, 10] and a long term (eight weeks) [9] of engagement with a chatbot, students showed a significant vocabulary test score improvement. Moreover, a study in [7] also compared a chatbot and a traditional Listeningand-Repeat (LRE) learning system, its result also shows that students engage 2.1 times longer with a chatbot system compared to the traditional LRE. Other previous research $[16,4]$ that measure the effects of chatbot usage on English writing skills found a significant improvement in students' words production and grammar skills. Lastly, in [2], researchers found that students who incorporate chatbots into their learning process achieve a better TOEIC Listening and Reading test score compared to others who do not.

Challenges - The previous research showed there are several challenges worth considering when chatbot technology is used for ESL learning.

1. Students might find the conversation with a chatbot is dull as it often gives monotonous responses [16, 17]. On top of that, the inappropriate responses from chatbots can lead students to frustration thus demotivating them [17].

2. Unintellectual nature of chatbot often makes its engagement with students short [17].

3. As a conversation partner, chatbots are often programmed for a specific and limited range of topics that don't suit the learners' needs and interests [10, 25].

## 4 Conclusion

In the last four years, most research showed the practicality and promising result of STT, $\mathrm{TTS}_{2}$ and Chatbot for ESL learning. Through the assistance of such technology, teachers can provide a great number of materials and exercises for ESL students. Furthermore, they can also allow students to engage in genuine communication despite their unideal learning environment. However, researchers in [4] argued that the novelty effects might be the main factor of their successful result so in the long term they will not truly contribute to students' English skills. Therefore, to help students in their learning process, it is better to view such technology as a supportive tool that aimed at a specific learning objective and designed them to fit that.

## References

[1] LeCun, Y., Bengio, Y., \& Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444.

[2] Kim, N. Y. (2018). A study on chatbots for developing Korean college students' English listening and reading skills. Journal of Digital Convergence, 16(8), 19-26.

[3] Shin, D., Kim, H., Lee, J. H., \& Yang, H. (2021). Exploring the Use of An Artificial Intelligence Chatbot as Second Language Conversation Partners. Korean Journal of English Language and Linguistics, 21, 375-391.

[4] Kim, N. Y. (2019). A study on the use of artificial intelligence chatbots for improving English grammar skills. Journal of Digital Convergence, 17(8), 37-4.

[5] Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., \& Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461-468.

[6] Nghi, T. T., \& Phuc, T. H. Bots as Foreign Language Learning Tools: From Theory To Practice.

[7] Ruan, S., Jiang, L., Xu, Q., Liu, Z., Davis, G. M., Brunskill, E., \& Landay, J. A. (2021, April). EnglishBot: An AI-Powered Conversational System for Second Language Learning. In 26th International Conference on Intelligent User Interfaces (pp. 434444).

[8] Shishido, M. (2018, June). Developing e-learning system for English conversation practice using speech recognition and artificial intelligence. In EdMedia+ Innovate Learning (pp. 226-231). Association for the Advancement of Computing in Education.

[9] Kim, N. Y. (2018). Chatbots and Korean EFL students' English vocabulary learning. Journal of Digital Convergence, 16(2), 1-7.

[10] Alsadoon, R. (2021). Chatting with AI Bot: Vocabulary Learning Assistant for Saudi EFL Learners. English Language Teaching, 14(6), 135157.

[11] Andriani, T. N., Herawati, Y. W., \& Sulistyo, T. (2020). Text-to-Speech Application for Foreign Language Learner' Listening Comprehension in Indonesia.

[12] Cardoso, W. (2018). Learning L2 pronunciation with a text-to-speech synthesizer. Future-proof CALL: language learning as exploration and encounters-short papers from EUROCALL, 16-21.

[13] Oktalia, D., \& Drajati, N. (2018). English teachers' perceptions of text to speech software and Google site in an EFL Classroom: What English teachers really think and know. International Journal of Education and Development using ICT, 14(3).

[14] Shishido, M. (2019, June). Evaluating e-learning system for English conversation practice with speech recognition and future development using AI. In EdMedia+ Innovate Learning (pp. 213-218). Association for the Advancement of Computing in Education.

[15] Shishido M. (2021). Developing and Evaluating the e-learning Material for Speaking Practice with the Latest AI Technology ISSN: 2189-1036 - The IAFOR International Conference on Education Hawaii 2021 Official Conference Proceedings.

[16] Bailey, D., Southam, A., \& Costley, J. (2021). Digital storytelling with chatbots: Mapping L2 participation and perception patterns. Interactive Technology and Smart Education.

[17] Haristiani, N. (2019, November). Artificial Intelligence (AI) chatbot as language learning medium: An inquiry. In Journal of Physics: Conference Series (Vol. 1387, No. 1, p. 012020). IOP Publishing.

[18] Moxon, S. (2021). Exploring the Effects of Automated Pronunciation Evaluation on L2 Students in Thailand. IAFOR Journal of Education, $9(3)$.
[19] Chiang, H. H. (2019). A Comparison between Teacher-Led and Online Text-to-Speech Dictation for Students' Vocabulary Performance. English Language Teaching, 12(3), 77-93.

[20] Matsuda, N. (2017). Evidence of effects of text-tospeech synthetic speech to improve second language learning. Jacet journal, 61, 149-164.

[21] Tejedor-Garcia, C., Escudero-Mancebo, D., Cardenoso-Payo, V., \& Gonzalez-Ferreras, C. (2020). Using challenges to enhance a learning game for pronunciation training of English as a second language. IEEE Access, 8, 74250-74266.

[22] Hwang, W. Y., Li, Y. H., \& Shadiev, R. (2018). Exploring effects of discussion on visual attention, learning performance, and perceptions of students learning with STR-support. Computers \& Education, 116, 225-236.

[23] Nakano, A., Theeramunkong, T., \& Taira, H. A Conversational Agent for English Writing Skill Learners.

[24] Ruan, S., Willis, A., Xu, Q., Davis, G. M., Jiang, L., Brunskill, E., \& Landay, J. A. (2019, June). Bookbuddy: Turning digital materials into interactive foreign language lessons through a voice chatbot. In Proceedings of the Sixth ACM Conference on Learning@ Scale (pp. 1-4).

[25] Fryer, L., Coniam, D., Carpenter, R., \& Lăpuşneanu, D. (2020). Bots for language learning now: Current and future directions.

[26] Han, D. E. (2020). The Effects of Voice-based AI Chatbots on Korean EFL Middle School Students' Speaking Competence and Affective Domains.

[27] Yin, Q., \& Satar, M. (2020). English as a foreign language learner interaction with chatbots: Negotiation for meaning. International Online Journal of Education and Teaching.
 Julio Christian Young ${ }^{1}$ Makoto Shishido ${ }^{2}$ Graduate School of Advanced Science and Technology, Tokyo Denki University 1julio.christian.young[at]_gmail.com ${ }^{2}$ shishido[at]mail.dendai.ac.jp }  \begin{abstract} Previous studies have shown how Artificial Intelligence (AI) technology can help the English as Second Language (ESL) learning process. Text to Speech, Speech to Text, and Chatbot are some of them. In this research, the research objective is to show the status, opportunities, and challenges associated with such technologies for ESL learning. \end{abstract 