# 訂正文の流暢性向上を目的とした 系列タグ付け文法誤り訂正器の強化学習手法

五藤巧 ${ }^{1}$ 渡辺太郎 ${ }^{1}$
1 奈良先端科学技術大学院大学
\{goto.takumi.gv7, taro\}@is.naist.jp

## 概要

テキストに対して編集のタグ付けを行う文法誤り 訂正器である GECToR は, 出力の説明性が高い一方 で訂正文の流暢性が低い問題がある。この問題の原因として，GECToR がタグの情報のみから最適化さ れており，推定した訂正文の情報を考慮できていな いことが考えられる. 本研究では，訂正文の流暢性 を考慮してタグ推定の最適化を行うために，GPT-2 言語モデルが与える文の perplexityを報酬とした強化学習を行なった. 実験の結果，訂正性能を維持し つつ，訂正文の流暢性が系列変換モデルと同等もし くは上回ることを示した.

## 1 はじめに

文法誤り訂正は，入力文に含まれる文法誤りや表記誤りを訂正するタスクである。近年では系列タグ 付けモデルに基づいた，推論が高速で説明性が高 いモデルが提案されている. 同モデルでは，入力 の各トークンについて編集操作を示すタグを推定 し，タグに従って後処理として入力文を編集する ことで訂正文を獲得する．代表的なモデルである GECToR [1] は，無編集・置換・挿入・削除を示す基本的なタグに加えて，「動詞を過去形に」といった 言語情報を考慮したタグを推定する．従来の系列変換モデルに基づく訂正手法が訂正文を直接推定す ることと比較して，タグの情報が直接ユーザへの フィードバックとして使用できるため有用である.

一方で，GECToR の訂正文は流暢性が低いことが 問題である. 詳細は 2 節で述べるが，系列変換モデ ルとの比較を行った結果, 訂正性能が同等であるに もかかわらず GECToR の訂正文の流暢性は低いこ とが明らかになった. Sakaguchi ら [2] によって流暢性の重要性が指摘されていることや，流暢性の低 さは GECToR の実応用の幅を狭めることを考慮す表 1 最小限の訂正と流暢な訂正の例。誤り文における誤 り単語を赤字で，訂正文において訂正された単語を青字 で示した.

 \\

ると，改善されることが望ましい。ここで，表 1 に Napoles ら [3] に倣い流暢性の高い訂正例を挙げる.最小限の訂正では冠詞 the の挿入と，主語動詞の一致の訂正（affects $\rightarrow$ affect）が起こっている。一方，流暢な訂正では動詞の語彙選択に関する訂正 (affect $\rightarrow$ cause）と，それに伴う語順の入れ替えによって大 きな書き換えが起こっている. 本研究の大局的な目的は，後者のような流暢な訂正の性能向上である.

本研究は，GECToR の流暢性が低い原因は，最適化がタグ情報のみから行われており，得られた訂正文の情報を考慮していないことにあるという仮説に 基づいている．そこで，言語モデルの生成確率に基 づいて流暢性の観点から訂正文の報酬を計算し，報酬を強化学習のアルゴリズムを用いてタグの最適化 に反映する学習手法を提案する。

## 2 GECToR の流暢性

流暢性に関して，GECToR と系列変換モデルの 比較実験を行った. GECToR には RoBERTa [4] ベー スの訓練済みモデルを用いる。また，推論時のパ ラメータである無編集タグへのバイアスと文レべ ルの誤り検出確率の間値は共に 0 とした。これら の値を小さくすることで訂正が積極的になり，よ り大きな書き換えを必要とする流暢な訂正が行 えると考えられる。系列変換モデルには，Kiyono ら [5] と Kaneko ら [6] のモデルを用いる. Kiyono

らは，Transformer [7] 系列変換モデルを大量の擬似誤りで事前学習する点が特徴である. Kaneko らも Transformer 系列変換モデルであり，エンコーダに より得られた表現に加えて BERT [8] から得られた 表現も加えて学習する。なお，いずれも著者らが 公開する訓練済みモデルを用いて1)，ビーム幅を 5 として生成した. これらの 3 種類のモデルは, 訂正性能そのものが乘離しないように，最小限の訂正性能が近いものを恣意的に選択した. 具体的に は，CoNLL-2014 ベンチマーク [9]における M M ス コアラ [10]の $\mathrm{F}_{0.5}$ は GECToR が 62.43, Kiyono らが 62.03, Kaneko らが 62.77 である.

流暢性の評価に用いるデータセットには， JFLEG [3] の開発データを用いる. JFLEG は流暢な 訂正性能を評価するためのデータセットである．複数の観点から流暢性を評価するため，評価尺度に は GLEU [11, 12]，SOME [13]，および GPT-2 [14] の perplexity の 3 種類を用いる. GLEU は n-gram の一致率に基づく JFLEG の標準的な参照あり評価尺度 である. SOME は文法誤り訂正の参照なし評価尺度 の一つであり，訂正文の人手評価に直接最適化する ように評価器を学習する. SOME は複数の観点から 評価するが，本研究では流暢性の評価結果のみを報告する. 最後に, Asano ら [15] が訂正文の流暢性を RNN 言語モデルを用いて計算したことを動機とし， より高性能な言語モデルとして GPT-2を用いて評価 する。評価データの訂正文それぞれについてトーク ン数で正規化した perplexity を独立に算出し，その 平均値を報告する。

実験結果を表 2 に示す. 表 2 の編集率は，入力文 と出力文の単語レベルの編集距離を, 入力文の単語数で割った値である. 表 2 の結果は，最小限の訂正性能は同等であるにもかかわらず，GECToR は系列変換モデルよりも訂正文の流暢性が低いことを示唆 する．また，系列変換モデルと同程度の編集率であ ることから，十分に書き換えを行なっているのにも かかわらず訂正文の流暢性が低いことが分かる.

## 3 流暢性を報酬とした強化学習

本研究では, 流暢性が低い原因は, GECToR の最適化がタグの情報のみから行われており，推定した タグを用いて編集することで得られる訂正文の情報

1) Kiyono らのモデルにはpretlarge+SSE (finetuned)を，Kaneko らの モデルにはhttps://drive.google.com/drive/folders/1h_ r46EswcT1q75qwje6h6yJp0xzAG8gP?usp=sharing を用いた.表 2 GECToR と系列変換モデルの比較結果

JFLEG-dev

を考慮できていないことにあると考える。したがっ て，訂正文を流暢性の観点から考慮した最適化を行 うための手法を提案する。

## 3.1 定式化

GECToR はトークン単位のタグ付けと誤り検出をマルチタスク問題として最適化する. 入力文を $\boldsymbol{x}=\left(x_{1}, x_{2}, \ldots\right), \boldsymbol{x}$ に対する正解のタグ列を $\boldsymbol{y}^{t}=\left(y_{1}^{t}, y_{2}^{t}, \ldots\right)$ および正解の誤り検出ラベルを $\boldsymbol{y}^{d}=\left(y_{1}^{d}, y_{2}^{d}, \ldots\right)$ とおく. ここで, $|\boldsymbol{x}|=\left|\boldsymbol{y}^{t}\right|=\left|\boldsymbol{y}^{d}\right|$ で あり，事前に定義したタグの集合を $y$ として $y^{t} \in \mathcal{y}$ である. また誤り検出ラベルは $y^{d} \in\{0,1\}$ であり， 1 のとき誤りであることを示す. GECToR は $x$ を入力したとき，何らかのエンコーダ（BERT [8] など） により獲得した $\boldsymbol{x}$ の表現ベクトルを，2 種類の線形層にそれぞれ独立に入力し，タグの推定確率および 誤り検出確率を計算する。この処理をタグの推定関数 $P^{t}(\cdot)$ および誤り検出確率の推定関数 $P^{d}(\cdot)$ と定義 するとき，従来の損失関数 $\mathscr{L}_{m l e}$ は次式で表される.

$
\mathscr{L}_{m l e}=-\sum_{i=1}^{|\boldsymbol{x}|}\left(\log P^{t}\left(y_{i}^{t} \mid x_{i}, \boldsymbol{x}\right)+\log P^{d}\left(y_{i}^{d} \mid x_{i}, \boldsymbol{x}\right)\right)
$

一方で，本研究では訂正文から計算される報酬を 考慮するため，貪欲に生成した系列に対する損失を 考える。まず $x \in \boldsymbol{x}$ について，貪欲に推定したタグ $\hat{y}^{t} \in \mathcal{y}$ および誤り検出ラベル $\hat{y}^{d} \in\{0,1\}$ を得る.

$
\hat{y}^{t}=\underset{y^{t} \in \mathcal{Y}}{\arg \max } P^{t}\left(y^{t} \mid x, \boldsymbol{x}\right), \hat{y}^{d}=\underset{y^{d} \in\{0,1\}}{\arg \max } P^{d}\left(y^{d} \mid x, \boldsymbol{x}\right)
$

得られたタグ系列 $\hat{\boldsymbol{y}}^{t}=\left(\hat{y}_{1}^{t}, \hat{y}_{2}^{t}, \ldots\right)$ をもとに $\boldsymbol{x}$ を編集する2)ことで，訂正文 $\hat{\boldsymbol{t}}=\left(\hat{t}_{1}, \hat{t}_{2}, \ldots\right)$ が得られる. ここで，訂正文の流暢性を計算する何らかの報酬関数を $r(\cdot) \in \mathbb{R}$ としたとき，REINFORCE アルゴリズ 么 [16] によって報酬を考慮した損失 $\mathscr{L}_{r l}$ を次式で計

2）この編集処理では誤り検出確率の閾値を 0 としているた め，タグのみの情報から訂正文が獲得される。
算する.

$\mathscr{L}_{r l}=-(r(\hat{\boldsymbol{t}})-\hat{r}) \sum_{i=1}^{|x|}\left(\log P^{t}\left(\hat{y}_{i}^{t} \mid x_{i}, \boldsymbol{x}\right)+\log P^{d}\left(\hat{y}_{i}^{d} \mid x_{i}, \boldsymbol{x}\right)\right)$

$\hat{r}$ は報酬の分散を抑えるためのべースライン報酬で ある. 式 3 で重要なのは，損失はタグ $\left(\hat{y}^{t}\right)$ および 誤り検出ラベル $\left(\hat{y}^{d}\right)$ の情報から計算するが, 報酬 は訂正文 $(\hat{\boldsymbol{t}})$ の情報から計算することである。これ により，訂正文の流暢性を考慮しながら最適化する ことができる.

最終的な損失 $\mathscr{L}$ は式 4 亿示すように, パラメータ $\lambda(0 \leq \lambda \leq 1)$ を用いて重み付き和を取る.この理由 は，訂正性能および訂正文の流暢性の両方の側面か らモデルを最適化するためである. $\mathscr{L}_{m l e}$ 項が訂正性能の向上に， $\mathscr{L}_{r l}$ 項が訂正文の流暢性の向上に貢献することを期待する.

$
\mathscr{L}=\lambda \mathscr{L}_{m l e}+(1-\lambda) \mathscr{L}_{r l}
$

## 3.2 報酬

式 3 の損失でモデルを学習するためには，報酬関数 $r(\cdot)$ が必要である. 本研究ではこの報酬関数とし て，トークン数で正規化した GPT-2 の perplexityを用 いる。これにより，言語モデルが捉える流暢性を反映した訂正文が生成できると考えられる．ただし， より流暢であるほど報酬が高くなるように設計する ため，式５のように負の perplextiyを Min-Max 法に よって正規化した值を報酬とする。ここで， $p p l(\cdot)$ は GPT-2 のパープレキシティであり， min および max は正規化における最小値と最大値に対応する八 イパーパラメタである. ただし，負の perplexity が 最小値を下回る場合は最小値に丸めることとする。

$
r(\boldsymbol{t})=\frac{\max (\min ,-p p l(\boldsymbol{t}))-\min }{\max -\min }
$

## 4 実験

## 4.1 実験設定

データセット 実験に用いるデータを表 3 に示す. 訓練データは擬似誤りデータである PIE-synthetic $[17]^{3}$ および実誤りデータである FCEtrain [18], Lang-8 [19], NUCLE [20], W\&I+LOCNESStrain [21]を用いる. 実験では Omelianchuk ら [1] と同
 た.表 3 実験に使用するデータセットの概要

様に 3 段階での訓練を行うため，データを用いる段階も表に併記する。評価データには，JFLEG-test [3] および CoNLL-2014 [9]を用いる.

評価方法 訂正文の流暢性や流暢な訂正性能は JFLEG-test に対する出力を 2 節で述べた評価尺度で 評価する．また，最小限の訂正性能を評価するため に CoNLL-2014 に対する出力を $\mathrm{M}^{2}$ スコアラ [10] に より評価する。なお，推論時の 2 種類のパラメータ （2 節を参照）は JFLEG-dev の性能を元に探索する.

学習方法 Omelianchuk ら [1] と同様に 3 段階 での訓練を行う１１段階目は擬似誤りによる訓練，2段階目は無編集の文ペアを除いた実誤り による訓練，3段階目は無編集の文ぺアも含めた W\&I+LOCNESS-trainによる訓練である. 1 段階目で は $\mathscr{L}_{m l e}$ の損失のみで最適化し, $2 \cdot 3$ 段階目では $\mathscr{L}$ の損失で最適化する。 入力文の表現ベクトルを得る ためのエンコーダは RoBERTa [4] とする. 式 4 の $\lambda$ は $\lambda \in\{0.1,0.3,0.5,0.7,0.9\}$ を試行する。また，式 3 のベースライン報酬 $\hat{r}$ にはミニバッチ内の報酬の平均を用いた. 式 5 の min と max はそれぞれ-300と-1 とした。その他の詳細なパラメータについては，付録 Aを参照されたい。

## 4.2 実験結果

実験結果を表 4 亿示す。表 4 では，系列変換モデ ルに基づく既存手法と， $\mathscr{L}_{m l e}$ 項のみで学習すること に等しい $\lambda=1$ の場合，および開発データで最も高 い性能を達成した $\lambda=0.5$ の場合の性能を記載する. まず， $\lambda=0.5$ と $\lambda=1$ の場合を比較すると， $\lambda=0.5$ の場合が JFLEG-test の評価値全てにおいて上回って いる. このことは，強化学習に基づく損失 $\mathscr{L}_{r l}$ が流暢性の向上に寄与していることを示す。また，編集率は 0.2 ポイント向上しており，より大きく書き換 えることで流暢性が向上したと考えられる．提案手法（ $\lambda=0.5 ）$ と系列変換モデル（Kiyono らおよび Kaneko ら）との比較では，GLEU の値は提案手法が
表 4 提案手法と既存手法との性能比較. いずれもシングルモデルの評価結果である．上のブロックは系列変換モデル，下のブロックは系列タグ付けモデルに基づく訂正器の評価結果を示す.

劣っているものの，SOME やGPT-2 のパープレキシ ティの值は競合する結果となった. このことは，提案手法は訂正文の流暢性を向上させるが，そのこと は流暢な訂正性能の向上には必ずしもつながらない ことを示唆する.

CoNLL-2014 において $\lambda=0.5$ と $\lambda=1$ の場合の性能を比較すると，提案手法は $\mathrm{F}_{0.5}$ が 0.6 ポイント低下した. しかしながら，大きな悪化ではないと考え られるため，提案手法は訂正性能を維持しつつ訂正文の流暢性を高められると考えられる。

## 5 分析

$\lambda$ の影響 $\lambda を \lambda \in\{0.1,0.3,0.5,0.7,0.9,1\}$ の中で 変化させたときの JFLEG-test における実験結果を表 5 に示す. $\lambda=0.5$ として, $\mathscr{L}_{m l e}$ と $\mathscr{L}_{r l}$ を同じ重みで 扱うことで最良の結果を得た. GPT-2の值からは， $\lambda$ の値が小さいほど（ $\mathscr{L}_{r l}$ 項の影響が大きくなるほ ど）訂正文が流暢になる傾向にあることが分かる.一方で，SOME の評価結果は逆の傾向を示した. こ のことは，訂正文の流暢性について一貫した評価は 難しいことを示唆する.

表 $5 \lambda$ と流暢性に関する性能値の関係

流暢性が向上する要因 どのような訂正が行 われたことで流暢性が向上したか分析するため, ERRANT [22,23]を用いて JFLEG-test の性能を誤り タイプ別に評価した. $\lambda=1$ の場合を基準として $\lambda=0.5$ の場合の訂正傾向の違いを分析したところ，名詞や動詞といった内容語の訂正が増加している 一方，冠詞・綴り・前置詞といった機能語もしくは表層の訂正はほとんど増加していなかった. 内容語 に関する訂正は機能語や表記の訂正よりも訂正文 の perplexity に大きく影響すると考えられるため， perplexityを報酬とする提案手法は内容語を積極的 に書き換えることで流暢性を向上させたと考えられ る. ただし，訂正の総数は増えているが偽陽性が増 える傾向にあったため，表 4 の GLEU や $\mathrm{M}^{2}$ による 参照あり評価では性能が伸び悩んだと考えられる。誤りタイプ別の評価の詳細は付録 B に示す.

## 6 関連研究

強化学習に基づく文法誤り訂正手法には, Sakaguchi ら [24] と Raheja ら [25] の研究がある. Sakaguchi らは，GRU 系列変換モデルの学習におい て，評価尺度である GLEUを報酬関数とした学習手法を提案した. Raheja らは, 報酬関数として訂正文 の作成者が人間か訂正器かを判別する分類器を用い て, Transformer 系列変換モデルと敵対的に学習させ る手法を提案した。

これらの関連研究と比較して, 本研究は系列タグ 付けに基づく訂正器に強化学習を導入した点が異 なる。また，提案手法では報酬関数を GPT-2 言語モ デルとしたが，関連研究に倣い GLEU や SOME と いった他の尺度を用いたり，新たな報酬関数を設計 して敵対的に学習することには考慮の余地がある.

## 7 おわりに

本研究では, 系列タグ付けに基づく文法誤り訂正器である GECTOR は，訂正文の流暢性が低い問題を 指摘した. 問題の原因は最適化に訂正文の情報が使 われていないことであると考えられる. したがっ て，言語モデルに基づいて計算した報酬を用いて強化学習を行うことで，訂正文の流暢性を考慮する学習手法を提案した. 実験の結果, 訂正性能を維持し ながら訂正文の流暢性が向上した。提案手法と異な る報酬関数による学習や，GECToR 以外の系列タグ 付けに基づく訂正器への応用は今後の課題とする.

## 参考文献

[1] Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. GECToR - grammatical error correction: Tag, not rewrite. In BEA, pp. 163-170, Seattle, WA, USA $\rightarrow$ Online, July 2020. Association for Computational Linguistics.

[2] Keisuke Sakaguchi, Courtney Napoles, Matt Post, and Joel Tetreault. Reassessing the goals of grammatical error correction: Fluency instead of grammaticality. TACL, Vol. 4, pp. 169-182, 2016.

[3] Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. JFLEG: A fluency corpus and benchmark for grammatical error correction. In EACL, pp. 229-234, Valencia, Spain, April 2017. Association for Computational Linguistics.

[4] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.

[5] Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, and Kentaro Inui. An empirical study of incorporating pseudo data into grammatical error correction. In EMNLP-IJCNLP, pp. 12361242, Hong Kong, China, November 2019. Association for Computational Linguistics.

[6] Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. Encoder-decoder models can benefit from pre-trained masked language models in grammatical error correction. In ACL, pp. 4248-4254, Online, July 2020. Association for Computational Linguistics.

[7] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, Vol. 30, , 2017.

[8] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.

[9] Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. The CoNLL2014 shared task on grammatical error correction. In CoNLL, pp. 1-14, Baltimore, Maryland, June 2014. Association for Computational Linguistics.

[10] Daniel Dahlmeier and Hwee Tou Ng. Better evaluation for grammatical error correction. In NAACL, pp. 568-572, Montréal, Canada, June 2012. Association for Computational Linguistics.

[11] Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. Ground truth for grammatical error correction metrics. In ACL, pp. 588-593, Beijing, China, July 2015. Association for Computational Linguistics.

[12] Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. Gleu without tuning, 2016.

[13] Ryoma Yoshimura, Masahiro Kaneko, Tomoyuki Kajiwara, and Mamoru Komachi. SOME: Reference-less sub-metrics optimized for manual evaluations of grammatical error correction. In COLING, pp. 6516-6522, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.

[14] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, Vol. 1, No. 8, p. 9, 2019.

[15] Hiroki Asano, Tomoya Mizumoto, and Kentaro Inui. Referencebased metrics can be replaced with reference-less metrics in evaluating grammatical error correction systems. In IJCNLP, pp. 343348, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing.

[16] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learn- ing, Vol. 8, No. 3, pp. 229-256, 1992.

[17] Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. Parallel iterative edit models for local sequence transduction. In EMNLP-IJCNLP, pp. 4260-4270, Hong Kong, China, November 2019. Association for Computational Linguistics.

[18] Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. A new dataset and method for automatically grading ESOL texts. In ACL, pp. 180-189, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.

[19] Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. Mining revision log of language learning SNS for automated Japanese error correction of second language learners. In ICJNLP, pp. 147-155, Chiang Mai, Thailand, November 2011. Asian Federation of Natural Language Processing.

[20] Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. Building a large annotated corpus of learner English: The NUS corpus of learner English. In BEA, pp. 22-31, Atlanta, Georgia, June 2013. Association for Computational Linguistics.

[21] Helen Yannakoudakis, Øistein E Andersen, Ardeshir Geranpayeh, Ted Briscoe, and Diane Nicholls. Developing an automated writing placement system for esl learners. Applied Measurement in Education, Vol. 31, No. 3, pp. 251-267, 2018.

[22] Mariano Felice, Christopher Bryant, and Ted Briscoe. Automatic extraction of learner errors in ESL sentences using linguistically enhanced alignments. In COLING, pp. 825-835, Osaka, Japan, December 2016. The COLING 2016 Organizing Committee.

[23] Christopher Bryant, Mariano Felice, and Ted Briscoe. Automatic annotation and evaluation of error types for grammatical error correction. In ACL, pp. 793-805, Vancouver, Canada, July 2017. Association for Computational Linguistics.

[24] Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. Grammatical error correction with neural reinforcement learning. In ICJNLP, pp. 366-372, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing.

[25] Vipul Raheja and Dimitris Alikaniotis. Adversarial Grammatical Error Correction. In Findings of EMNLP, pp. 3075-3087, Online, November 2020. Association for Computational Linguistics.

## A 実験における詳細なパラメータ

学習時 4 節で述べたように，モデルの学習は 3 段階で行う. 訓練時のパラメータは表 6 のとおりで ある. 特に表記のない限り，3 段階全てで同じパラ メータを用いた。

表 6 実験に用いたパラメータの詳細

## B 誤りタイプに関する詳細な性能

5 節では，提案手法により流暢性が向上する原因 として，内容語の訂正をより積極的に行っているこ とを述べた。より詳細な性能を表 7 に示す. モデル が行った訂正に注目するため，表 7 には一部の誤 りタイプについて，True Positive に該当する訂正数 (TPカラム), False Positive に該当する訂正数（FPカ ラム）および適合率を示す. さらに，提案手法が従来手法に比べて何倍の訂正数かを示すため, 訂正数 の合計（True Positive と False Positive の和）につい て, $\lambda=0.5$ の場合の値を $\lambda=1$ の場合の値で割った 值を示す ( $\triangle$ カラム).

5 節で述べたように，機能語や表層の訂正と内容語の訂正には訂正数の傾向に違いがある. 名詞につ いては，語彙選択に関する訂正である「名詞」は提案手法により 1.47 倍訂正数が増加した。一方,「名詞の数」は訂正数に変化がない，名詞の数に関する 訂正は文意を大きく変えないため，提案手法を適用 しても変化がなかったと考えられる. 同様のこと は，「動詞」や「動詞の時制」は若干訂正数が増加す るが，「動詞の形態」については変わらないことか らも言える。「形容詞」や「副詞」も訂正数が増加し ていることが分かる．ただし，「形容詞」や「副詞」 は頻度が少ないため $\Delta$ に関して他の誤りタイプと比較するのは難しい。表 7 誤りタイプ別の訂正数と性能値


# The effect of subliminal facial expression on perspective adoption during
## language comprehension in Japanese

\author{
小波津 豪 ${ }^{1}$ ，赤嶺 奨 ${ }^{2}$, 大城 彩佳 ${ }^{3}$ ，新国 佳祐 ${ }^{4}$ ，里 麻奈美 ${ }^{3}$ \\ ${ }^{1}$ University of the Basque Country, ${ }^{2}$ Max Planck Institute for Psycholinguistics, ${ }^{3}$ 沖䋲国際大学, ${ }^{4}$ 新潟青陵大学 \\ tkohatsu001@ikasle.ehu.eus
}

\begin{abstract}
This study explores the link between the perception of positive/negative facial expressions and the broadening/narrowing of cognitive processing by examining whether brief exposure to facial expressions affects participants' tendency to adopt an agent or observer perspective when comprehending null-subject sentences in Japanese. We found no influence of positive facial expressions on perspective adoption, supporting the hypothesis that perceiving positive emotions broadens cognition. We also found that negative facial expressions facilitated the adoption of an actor's perspective, supporting the hypothesis that perceiving negative emotions narrows cognition. The study discusses the effects of subliminal facial priming on cognitive processing and subsequent perspective adoption during language comprehension.
\end{abstract}

## 1 Introduction

During the processing of a sentence, language comprehenders mentally represent the event described in the sentence from a particular perspective (Brunyé, et al., 2009; Sato \& Bergen, 2013). For example, Brunyé et al. (2009) showed that comprehending the pronouns you or $I$ in a sentence (e.g., you are / I am slicing the tomato) primed an actor's perspective (i.e., as if the comprehender is conducting the action), while the pronoun he (e.g., he is slicing the tomato) primed an observer's perspective (i.e., as if the comprehender is observing another's action). In line with Brunyé, et al.'s results, Sato and Bergen (2013) found that no perspective adoption was necessary when agent/observer were not linguistically specified, as in Japanese nullsubject sentences. Their participants saw three sentences: a context sentence; a sentence describing an event in

which the grammatical subject was overtly stated (e.g., You are / He is making an apple pie); and a target sentence with one of three types of grammatical subject (e.g., You are / He is / $\phi$ cutting an apple). They then saw an image of the event depicted from either the actor's or the observer's perspective, and judged whether the image matched what they had read. A reaction time analysis showed that while second-person pronouns primed actor perspective and third-person pronouns primed observer perspective, null-subject sentences led to no significant difference in reaction times to the actor's and observer's perspective images. Interestingly, Niikuni, et al. (2021) found that people with a high sense of agency ${ }^{i}$ (SoA) showed no preference in perspective adoption, consistent with Sato and Bergen's results, but that people with low SoA showed a preference for the observer's perspective ${ }^{\mathrm{ii}}$. Taken together, these studies suggest the possibility that perspective adoption during language comprehension is affected by not only by linguistic cues but also by psychological factors, such as SoA.

Much research on the interaction between emotion and cognitive processing has stated that positive emotions broaden thought, action, and processing while negative emotions narrow them down (e.g., The broaden-andbuild theory of positive emotions in Fredrickson, 2001; Fredrickson \& Branigan, 2005; Schwarz, 2002). This has some empirical support from research on the distribution of attention to a visual field (Fujimura \& Suzuki, 2010) and language processing (Hernandez-Guitierrez et al., 2022; Yano et al., 2018). For instance, Fujimura and Suzuki (2010) found higher accuracy rates on sad-face recognition when the sad face appeared in the central

ii SoA was evaluated on Asai et al.'s (2009) "SoA scale," where a high score indicates unstable SoA and a low score, stable SoA.
visual field rather than in a peripheral field, although they found no such difference for happy-face recognition. Moreover, Yano and Koizumi (2018) found the larger electric activities in a happy mood group for semantic reversal anomalies which required multipleprocessing (world knowledge, syntactic, and semantic processing) compared to a sad mood group.

## 2 The present research

This study uses a sentence-picture verification task with visual priming to investigate the effects of emotional facial expressions on perspective adoption during the comprehension of null-subject sentences in Japanese. It builds on the findings of the studies reviewed above, which all suggest the possibility that observing another person's facial expressions can interact with a language comprehender's perspective adoption. It also draws on social psychology research showing that social information perceived below the level of conscious awareness may activate knowledge that shapes subsequence behavior or various types of decision-making (e.g., Winkielman \& Berridge, 2004; Ferguson and Bargh, 2004, for review). Thus, a brief exposure to another's facial expressions may influence our cognitive processing. More specifically, seeing a positive face would broaden subsequent linguistic processing while seeing a negative face would narrow it. In fact, Akamine, et al., (2022) found that subliminal priming with positive/negative facial expressions facilitated the processing of Japanese null-subject sentences with up/down vertical spatial meanings (e.g., Hashigo-o nobotteimasu '(Somebody is) climbing the ladder'; Hashigo-o kudatteimasu '(Somebody is) going down the ladder'). The current study looks for a similar subliminal priming effect of emotional facial expressions on perspective adoption.

We made two predictions based on the studies of Fredrickson (2001) and Schwarz (2002), which suggest a link between positive/negative emotions and the broadening/narrowing of processing. First, we predicted that a brief exposure to a positive facial expression will make comprehenders engage in heuristic processing, which will enable them to adopt both actor's and observer's perspectives when comprehending Japanese null-subject sentences. Second, we predict that a negative facial expression will make comprehenders engage in analytic processing, which will hinder them from adopting both perspectives, and instead lead them to take one or the other perspective.

## 3 Experiment

## 3.1 Participants

Forty-five native speakers of Japanese, all students at Okinawa International University (Female $=30$, Mage $=$ 21.6, $\mathrm{SD}=1.62$ ), participated in the study.

## 3. 2 Materials

Prime Pictures (Faces): For each trial, participants were first briefly primed by seeing an image of a male's or a female's face $(N=6)$ with either a smile, a neutral expression or a frown for $17 \mathrm{~ms}$ (ATR-promotions, 2006).

Sentence Materials: The sentence materials comprise 72 transitive, past-tense, null-subject Japanese sentences: 36 targets and 36 fillers. Of each set of 36 sentences, 18 describe positive events, as in (1), while the other 18 describe negative events, as in (2).

(1) Target positive sentence

Houseki-o migai-ta tokorodesu.

jewelry-ACC polish-past just

" $\phi$ just polished jewelry"

(2) Target negative sentence Fuusen-o wat-ta tokorodesu. balloon-ACC pop-past just " $\phi$ just popped a balloon"

Target Pictures (Events): The picture materials were drawn specifically for this experiment. For each target and filler sentence, a pair of pictures was created, one from an actor's perspective and another from an observer's perspective. For the actor's perspective $(N=36)$, the pictures showed actions depicted from the participant's viewpoint; that is, as if the participant were conducting the action (Fig. 1a). For the observer perspective $(N=36)$, the pictured actions were depicted from the participant's viewpoint, as if the participant were watching another's action (Fig. 1b).

Figure 1. Popping a balloon from (a) actor's perspective on the left and (b) observer's perspective on the right.

## 3. 3 Procedure

The experiment was conducted with each participant individually in a quiet room, using Psytoolkit (Stoet, 2010, 2017). Participants completed nine practice trials, repeating them if necessary until they reached $100 \%$ accuracy. In the main experiment, each trial started with a fixation cross for $3000 \mathrm{~ms}$, followed by the prime image, which was either the smile, the neutral, or the frown face, presented for $17 \mathrm{~ms}$; the prime was followed by an image of a puzzle for $183 \mathrm{~ms}$. To ensure that participants saw the brief representation of a facial prime, they were told not to blink until they saw the puzzle image. Next, a nullsubject sentence was presented for $3000 \mathrm{~ms}$. The sentence was followed by a fixation cross for $1500 \mathrm{~ms}$, which was succeeded by the picture depicting the sentence action from either the actor's or the observer's perspective. Participants judged whether the picture correctly described the prior sentence by pressing a key on the keyboard: the F key for incorrect or the $\mathrm{K}$ key for correct, having been instructed to respond as quickly and accurately as possible (Fig. 2). As soon as the participant pressed one of the keys for the response, the next trial began.

Figure 2. A trial for an actor's perspective in the smile condition.

## 3. 4 Results

Using the lme4 (Bates, et al., 2015) and the lmerTest (Kuznetsova, et al., 2017) packages in the R programming language ( $\mathrm{R}$ Core Team, 2019), linear mixed-effects model analyses with participants and items as random factors (Baayen, et al., 2008) were conducted. The dependent variable was reaction times (RTs) to the target pictures. Face (frown/neutral/smile) and Perspective (observer/actor) were included as fixed effects with interactions between the factors. The accuracy of the response was included as an additional fixed factor (Accuracy) in the model, without interactions with Face and Perspective.

One participant's data were excluded because the accuracy rate for the target trials was $<75 \%(72.2 \%)$; all the other participants showed an accuracy rate of $>83 \%$. In addition, one trial in which the RT was $>7000 \mathrm{~ms}$ was excluded as an outlier; all remaining RTs were $<5000 \mathrm{~ms}$. Furthermore, for each participant, trials with RTs more than 2.5 SDs away from that individual's mean RT for target trials were excluded. In total, $5.4 \%$ of the target data were excluded from the analysis.

In the mixed model analyses, Face conditions (frown $/$ neutral $/$ smile) were treatment-coded with the neutral condition as the reference level, and Perspective conditions (observer/subjective) were deviation-coded.

Figure 3 shows the mean RTs for the target pictures. The analysis revealed no significant main effect of Perspective $(\beta=-55.4, \mathrm{SE}=66.3, \mathrm{t}=-1.44, \mathrm{p}=.15)$ or of Smile-Face $(\beta=-24.7, \mathrm{SE}=24.1, \mathrm{t}=1.03, \mathrm{p}=.30)$, and no significant interaction between Smile-Face and Perspective $(\beta=32.9, \mathrm{SE}=48.2, \mathrm{t}=0.68, \mathrm{p}=.50)$. In contrast, the main effect of Frown-Face was significant ( $\beta$ $=-60.7, \mathrm{SE}=24.2, \mathrm{t}=-2.51, \mathrm{p}<.05)$, and there was a significant interaction effect between Frown-Face and Perspective $(\beta=103.0, S E=48.4, t=2.13, p<.05)$.

Figure 3. Mean RTs for each experimental condition. Error bars represent the standard error of the mean.

In addition to the RT analysis, we conducted a logistic mixed-effect model analysis for the accuracy of the responses. The analysis revealed no significant main effect of Face or Perspective, and no interaction between them $(p s>.20)$. Table 1 shows the mean accuracy rate for each Face and Perspective conditions.

Table 1: Mean accuracy rate for each experimental condition.

## 4 Discussion

Based on the broaden-and-build theory of positive emotions (Fredrickson, 2001), this study investigated the effects of subliminal facial expressions on perspective adoption during language comprehension in Japanese. In line with Sato and Bergen (2013), the results showed that no particular perspective was adopted in Japanese nullsubject sentences. This finding bears out our first prediction that perceiving positive facial positive facial expressions broadens cognition, allowing comprehenders to adopt either the actor or the observer perspective. We also found that brief exposure to negative facial expressions did influence perspective adoption. Specifically, the Frown-Face $\times$ Perspective interaction was significant in the RT analysis, indicating that the (non-significant) tendency to respond faster to observer- perspective pictures than to actor-perspective ones in the neutral-face condition is reversed in the frown-face condition (see Figure 3). Thus, we can conclude that the brief exposure to negative faces encouraged the participants to adopt an actor perspective rather than an observer perspective when comprehending null-subject sentences, which supports our second prediction. This finding suggests that a negative face might narrow cognitive processing, preventing comprehenders from taking both perspectives but influencing them to take one particular perspective, namely, the actor's perspective.

Furthermore, the main effect of the Frown-Face prime was significant, indicating that the RTs were significantly faster in the negative-face condition than in the neutral condition. However, the finding of no significant main effect of the Smile-Face prime indicates no significant difference in RTs between the positive-face and neutral conditions. The rapid responses in the negative-face condition might possibly be due to a higher arousal level driven by the perception of a negative-face expression compared to neutral and positive-facial expressions (Robinson, et al., 2004).

## Acknow ledgment

This study is supported by KAKENHI \#19H01263

(PI: Manami Sato).

## References

Akamine, S., Kohatsu, T., Niikuni, K., Schafer, A. J., \& Sato, M. (2022). Emotions in language processing:

Affective priming in embodied cognition. In proceedings of the 39th Annual Meeting of Japanese Cognitive Science Society, 326-332.

Asai, T., Takano, K., Sugimori, E., \& Tanno, Y. (2009). Development of the sense of agency scale and its factor structure. Japanese Journal of Psychology, 80, 414-421.

Baayen, R. H., Davidson, D. J., \& Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.

Brunyé, T. T., Ditman, T., Mahoney, C. R., Augustyn, J. S., \& Taylor, H. A. (2009). When you and I share perspectives: Pronouns modulate perspective taking
during narrative comprehension. Psychological Science, 20(1), 27-32.

Ferguson, M. J., \& Bargh, J. A. (2004). How social perception can automatically influence behavior. Trends in Cognitive Sciences, 8(1), 33-39. https://doi.org/10.1016/j.tics.2003.11.004

Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218-226.

Fredrickson, B. L., \& Branigan, C. (2005). Positive emotions broaden the scope of attention and thoughtaction repertoires. Cognition \& Emotion, 19(3), 313332.

Fujimura, T., \& Suzuki, N. (2010). Recognition of dynamic facial expressions in peripheral and central vision. The Japanese Journal of Psychology, 81(4), 348-355.

Kuznetsova, A., Brockhoff, P. B., \& Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13). https://doi.org/10.18637/jss.v082.i13

Moore, J. W. (2016). What is the sense of agency and why does it matter? Frontiers in Psychology, 7, 1272. https://doi.org/10.3389/fpsyg.2016.01272

Niikuni, K., Sato, M., \& Muramoto, T. (2021) Individual differences in sense of agency and perspective adoption in comprehending Japanese null-subject sentences. The Japanese Journal of Psychology, 92(2), 89-99.

R Core Team (2019). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www. R-project.org/ (February 6, 2020)

Robinson, M. D., Storbeck, J., Meier, B. P., \& Kirkeby, B. S. (2004). Watch out! That could be dangerous: Valence-arousal interactions in evaluative processing. Personality and Social Psychology Bulletin, 30(11), 1472-1484.

Sato, M., \& Bergen, B. K. (2013). The case of the missing pronouns: Does mentally simulated perspective play a functional role in the comprehension of person? Cognition, 127(3), 361374.

Schwarz, N. (2002). Situated cognition and the wisdom of feelings: Cognitive tuning. In L. F. Barrett \& P. Salovey (Eds.), The wisdom in feeling: Psychological processes in emotional intelligence (pp. 144-166). The Guilford Press.

Stoet, G. (2010). PsyToolkit—A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 10961104. https://doi.org/10.1016/j.cognition.2004.04.003

Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.

Winkielman, P., \& Berridge, K. C. (2004). Unconscious emotion. Current Directions in Psychological Science, 13(3), 120-123.

Yano, M., Suzuki, Y., \& Koizumi, M. (2018). The effect of emotional state on the processing of morphosyntactic and semantic reversal anomalies in Japanese: Evidence from event-related brain potentials. Journal of Psycholinguistic Research, 47(1), 261-277.


# 左隅型再帰的ニューラルネットワーク文法による 日本語 fMRI データのモデリング

杉本侑嗣 1 吉田遼 1 鄭 嫣婷 2
小泉政利 ${ }^{3}$ Jonathan R. Brennan ${ }^{4}$ 大関洋平 1
1 東京大学大学院総合文化研究科 2 東北大学大学院国際文化研究科
3 東北大学大学院文学研究科言語学研究室 ${ }^{4}$ University of Michigan, Department of Linguistics
\{yushis, yoshiryo0617, oseki\}@g.ecc.u-tokyo.ac.jp \{koizumi, jeong\}@tohoku.ac.jp
jobrenn@umich.edu

## 概要

本研究では、日本語を対象として、新たな fMRI データセットを構築するとともに、(i) 階層構造 を明示的に扱うモデルである再帰的ニューラル ネットワーク文法 (Recurrent Neural Network Grammar, RNNG) の確率的な予測が、扱わない言語モデルで ある Long Short-Term Memory (LSTM) の確率的な予測よりもよく脳活動データを説明できる脳部位は何処なのか、さらに、(ii) 左隅型 RNNG が、トップダ ウン型 RNNG よりも高い説明力を持つ部位は何処 なのか、を調査する。結果として、言語処理に関連 する複数の部位において、RNNG が LSTM よりも脳活動データをより良く説明できるが、その効果は左隅型 RNNG に限定的であることが示され、人間の脳内における左隅型解析による階層構造の構築の実在性が示唆された。

## 1 はじめに

人間の脳内で起こる逐次的な文理解は、理論言語学が仮定するような、自然言語の階層構造 $[1,2]$ の構築を伴うのであろうか。神経言語学では、非常に制約された言語表現の刺激 (e.g., 文vs. 単語 のリスト）を用い、階層構造の処理に関わる脳部位を特定することで、この問いが検証されてきた $[3,4,5,6,7]$ 。一方近年では、自然言語処理のモデル の発達により、自然言語の階層構造を明示的に扱 う・扱わない言語モデルそれぞれの確率的な予測 と、人間の脳活動データを比較することで、「自然 な」刺激文を用いてこの問いを検証することも可能 になってきている $[8,9]$ 。この計算神経言語学のア プローチによる先行研究では、特定の脳部位におい
て、階層構造を明示的に扱う言語モデルの確率的な 予測が、扱わない言語モデルの確率的な予測より も、脳活動データをよく説明できることから、人間 の逐次的な文理解は階層構造の構築を伴うことが示唆されてきた [10]。

しかしながら、先行研究では、階層構造を明示的に扱うモデル・扱わないモデルの二分法による 比較が行われており、階層構造を明示的に扱うモ デルの解析戦略には焦点が当てられていない。特 に、Brennan ら [10] は、主要部先導型右枝分かれの 言語である英語を対象とし、トップダウン型解析戦略の再帰的ニューラルネットワーク文法 (Recurrent Neural Network Grammar, RNNG; [11])を用いて階層構造の処理に関わる脳部位の特定を行なっている が、計算心理言語学の先行研究 $[12,13,14]$ では、 トップダウン型解析戦略は主要部終端型左枝分かれ の言語の解析には適さず、左隅型解析戦略がより認知的に妥当であるとされている。

よって、本研究では、主要部終端型左枝分かれの 言語である日本語を対象として、新たな fMRI デー タセットを構築するとともに、(i) 階層構造を明示的に扱うモデルである RNNG の確率的な予測が、扱わない言語モデルである Long Short-Term Memory (LSTM; [15]) の確率的な予測よりもよく脳活動デー タを説明できる脳部位は何処なのか、さらに、(ii) 左隅型 RNNG が、トップダウン型 RNNG よりも高い 説明力を持つ部位は何処なのか、を調査する。結果 として、言語処理に関連する複数の部位において、 RNNG が LSTM よりも脳活動データをより良く説明 できるが、その効果は、トップタウン型 RNNG には なく、左隅型 RNNG に限定的であることが示され、人間の脳内における左隅型解析による階層構造の構
築の実在性が示唆された。

## 2 橋渡し仮説: サプライザル理論

サプライザル理論 $[16,17]$ によれば、人間は文処理の際、逐次文脈から次にくる語や文節を予測して おり、予測しやすい語や文節は認知的負荷が低く、予測しにくい語や文節の認知的負荷が高くなるとさ れる。さらに、その語や文節の予測しやすさは、(1) のように定式化できるとされる。

$
-\log p(\text { 語や文節 | 文脈 })
$

近年、様々なアーキテクチャの言語モデルが算出し たサプライザルと、人間から得られる視線、脳波や fMRIを比較することで、各アーキテクチャの人間 の逐次的な文理解のモデルとしての認知的妥当性が 検証されている $[18,10,14]$ 。fMRI データを扱う本研究では、認知的に妥当な言語モデルにより得られ た文節単位のサプライザルは、人間の認知的負荷を 表す血中酸素レベル依存性 (BOLD) 信号の増加と近 い傾向を示すことが予想される。

## 3 実験

## 3.1 言語モデル

n-gram $n$-gram は、自然言語の階層構造を明示的 に扱わない言語モデルである。 $n$-gram は、固定窓幅内のみの単語を文脈として扱う（cf. マルコフモデ ル）ため、階層構造を明示的に扱う言語モデルに対 する、単語列に基づく予測の弱いべースラインであ ると言える。本研究では、KenLM 実装の 5 -gram を 用いた。1)

LSTM Long Short-Term Memory (LSTM; [15]) は、自然言語の階層構造を明示的に扱わない言語モデ ルである。LSTM は、先行文脈全体を 1 つの単語べ クトルとして扱うため、階層構造を明示的に扱う言語モデルに対する、単語列に基づく予測の強いべー スラインであると言える。本研究では、単語埋め込 み・隠れ層の次元数 256 の 2 層 LSTM を用いた。

トップダウン型・左隅型 RNNG 再帰的ニュー ラルネットワーク文法 (Recurrent Neural Network Grammar, RNNG; [11]) は、自然言語の階層構造を明示的に扱う言語モデルである。生成の各時刻では、 RNNG は逐次スタック LSTM を用いて先行する単語列とその階層構造を 1 つのべクトルに集約し、その

ベクトルに基づき次の時刻のアクションが次の 3 つ から選択される：
- GEN：終端記号（単語）の生成。

・NT : 非終端記号（句）を開く。

・REDUCE：非終端記号（句）を閉じる。

REDUCE の際には、双方向 LSTM に基づく構成関数 により、閉じられた句はそれを表す一つのべクトル へと集約される。本研究では、トップダウン型解析戦略、および左隅型解析戦略、という、2つの異 なる解析戦略の RNNG（それぞれ、トップダウン型 RNNG、および、左隅型 RNNG）を対象として検証 を行う。Noji and Oseki [19]により実装2 された、単語埋め込み・隠れ層の次元数 256 の 2 層スタック LSTM を持つトップダウン型・左隅型 RNNG を用い た。また、それぞれの RNNG の推論には、単語同期型ビームサーチ [20]を用いた。

各言語モデルは、吉田ら [14]により NINJAL Parsed Corpus of Modern Japanese (NPCMJ) ${ }^{3}$ )で学習された ものを用いた。

## 3.2 fMRI 実験手続

実験参加者 本研究の fMRI 実験には 42 名の日本語母語話者（全員右利きの健常成人、女性 23 名、平均年齢 21.1 歳、 $\mathrm{SD}=1.7$ ) が参加した。参加者は全員正常な視力（矯正を含む）を有していた。

刺激文と課題 本研究で使用する刺激文は、「現代日本語書き言葉均衡コーパス (Balanced Corpus of Contemporary Written Japanese: BCCWJ、[21]) にある 様々なジャンルの中から新聞記事の一部を使用して いる。 BCCWJ-EyeTrack [22] と BCCWJ-EEG [23] と 同様に、新聞記事 20 件の記事を使用し、 5 件の記事ごとに 4 つのブロックに分けられる。各記事の提示には PychoPy を使い [24, 25]、 Rapid Serial Visual Presentation (RSVP) によって文節が $500 \mathrm{~ms}$ ごとに表示される (各文節の間には $500 \mathrm{~ms}$ 何も表示されな い)。また記事を読み終える度に質問が提示される。本実験では、fMRIによって実験参加者が文節ごと に区切られた刺激文を MRI 装置の中で読んでいる 際の脳活動データ収集を行った。

前処理 fMRI データの前処理はSPM12を使用 し、頭の動きの補正 (realignment)、スライスタイミ ング補正 (slice timing correction)、脳解剖画像の脳機能画像への登録 (coregistration) と脳解剖画像の分解

2) https://github.com/aistairc/rnng-pytorch

3) http://npcmj.ninjal.ac.jp
(segmentation)、脳機能画像の標準化 (normalization)、脳機能画像の空間的平滑化 (smoothing) の処理を 行った。

## 3.3 fMRI データ解析

fMRI 解析するにあたり、42 名の fMRI データのう ち 34 名分の fMRI データを使用した。また、最初の データポイント 10 スライス分 $(20$ 秒、 $\mathrm{TR}=2.0)$ は、刺激提示されていないため、解析からは除いた。

説明変数 本研究では、サプライザルを用いて人間の文処理における予測処理をモデル化することを 目的としているが、人間の言語処理に際しては、そ の他の関連する脳活動も生じるため、これらをモデ ル化する説明変数をべースラインとして使用する。先行研究を踏襲し [10]、このベースラインには、文節のオフセットに 1、それ以外は 0 を与える word rate、文節の長さ (word length)、文節内単語頻度の対数平均 (word frequency)、記事内の文の位置 (sentid)、文中の文節の位置 (sentpos)、fMRI データ取得の際 の各参加者の頭の動きのパラメータ $6 \supset(\mathrm{dx} 、 \mathrm{dy} 、$ $\mathrm{dz} 、 \mathrm{rx} 、 \mathrm{ry} 、 \mathrm{rz})$ を用いる。

本研究で関心のある各言語モデルのサプライザル の値とベースラインで使用する説明変数は Python の パッケージである nilearn ${ }^{4}$ を使用し、Hymodynamic Response Function (HRF) によって推定 BOLD 信号に 変換した。また、各言語モデルのサプライザルは、 word rate に対して直交処理 (orthogonalization) を行 なった。fMRI データポイントは 2 秒ごとに記録さ れるため $(\mathrm{TR}=2.0)$ 、各説明変数の推定 BOLD 信号に は $0.5 \mathrm{~Hz}$ で再サンプリング処理を行った。

関心領域解析 関心領域解析に伴い、先行研究に基づいて左脳の 8 つの関心領域を選んだ [4, 10, 26, 27, 28, 5]。具体的には、以下の関心領域を 使用した; 下前頭回の弁蓋部 (Inferior frontal gyrus opercular part、IFG_Oper)、三角部 (Inferior frontal gyrus triangular part、IFG_Tri)、眼窝部 (Inferior frontal gyrus orbital part、IFG_Orb)、下頭小葉 (Inferior parietal lobule、 IPL)、角回 (Angular gyrus、AG)、上側頭回 (Superior temporal gyrus、STG)、上側頭極 (Temporal pole: superior temporal gyrus、sATL)、中側頭極 (Temporal pole: middle temporal gyrus、mATL)。fMRI データの各関心領域 ごとの BOLD 信号の抽出には、Automated Anatomical Labeling (AAL) アトラス [29]を使用した。

これらの各領域のモデル化には、R の lmer パッ ケージの線形混合モデルを用いた。具体的には、 ベースラインの回帰モデルに各言語モデルのサプラ イザルを説明変数として 1 つずつ付け加え、モデル を付け加える前の回帰モデルとの間で R の anova 関数により分散分析を行う。分析の結果、統計的な有意差が出れば、より fMRI に対して言語モデルの精度の高いことを示す。ベースライン回帰モデルには 参加者 (subject_number) をランダム切片として加え た、以下の線形混合モデルを用いる。

$
\begin{aligned}
& \text { ROI } \sim \text { word_rate }+ \text { word_length }+ \text { word_freq } \\
& + \text { sentid }+ \text { sentpos }+d x+d y+d z+r x+r y+r z
\end{aligned}
$

$
+(1 \mid \text { subject_number })
$

全脳解析 探索的な全脳解析では、nilearn の一般線形モデルを用い、解析を行った。個人レベル解析 では、頭の動きのパラメータ以外の説明変数を元に デザインマトリックスが作成され、各実験参加者の fMRI データごとに係数がマップされた NIfTI ファイ ルが集団解析のために作成された。集団解析では、 Yeo ら [30] のマスクを使用し、ガウスフィルター 処理 (8mm の半値全幅) が行われた。閾値マップは z-scored されたものを使用した。false discovery rate $\alpha$ $=0.05$ が設定され、 50 voxels のクラスターサイズの 閾値とした。閾値のピークの座標と AAL での領域名を特定するため、AtlasReader [31]を使用した。

## 4 結果と考察

## 4.1 関心領域解析

関心領域解析の結果は、表 $1 、 2$ に示した。表 1 の 結果では、階層構造を明示的に扱う RNNG が、扱 わないモデルである LSTM よりもよく脳活動デー タを説明できる脳部位を同定することを目的とし た。まず左隅型 RNNG の説明力を調べるため (表 1 上部)、ベースライン回帰モデルに $n$-gram、LSTM を加えたモデルと、更に左隅型 RNNG(RNNG_LC) を加えたモデルの分散分析を行うと、IFG_Oper、 IFG_Tri、IPL、AG、STGの 5 つの領域で統計的な有意差が認められた。同様の分散分析をトップダウン 型 RNNG(RNNG_TD) で行った結果、どの領域でも 統計的な有意差は認められなかった (表 1 下部)。表 2 では、トップダウン型 RNNG よりも左隅型 RNNG がどの脳領域で説明力があるかを調べるため、ベー スライン回帰モデルに $n$-gram、LSTM、RNNG_TDを

表 1 Model Comparison の結果。Bonferroni 法により有意水準 $\alpha=0.003125$ で検定した.

加えたモデルとそのモデルに RNNGLC を加えたモ デルとの分散分析を行った。結果として、mATL 以外の部位で統計的有意差が見られた。表 1 と表 2 の 結果を踏まえると、左隅型 RNNG は、LSTM に比べ て、関心領域の 5 つでより脳活動データを説明で き、トップダウン型 RNNG と比べても mATL 以外 の部分で説明力を示している。一方トップダウン型 RNNG は、LSTM に対してどの脳領域でもより高い 説明力を示す結果にはならなかった。

表 2 Model Comparison の結果。Bonferroni 法により有意差 $\alpha=0.00625$ で検定を行った。

## 4.2 全脳解析

全脳解析では、図 $1 、 2 、 3$ に示される結果となっ た (詳細な結果は表 3 を参照)。LSTM(補正なし)で は、左脳後側頭葉の脳活動が見られ、トップダウン 型 RNNG（補正なし）では、左脳紡錘状回での脳活動が見られ、本研究の関心領域解析で扱った脳部位
では、活動が見られなかった。左隅型 RNNG では、左脳下頭小葉での脳活動が見られた。この脳領域 は、領域関心解析の結果でも統計的有意差の出てお り、先行研究ではトップダウン型 RNNG のサプライ ザルに対して、この部位での脳活動が報告されてい る [10]。つまり英語では、トップダウン型の RNNG で効果の見られる領域が、本研究では左隅型 RNNG で見られている結果となった。このことは、少なく とも日本語では、左隅型 RNNG のほうが、トップダ ウン型 RNNG より脳活動データに対して説明力を 持つことを支持している。

図 1 LSTM (補正なし)

図 2 RNNG_TD (補正なし)

図 3 RNNGLC

## 5 おわりに

本研究では、日本語を対象として、新たな fMRI データセットを構築し、(i) 階層構造を明示的に扱 うモデルである RNNG の確率的な予測が、扱わな い言語モデルである LSTM の確率的な予測よりも よく脳活動データを説明できる脳部位は何処なの か、さらに、(ii) 左隅型 RNNG が、トップダウン型 RNNG よりも高い説明力を持つ部位は何処なのか、 を調査した。結果として、言語処理に関連する複数 の部位において、RNNG が LSTM よりも脳活動デー タをより良く説明できるが、その効果はトップダウ ン型 RNNG にはなく、左隅型 RNNG に限定的であ ることが示され、人間の脳内における左隅型解析に よる階層構造の構築の実在性が示唆された。

## 謝辞

本研究は、JSPS 科研費 JP21H05061、JP19H05589、 JST さきがけ JPMJPR21C2 の助成を受けたものです。

## 参考文献

[1] Noam Chomsky. Syntactic Structures. Mouton, The Hague, 1957.

[2] Martin B.H. Everaert, Marinus A.C. Huybregts, Noam Chomsky, Robert C. Berwick, and Johan J. Bolhuis. Structures, not strings: Linguistics as part of the cognitive sciences. Trends in Cognitive Sciences, Vol. 19, No. 12, pp. 729-743, 2015.

[3] Christophe Pallier, Anne-Dominique Devauchelle, and Stanislas Dehaene. Cortical representation of the constituent structure of sentences. Proceedings of the National Academy of Sciences, Vol. 108, No. 6, pp. 2522-2527, 2011.

[4] D.K. Bemis and Liina Pylkkänen. Basic Linguistic Composition Recruits the Left Anterior Temporal Lobe and Left Angular Gyrus During Both Listening and Reading. Cerebral Cortex, Vol. 23, No. 8, pp. 1859-1873, 062012.

[5] Emiliano Zaccarella and Angela D. Friederici. Merge in the human brain: A sub-region based functional investigation in the left pars opercularis. Frontiers in Psychology, Vol. 6, , 2015.

[6] Emiliano Zaccarella, Lars Meyer, Michiru Makuuchi, and Angela D. Friederici. Building by Syntax: The Neural Basis of Minimal Linguistic Structures. Cerebral Cortex, Vol. 27, No. 1, pp. 411-421, 102017.

[7] William Matchin, Christopher Hammerly, and Ellen Lau. The role of the IFG and pSTS in syntactic prediction: Evidence from a parametric study of hierarchical structure in fMRI. Cortex, Vol. 88, pp. 106-123, 2017.

[8] Jonathan R. Brennan. Naturalistic sentence comprehension in the brain. Language and Linguistics Compass, Vol. 10, No. 7, pp. 299-313, 2016.

[9] John Hale, Luca Campanelli, Jixing Li, Shohini Bhattasali, Christophe Pallier, and Jonathan R. Brennan. Neurocomputational models of language processing. Annual Review of Linguistics, Vol. 8, No. 1, pp. 427-446, 2022.

[10] Jonathan R. Brennan, Chris Dyer, Adhiguna Kuncoro, and John T. Hale. Localizing syntactic predictions using recurrent neural network grammars. Neuropsychologia, Vol. 146, p. 107479, 2020.

[11] Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. Recurrent neural network grammars. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 199-209, San Diego, California, June 2016. Association for Computational Linguistics.

[12] Steven P. Abney and Mark Johnson. Memory requirements and local ambiguities of parsing strategies. Journal of Psycholinguistic Research, Vol. 20, No. 3, pp. 233-250, 1991.

[13] Philip Resnik. Left-corner parsing and psychological plausibility. In Proceedings of the 14th Conference on Computational Linguistics - Volume 1, COLING '92, pp. 191-197, USA, 1992. Association for Computational Linguistics.

[14] Ryo Yoshida, Hiroshi Noji, and Yohei Oseki. Modeling human sentence processing with left-corner recurrent neural network grammars. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2964-2973, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.

[15] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, Vol. 9, No. 8, pp. 1735-1780, 111997.

[16] John Hale. A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics, 2001.

[17] Roger Levy. Expectation-based syntactic comprehension. Cognition, Vol. 106, No. 3, pp. 1126-1177, 2008.

[18] John Hale, Chris Dyer, Adhiguna Kuncoro, and Jonathan Brennan. Finding syntax in human encephalography with beam search. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2727-2736, Melbourne, Australia, July 2018. Association for Computational Linguistics.

[19] Hiroshi Noji and Yohei Oseki. Effective batching for recurrent neural network grammars. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 43404352, Online, August 2021. Association for Computational Linguistics.

[20] Mitchell Stern, Daniel Fried, and Dan Klein. Effective inference for generative neural parsing. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1695-1700, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.

[21] Kikuo Maekawa, Makoto Yamazaki, Toshinobu Ogiso, Takehiko Maruyama, Hideki Ogura, Wakako Kashino, Hanae Koiso, Masaya Yamaguchi, Makiro Tanaka, and Yasuharu Den. Balanced corpus of contemporary written Japanese. Language Resources and Evaluation, Vol. 48, No. 2, pp. 345-371, 2014.

[22] Masayuki Asahara, Hajime Ono, and Edson T. Miyamoto. Reading-time annotations for "Balanced Corpus of Contemporary Written Japanese". In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 684-694, Osaka, Japan, December 2016. The COLING 2016 Organizing Committee.

[23] Yohei Oseki and Masayuki Asahara. Design of BCCWJ-EEG: Balanced corpus with human electroencephalography. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 189-194, Marseille, France, May 2020. European Language Resources Association.

[24] Jonathan W. Peirce. Psychopy-psychophysics software in python. Journal of neuroscience methods, Vol. 162, No. 1-2, pp. 8-13, 2007.

[25] Jonathan W. Peirce. Generating stimuli for neuroscience using psychopy. Frontiers in Neuroinformatics, Vol. 2:10, , 2009.

[26] Angela D. Friederici. Language in our brain: The origins of a uniquely human capacity. MIT Press, 2017.

[27] Peter Hagoort. Chapter 28 - muc (memory, unification, control): A model on the neurobiology of language beyond single word processing. In Gregory Hickok and Steven L. Small, editors, Neurobiology of Language, pp. 339-347. Academic Press, San Diego, 2016.

[28] William Matchin and Gregory Hickok. The cortical organization of syntax. Cerebral Cortex, Vol. 30, No. 3, pp. 1481-1498, 2020.

[29] N. Tzourio-Mazoyer, B. Landeau, D. Papathanassiou, F. Crivello, O. Etard, N. Delcroix, B. Mazoyer, and M. Joliot. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neurolmage, Vol. 15, No. 1, pp. 273-289, 2002.

[30] B. T. Yeo, Fenna M. Krienen, Jorge Sepulcre, Mert R. Sabuncu, Danial Lashkari, Marisa Hollinshead, Joshua L. Roffman, Jordan W. Smoller, Lilla Zöllei, Jonathan R. Polimeni, Bruce Fischl, Hesheng Liu, and Randy L. Buckner. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology, Vol. 106, No. 3, pp. 1125-1165, 2011. PMID: 21653723.

[31] Michael Philipp Notter, Dan Gale, Peer Herholz, Ross Markello, Marie-Laure Notter-Bielser, and Kirstie Whitaker. AtlasReader: A python package to generate coordinate tables, region labels, and informative figures from statistical MRI images. Journal of Open Source Software, Vol. 4, No. 34, p. 1257, 2019.

## A 全脳解析結果一覧

} & \multirow[t]{2}{*}{ Region (AAL) } \\

表 3 LSTM、RNNG_TD、RNNG_LCの全脳解析の結果。false discovery rate: $\alpha=0.05 、 50$ voxel のクラスターサイズの閾値 を設定した。


# Developing a Typology for Language Learning Feedback

Steven Coyne ${ }^{1,2}$ Diana Galvan-Sosa ${ }^{1,2} \quad$ Keisuke Sakaguchi $^{1,2} \quad$ Kentaro Inui $^{1,2}$
${ }^{1}$ Tohoku University ${ }^{2}$ RIKEN
coyne.steven.charles.q2@dc.tohoku.ac.jp
\{dianags, keisuke.sakaguchi, kentaro.inui\}@tohoku.ac.jp

## Abstract

Writing is an important part of language learning. With the recent release of corpora containing feedback on learner writing, it has become easier for NLP researchers to examine this process and work towards such tasks as automatic feedback comment generation. However, analysis and generation are hindered by a lack of a typology for such comments, and it is costly to determine frequency distributions or generation error rates for different kinds of comments. In this paper, we discuss typologies from both NLP and educational research, and propose a system to combine them to create an annotation scheme for feedback comments.

## 1 Introduction

Written corrective feedback on learner text is widespread in language education, and an active area of research in the field of second language acquisition [1]. Research has shown that properly administered written feedback has a positive effect on language learning [2, 3], including in electronic settings [4]. Analysis of the content of these feedback comments, as well as where and when they are added by instructors, is of great interest to educators, educational technology developers, and NLP researchers focusing on learner writing.

In addition, it is ideal if realistic comments can be generated automatically during assessment in learning environments. This would save teachers time and energy by providing suggestions and allowing them to accept or edit suitable ones while rejecting unsuitable ones. Similar technology can be used to provide feedback comments directly to learners in an intelligent tutoring setting.

In NLP, work on language education feedback comments has mostly been from the perspective of feedback comment generation, defined as the task of generating hints or explanatory notes for language learners [5]. Data consists

Figure 1 Visualization of the use of manually-labeled feedback comments to support the development of a template-based feedback comment generation system.

of learner sentences, associated feedback comments, and offsets highlighting where the comments were attached to the sentence. There are very few corpora of such sentence pairs, and work on them has mostly focused on the ICNALE Learner Essays with Feedback Comments Dataset[6], ${ }^{1)}$ summarized in in Table 1. There is also a translated subset of this dataset used in GenChal 2022 [7], and a separate corpus focusing on transition word comments [8].

One challenge to working with feedback comment data is the lack of a dedicated typology for feedback comments and their connection to the original sentence. If such a typology existed, it could enable more detailed analysis of these corpora and support certain approaches to feedback comment generation, as detailed in Figure 1.

Educational research characterizing feedback comments often focuses on the learning effect of comments based on dimensions such as directness [2], presence of metalinguistic terms [3], and hedging [9]. However, these approaches do not touch on the comment's editorial intent. Meanwhile, the closest typologies from NLP are the error types seen in the field of Grammatical Error Correction (GEC). These error types model grammatical errors well, but can not capture many broader topics that instructors tend to comment on, as described in section 2. We seek to develop a typology specifically for the context of feedback comment corpora consisting of learner sentences and instructor comments.

Table 1 Information about the "ICNALE Learner Essays with Feedback Comments" dataset. It is divided into two sub-corpora, one with comments on general topics, and the other focusing on preposition use.

## 2 Background

The type of feedback comment data we discuss in this work was first presented in the context of feedback comment generation [5]. Challenges revealed in subsequent work in this subfield serve as motivation for creating a dedicated typology. Several issues were highlighted by Hanawa et al. [10] and the participants of GenChal 2022. First, generation is confounded by a many-to-one problem in which multiple comments can have the same topic:

## (1) ${ ^{\star}$ We reached to the station.}

Because the verb "reach" is a transitive verb, the preposition "to" is not required.

## (2)`I reached to New York.

"Reach" is a transitive verb. This verb does not require a preposition prior to the object.

These comments are about the same error, but the text content is superficially different. This can result in mixed generations which are less clear [10].

Furthermore, there are a large number of very specific comments relating to particular words and their collocations. The number and diversity of such comments contributes to the mixed output problem as well.

Finally, generation can result in inaccurate or misleading comments [10]. It is important to constrain these false generations, which can have a negative learning effects or reduce confidence in the system.

It is difficult to address the above issues without first identifying in greater detail what kinds of feedback comments exist and how these issues manifest across those categories. It is possible that a few kinds of feedback comments are responsible for a large portion of generation errors. Currently, such investigation would involve manual analysis using ad-hoc categories, and any further analysis by another team would involve a similar process starting over from scratch. It is therefore beneficial to develop a categorization system for data from this task.

When approaching the issue of how to tag feedback comments on learner sentences, a natural first step is to consider the error type tags which have been used in the NLP subfield of GEC. These include the system used in the NUCLE dataset [11], the system used in Cambridge Learner Corpus [12] and seen in the First Certificate in English (FCE) dataset [13], and the system from ERRANT [14], which has become the dominant system in GEC research at this time. These typologies model grammatical errors fairly well, often by using a combination of parts of speech and the actions of deletion, insertion, or replacement to describe the errors.

There are many cases where an instructor's comment identifies a grammatical error in a straightforward manner easily modeled by these systems. However, there are many additional cases in which the highly local and orthographically-focused GEC categories can not fully characterize the issue identified in the comment. The feedback comments instructors write often refer to complex errors or combinations of errors. Even a simpler operation such as adding the preposition "for" could be done for a number of reasons, such as establishing a purpose clause, completing an idiom, or to serve as a transition word (see Figure 2). Meanwhile, the ERRANT system would tag all such cases as "M:PREP" (Missing Preposition). This describes the desired edit, but not the reason for the edit.

The reverse is also true, as any number of errors could be connected to a comment about an "unclear" sentence. These many-to-one and one-to-many cases suggest the need for additional tags concerning "higher level" concepts common in instructor comments, such as purpose clauses and conditionals. Furthermore, some common types of feedback comments are broader than grammatical error correction itself. There are comments about idiom, tone, the argument made by a sentence, and praise, resulting in feedback which may not align with GEC error types at all.

Figure 2 Abstraction of a one-to-many relationship between an ERRANT error type and different kinds of feedback comments.

## 3 Method

To address this, we can search beyond NLP error typologies and look to the field of education. One direct source of topics for teacher comments is the various sets of error code annotations used by English teachers. These tend to include many of the higher-level categories we wish to address, such as redundancy and idiom. While there does not seem to be a well-accepted correction code standard in literature, there are a variety of systems shared online, many covering similar topics. One example is the system used for writing programs at the University of California, Irvine [15]. Ideally, a classification system for instructor feedback comments can incorporate some of these principles when the GEC error types can not fully express the content of the comment.

We propose a loosely hierarchical tagging system in which the "lower" level tags correspond to GEC-style error types such as "Unnecessary Preposition," and "higher" level tags are used to characterize comments which exceed the scope of GEC, such as praise, idiom corrections, and advice about forming specialized clauses such as conditionals and dummy subject clauses. For a given sentencecomment pair, we apply the highest level tag which can accurately characterize the comment's topic.

The tags are developed by first consulting previous typologies, considering which categories are most likely to be used by English language teachers. ${ }^{2)}$ Furthermore, we define several principles for the development of the tag set:

1. It exists to descriptively model a phenomenon in natural language, the content of feedback comments. Anything common in this domain should be incorporated.

2. For now, we focus on the the sentence level, not considering the broader coherence of ideas in the text.

2) The first author worked in English education for five years, and drew on that experience in the process.
3. The categories should be easily understandable to educators and potential corpus annotators.

4. Each feedback comment should be labeled with a single tag which best characterizes it.

5. For comments about issues in the text, the relevant grammar point or type of clause is identified.

6. Tags are applied in hierarchical order, except when a comment clearly focuses on the lower level topic.

After drafting candidate tags, we test them against the contents of the ICNALE Learner Essays with Feedback Comments dataset, adapting to the realities of the corpus sentences. As a first step, we considered a subset consisting of 500 sentences, 250 each from the General and Preposition sub-corpora. We consider only sentences with exactly one feedback comment, and comments extending across multiple sentences were excluded, since they exceed the sentence-level scope of this work. Sentences were then sampled with a particular random seed.

The current high-level tags are given in Table 2, and the remainder of the current tag set is presented in the appendix. Work is ongoing, and the proposed tag set may evolve further by the time all sentences in the dataset have been considered and annotated.

## 4 Discussion

As the tag set developed, we obtained several insights. The frequency of certain topics was not always in accordance with expectations. Praise comments were very common, necessitating a dedicated tag for them. Comments proposing complex, specific rewrites were also quite common, leading to a "Rewrite" tag for comments of this nature. Depending on goals of developers or researchers, it may be desirable to exclude such comments from generation. It is thus prudent to tag them at this stage to allow toggling, and to facilitate any future attempts to classify comments as praise or direct rewrite suggestions.

It was also necessary to determine how fine-grained to make the tagging system. Ideally, there is enough coverage to describe the majority of instructor comments in a meaningful way, with only a few placed into an "other" category. The number of tags should be expressive enough to allow for disambiguation of cases as seen in Figure 2. At the same time, a large number of rare categories may not be as interpretable, and may be more difficult for annotators. We will continue to consider this balance as work progresses.

## 5 Future Work

With categorized feedback comments, it becomes possible to compare the comments from each category against other information. One such possibility is to cluster the sentences and compare the clusters to the manual labels. This can be used to judge the variability of different kinds of feedback comments. [16] attempted to address the superficial diversity issue by clustering the comments with textual similarity, but the interpretability of the resulting categories is limited. It would be useful to compare such results to class labels added by a human.

In addition to surface similarity, we will experiment with clustering based on semantic similarity or with a topicmodeling approach as seen in [17]. Topic labels can be identified in the feedback comment text, and placed into hierarchical clusters. Given that there are many synonyms for grammatical terms in the feedback comments, we hypothesize that semantic or topic modeling will perform better than surface similarity. We will compare clusters to the manual tags, potentially revealing additional topic subtypes which can help improve the tagging logic.

It is also possible to compare the manual tags to the distribution of tags from various NLP tools. These include sequence taggers for parts of speech and dependencies as well as error correction systems. If some feedback comment topics correlate very strongly with certain parse patterns or GEC error types, those system outputs may be useful as predictive features for the feedback comments.

Furthermore, it is possible to replace the outputs of highly diverse comment categories with generalized tem- plates. Manual tags allow us to identify the categories which most need such attention. The data may simplify if such comments are unified into a limited number of semiautomatically generated templates with slot-filling. The slots can be filled with words from the original sentence and information from lexical resources. This can help with the reliability challenge in feedback comment generation, since we can more tightly control outputs in these cases.

Creating templates also allows us a chance to rewrite comments to be more suitable to the task. We find that many of the comments in the ICNALE feedback dataset have fairly advanced grammatical explanations, which can be simplified to help learners understand them. Furthermore, there are comments with an error diagnosis, comments with edit suggestions, and comments containing both. If these are differentiated in the template system, developers or users of a comment generation system can decide whether to disable outputs from one or the other.

## 6 Conclusion

To assist with the analysis of written feedback comments for language learning, we propose a system to annotate feedback comment data with comment topics combining principles from grammatical error correction and the field of education, applying it to a subset of the ICNALE Learner Essays with Feedback Comments dataset as a first step in a plan to annotate the entire corpus. In the context of feedback comment generation, we describe how these tags can be used to assist with the analysis of data and the creation of template-guided generation systems.

## Acknowledgments

This work was supported by JSPS KAKENHI Grant Numbers JP22H00524 and JP21K21343.

## References

[1] Eun Young Kang and ZhaoHong Han. The efficacy of written corrective feedback in improving 12 written accuracy: A meta-analysis. The Modern Language Journal, Vol. 99, pp. 1-18, 2015.

[2] Dana Ferris and Barrie Roberts. Error feedback in 12 writing classes: How explicit does it need to be? Journal of Second Language Writing, Vol. 10, No. 3, pp. 161184, 2001.

[3] John Bitchener. Evidence in support of written corrective feedback. Journal of Second Language Writing, Vol. 17, No. 2, pp. 102-118, 2008.

[4] Estela Ene and Thomas A. Upton. Learner uptake of teacher electronic feedback in esl composition. System, Vol. 46, pp. 80-95, 2014.

[5] Ryo Nagata. Toward a task of feedback comment generation for writing learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3206-3215, Hong Kong, China, November 2019. Association for Computational Linguistics.

[6] Ryo Nagata, Kentaro Inui, and Shin'ichiro Ishikawa. Creating corpora for research in feedback comment generation. In Proceedings of the 12th Language Resources and Evaluation Conference, pp.340-345, Marseille, France, May 2020. European Language Resources Association.

[7] Ryo Nagata, Masato Hagiwara, Kazuaki Hanawa, Masato Mita, Artem Chernodub, and Olena Nahorna. Shared task on feedback comment generation for language learners. In Proceedings of the 14th International Conference on Natural Language Generation, pp. 320-324, Aberdeen, Scotland, UK, August 2021. Association for Computational Linguistics.

[8] Ildiko Pilan, John Lee, Chak Yan Yeung, and Jonathan Webster. A dataset for investigating the impact of feedback on student revision outcome. In Proceedings of the 12th Language Resources and Evaluation Conference, pp. 332-339, Marseille, France, May 2020. European Language Resources Association.

[9] Wendy Baker and Rachel Hansen Bricker. The effects of direct and indirect speech acts on native english and esl speakers' perception of teacher written feedback. System, Vol. 38, No. 1, pp. 75-84, 2010.

[10] Kazuaki Hanawa, Ryo Nagata, and Kentaro Inui. Exploring methods for generating feedback comments for writing learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9719-9730, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
[11] Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. Building a large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 22-31, Atlanta, Georgia, June 2013. Association for Computational Linguistics.

[12] Diane Nicholls. The cambridge learner corpus: Error coding and analysis for lexicography and elt. Proceedings of the Corpus Linguistics 2003 conference, p. 572-581, 2003.

[13] Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 180-189, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.

[14] Christopher Bryant, Mariano Felice, and Ted Briscoe. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 793-805, Vancouver, Canada, July 2017. Association for Computational Linguistics.

[15] UCI Writing Center. Correction symbols for uci writing programs, 2008.

[16] Ryo Nagata and Kazuaki Hanawa. Kasotekina ayamari taipu no wariate ni yoru kaisetsubun seisei no seino kojo [improving the performance of commentary generation by assigning virtual error types]. In Proceedings of the 27th Annual Conference of the Association for Natural Language Processing, pp. 679-684, March 2021.

[17] Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794, 2022.

Table 3 Hierarchical Annotation System for Feedback Comment Topics, Mid-level and Low-level tags


# VO Word Order and Monotheism VO 語順と一神教

EHARA Terumasa
江原 暉将
Ehara NLP Research Laboratory
江原 NLP 研究室
eharate @ gmail.com
}

\begin{abstract}
The relationship between monotheism and the word order of verb $(\mathrm{V})$ and object $(\mathrm{O})$ in declarative sentences of its practiced language is analyzed. The oldest monotheistic religion is the Aten belief of ancient Egypt, which was practiced in the ancient Egyptian language. Abrahamic religions, which are typical monotheistic religions are practiced in the following languages: Judaism was in Aramaic and biblical Hebrew; Christianity was in Aramaic, ancient Hebrew, and Koine Greek; and Islam was in Arabic. The word order of these languages are all verb object (VO). Moreover, I have investigated the relationship between the world's ethnic religions and the word order of their practiced languages. For 284 data points, monotheism is practiced in VO vs. OV as 20 vs. 8. On the other hand, polytheism is practiced in VO vs. OV as 111 vs. 145. From these findings, I conclude that monotheism is more frequently practiced in the VO language than in the OV language.
\end{abstract
## 1 Introduction

I have quantitatively investigated the relationship between the word order of language and the thinking patterns of its speakers. I used the following quantitative measures to express the human thinking patterns: suicide rate and homicide rate (Ehara 2015), distribution of Ychromosome DNA haplogroups (Ehara 2018), distribution of mitochondrial DNA haplogroups (Ehara 2019), and relative frequency of the short allele of the serotonin transporter gene (5-HTTLPR) (Ehara 2021).

In this paper, I will analyze the relationship between monotheism and word order. The relationship between monotheism and word order was partly treated in Ehara (2017). However, it only distinguished Abrahamic religions (Judaism, Christianity, and Islam) from other religions. In this paper, I will moreover consider monotheisms other than Abrahamic religions.

I will consider only the word order of verb $(\mathrm{V})$ and object (O) in declarative sentences. This word order feature is the most significant in the word order typology. In this paper, this feature or feature values (VO or OV) will be simply called "word order". Word order data are mainly obtained from Yamamoto (2003) and Dryer and Haspelmath (2013).

My conjecture is that "monotheism is more frequently practiced in the VO language than in the OV language".

## 2 Monotheism and polytheism

What is monotheism? What is the criterion to distinguish monotheism from polytheism? It is difficult to answer these questions. In this paper, I set the distinction shown in Table 1.

Table 1. The distinction between monotheism (mono) and polytheism (poly)

## 3 Ancient Egypt

Most of the ancient Egyptian religions are polytheisms.

Exceptionally, during the reign of Akhenaten (BC 13641347), he introduced monotheism that worships only the Aten (god of the sun). Aten belief is considered the oldest monotheism (Motomura, 2005, pp. 62-74). Aten belief is practiced in the ancient Egyptian language (in the AfroAsiatic family), which has VO word order.

## 4 Abrahamic religions

Typical monotheisms are Abrahamic religions (Judaism, Christianity, and Islam). Languages related to these religions and their word orders are listed in Table 2. The word order of Greek (Koine) is obtained from Kirk (1981). The word order of languages in Table 2 are all VO. Only Greek (Koine) belongs to the Indo-European family. Other languages in Table 2 belong to the Afro-Asiatic family.

Table 2. Abrahamic religions and the word order of the related languages

 &  &  &  \\

## 5 Change to monotheism in Greece

## and Rome

Ancient Greece and Rome were both in polytheistic cultures. The period that these areas changed to monotheistic culture is considered AD 311 when Roman Emperor Constantine I sanctioned Christianity.

Word order changes in Greek and Latin are listed in Table 3 and Table 4. In Table 4, VO-ratio means the ratio of the number of VO word order sentences in the total
^{i}$ The religious data in Appendix 1 are obtained mainly from the following Wikipedia pages: List of ethnic religions, Traditional African religions, Native American religions, Australian Aboriginal religion and mythology, and Traditional religions in Papua New Guinea.
}

number of VO or OV word order sentences in the document or writer. During the period of changing to monotheism (AD 311), the word orders of Greek and Latin moved from OV predominant to VO predominant (except for Mycenaean Greek).

Table 3 Word order change in Greek

(Matsumoto 1975, Kirk 1981)

Table 4 Word order change in Latin (Matsumoto 1975, Bauer 2009)

## 6 Ethnic religions

Several ethnic religions are monotheistic. I investigated 135 ethnic religions. The number of populations believing these religions is 295, and the number of languages is 294. Combining these data, 320 data points are obtained. 284 data points among them have a word order of VO or OV. In these 284 data points, the number of monotheisms is 28 and the number of polytheisms is 256 . Part of the detailed data is listed in Appendix 1, , ii.

The frequency of word orders for monotheism and polytheism is shown in Table 5. In this table, 1417 languages in WALS (all) (Dryer and Haspelmath 2013) are also shown.

Table 5 Number of languages classified by word order and monotheism / polytheism and WALS data

ii The full data (Excel file) is downloadable from the author's website:

http://www.ne.jp/asahi/eharate/eharate/ronbun.files/NLP 2022.xlsx.

The relative frequencies of the VO word order languages (VO-ratio) are shown in Figure 1. The VO-ratio of the languages for monotheism is higher compared with polytheism and WALS data (all).

Figure 1 Relative frequency of VO languages (VOratio)

The $\chi 2$ tests to the data in Table 5 are executed. The results are shown in Table 6. The difference between monotheism and polytheism is significant at $1 \%$ level, and the difference between monotheism and WALS data (all) is also significant at $5 \%$ level. However, the difference between polytheism and WALS data (all) is nonsignificant.

Table 6 The results of the $\chi 2$ test

## 7 Conclusion

The relationship between word order and monotheism is analyzed. The oldest monotheism or Aten belief was practiced in Ancient Egyptian. Typical monotheisms or Abrahamic religions (Judaism, Christianity, and Islam) were practiced in Egyptian, Aramaic, Hebrew, Koine Greek, or Arabic. These languages all have VO word order. Moreover, an analysis of 135 ethnic religions showed that monotheistic ethnic religions are more frequently practiced in the $\mathrm{VO}$ languages than in the $\mathrm{OV}$ languages.
One of the future works is to clarify the reason why monotheism is more frequently practiced in the VO language than in the OV language.

## References

Bauer, Brigitte L. M. 2009. Word order. In New perspectives on historical Latin syntax : syntax of the sentence / edited by Philip Baldi, Pierluigi Cuzzolin. Deutsche Nationalbibliothek.

Dryer, Matthew S. and Martin Haspelmath. 2013. The World Atlas of Language Structures Online, https://wals.info.

Ehara, Terumasa. 2015. Relation between Word Order Parameters and Suicide / Homicide Rates. Jounal of Yamanashi Eiwa College, 13:9-29.

Ehara, Terumasa. 2017. Relation between Word Order Characteristics and Suicide / Homicide Rates (6) Adding a Religious Feature -, In Proceedings of The 23rd Annual Meeting of The Association for Natural Language Processing: 190-193.

Ehara, Terumasa. 2018. Relation between Word Order of Languages and the Entropy of Y-chromosome Haplogroup Distribution of the Speakers' Population. In Proceedings of The 24th Annual Meeting of The Association for Natural Language Processing: 10191022.

Ehara, Terumasa. 2019. Relation between Word Order of Languages and the Entropy of Mitochondrial DNA Haplogroups Distribution of the Speakers' Population. Proceedings of The 25th Annual Meeting of The Association for Natural Language Processing:490493.

Ehara, Terumasa. 2021. Relation between Word Order of Languages and Serotonin Transporter Gene. Proceedings of The 27th Annual Meeting of The Association for Natural Language Processing:579584.

Kirk, Allison. 1981. Word order and information structure in New Testament Greek, Doctoral Thesis of Faculty of Humanities, Leiden University.

Matsumoto, Katsumi. 1975. Developments of the Syntactic Patterns in the Indo-European Languages -A comparative and typological study-, Gengo Kenkyu, Vol. 68, 15-43 (in Japanese).

Motomura, Ryoji. 2005. Polytheism and Monotheism, Iwanami Shinsho 967, Iwanami Shoten (in Japanese).

Yamamoto, Hideki. 2003. Survey and Historical Study of Geographical and Genealogical Distribution of Word Order around the World. Tansuisha (in Japanese).

## Appendix 1 Monotheism(M) and polytheism $(P)$ for ethnic religions and

## the word order (OV I VO) of the practiced languages

} & \multirow{2}{*}{} & \multirow{2}{*}{} & \multirow{2}{*}{} & \multirow{2}{*}{} & \multirow{2}{*}{} \\

 & China (Yunnan) & Bai & Sino-Tibetan & VO \\

 & Indo-European & VO \\


# Vertical motions in the learning of spatial, emotion, and social words

Sho Akamine ${ }^{1}$ Tsuyoshi Kohatsu ${ }^{2}$ Ayaka Oshiro ${ }^{3}$ Manami Sato ${ }^{3}$
${ }^{1}$ Max Planck Institute for Psycholinguistics ${ }^{2}$ University of the Basque Country ${ }^{3}$ Okinawa International University
sho.akamine@mpi.nl
}

\begin{abstract}
People express and understand abstract concepts in terms of concrete concepts. For example, the abstract concept of "good-bad" is structured via the concrete source domain of "up-down." Recent studies demonstrate that activating a concrete source domain by performing or perceiving vertical motions can enhance the learning of valence words (e.g., joy, misery). This study attempts to replicate these findings and extend them to spatial and social words.
\end{abstract
## 1 Introduction

Theories of embodied cognition propose that individuals' perceptual experiences shape the structure of concepts in their minds [1-7] For instance, according to conceptual metaphor theory [8], abstract concepts (e.g., time) are expressed and constructed in terms of concrete concepts (e.g., money) that individuals have perceived and stored via their interactions with the physical world $[2,4,8]$. One type of conceptual metaphor is orientational metaphors, in which abstract target domains (e.g., positive or negative valence) are structured in terms of the more concrete source domains of spatial orientations (e.g., up and down) [4,8].

Recent findings empirically support cognitive mappings between the source domains of space and motion and the target domain of emotional valence. For example, socially meaningful postures and gestures as well as meaningless upward and downward motor actions can generate positive or negative feelings and memories [9].

Further, a recent study showed that metaphorcongruent motor actions can improve word learning [10]. In the study, 72 L1-Dutch speakers were asked to learn 16 words in an "alien" language (i.e., pseudowords), which were paired with eight positive emotion and eight negative emotion Dutch words, using paper flashcards. After studying a flashcard for six seconds, the participants placed it on either an upper or a lower shelf based on the condition to which they were assigned. For instance, in the metaphor-congruent condition, participants were instructed to place flashcards with positive definitions on the upper shelf and flashcards with negative definitions on the lower shelf; thus, the directions metaphorically associated with the meaning matched the direction of the motor action (i.e., positive meaning-upward motion; negative meaning-downward motion). In contrast, participants assigned to the metaphor-incongruent condition were told to place flashcards with positive meanings on the lower shelf and flashcards with negative meanings on the upper shelf; thus, the directionality was mismatched. Participants in the neutral condition pilled all the flashcards together on their desktop. Each participant performed these actions with each flashcard three times. They then took a two-alternative forced-choice recognition test. The results showed the highest proportion of correct responses in the metaphor-congruent condition, followed by the neutral and then the metaphorincongruent condition. Based on these results, the researchers suggested that metaphor-congruent motor actions can improve the learning of emotionally charged words, presumably because the metaphor-congruent motor actions activate the source domain of the emotion words, facilitating the process of connecting existing concrete concepts (upward/downward direction) and novel words, whereas metaphor-incongruent motor actions impede such connections [10].

One question raised by Casasanto and de Bruin's study [10] is which parts of a motor action (e.g., motor planning,
kinesthetic feedback, and visual feedback) contribute to the learning enhancement effect. Another question is whether the learning enhancement effect they found would apply to the acquisition of words that activate vertical spatial schemas either directly (spatial words) or metaphorically (emotion or social words).

Akamine et al. [11] conducted an experiment to investigate whether the visual perception of vertical motions, either alone or along with the performance of motor action, would improve the learning of valence words. The study found that: (i) accuracy was highest in the metaphor-congruent condition regardless of modality, which is consistent with the previous findings [10]; and (ii) reaction times in the visual- perception-withoutmotor-actions condition were significantly faster than in the visual-perception-with- motor-actions condition. This unexpected finding might suggest that the motor actions impeded learning, presumably because the actions occupied too much of the working memory capacity that the participants required for their word learning.

To extend the previous findings, the present study tests two hypotheses: (i) visual perception of vertical motions improves the learning of words that activate up/down spatial concepts either metaphorically (emotion and social words) or directly (spatial words), and (ii) the degree of learning enhancement effect does not change for spatial, emotion, and social words.

## 2 Methodology

## 2. 1 Participants

Fifty-nine (43 female, 16 male) right-handed native Japanese speakers living in Okinawa participated in the experiment. Their age range was $18-27$ years $\left(M_{\text {age }}=21.7\right.$, $\left.S D_{\text {age }}=1.62\right)$. The participants received gift cards.

## 2. 2 Word Materials

Pseudowords We created 54 English-based pseudowords by replacing the second or later consonantal syllable onset of basic English words with another consonant (e.g., "orber" based on "order"). Half of the pseudowords consisted of two syllables, and the other half consisted of three.

Associated Meaning Each pseudoword was paired with a real Japanese word as the meaning of the pseudoword. Of the 54 pseudowords, 18 were paired with Japanese translations of emotionally charged English words from the ANEW corpus [12], 18 with social words, and 18 with spatial words. Each word category had two subcategories, one associated with the source domain of UP and the other associated with the source domain of DOWN. Thus, half of the emotionally charged words carried positive meanings (e.g., joy), and the other half carried negative meanings (e.g., war). Likewise, half of the social words had positive meanings, implied by higher social status (e.g., CEO), and the other half had negative meanings, implied by lower social status (e.g., poor). Of the spatial words, half were what we call "up" words, meaning they refer to higher spatial positions (e.g., cloud), and the other half were "down" words, indicating lower spatial positions (e.g., floor). These categories are summarized in Table 1.

Table 1. Summary of associated meanings

 & Example \\

& Negative & DOWN & 不幸 (misery) \\
Social & Positive & UP & 社長 (CEO) \\
& Negative & DOWN & 無職 (unemployed) \\
Spatial & Up & UP & 雲 (cloud) \\
& Down & DOWN & 床 (floor) \\

## 2. 3 Experimental Design

Nine conditions were created by manipulating the two independent variables of directional congruency (congruent, incongruent, control) and word category (emotion, social, spatial).

In the experiment, participants saw a series of digital flashcards, each of which, after six seconds, automatically moved until it left the screen and was replaced by the next flashcard. In the congruent condition, the flashcard moved in the same direction as the source domain of the word on the flashcard: upward for positive (emotion/social words) and up (spatial words) meanings, and downward for negative (emotion/social words) and down (spatial words) meanings. In the incongruent condition, the flashcards moved in the opposite direction from the associated meaning: flashcards with positive/up meanings moved downward and ones with negative/down meanings moved upward. In the control condition, the flashcards moved
rightward or leftward randomly. The three conditions are represented in Figure 1.

Figure 1. Directional congruency conditions

## 2. 4 Procedure

Learning Phase Participants first completed a learning phase of three blocks, each with six words for each word category, for a total of 18 words per block (i.e., three positive emotion, three negative emotion, three positive social, three negative social, three up spatial, and three down spatial), and a total of 54 words for the entire learning phase. The congruency conditions were assigned to each word such that they were equally distributed within blocks.

Each block of the learning phase presented the 18 words three times. The participants saw a pseudoword with its Japanese translation on a digital flashcard and were asked to study it for six seconds while thinking about whether the word carried a positive/up or negative/down meaning. After six seconds, the flashcards automatically moved upward, downward, or right/leftward depending on the condition. Once the participant had seen all 18 words, the next round began, with the same 18 words presented in a different order. The same procedure was repeated for each of the three sets of 18 words, so that the participants studied all 54 words, three times each, during the learning phase.

Test Phase Fifteen minutes after the learning phase, participants completed the two-alternative forced choice recognition test. In each trial, a fixation cross with a gray background appeared for $500 \mathrm{~ms}$, and then a pseudoword was presented for 3000 ms. Next, they saw another fixation cross with a white background, followed by two side-by-side options for the meaning of the pseudoword; participants selected between the two meanings by pressing $\mathrm{F}$ for the left option and $\mathrm{J}$ for the right (see Figure 2). The two options were the meaning of the target pseudoword (i.e., correct response) and the meaning of another of the pseudowords that had been in the same block in the learning phase (i.e., incorrect response). The entire experiment was programmed in PsyToolkit [13-14].

Figure 1. Flow of the experiment

## 2. 5 Analysis

Data Exclusion Data from four participants with a mean accuracy below 68.5\% were removed (based on a binomial test with an alpha of 0.01 ); this procedure resulted in the removal of 216 responses (54 items $x 4$ participants).

For the reaction time analyses, two timed-out responses (i.e., $10000 \mathrm{~ms}$ ) and 50 exceedingly fast responses (i.e., below $400 \mathrm{~ms}$ ) were removed from the data. In addition, 118 responses with reaction times of more than $2.5 S D$ above or below the participant's mean were removed.

As a result, we performed statistical analyses on 2970 responses from 55 participants for accuracy and on 2800 responses for reaction time.

Coding We adopted a treatment coding scheme for coding two categorical independent variables: congruency and word category. For congruency, the control condition was set as the reference level (0). We created two treatment levels that contrasted the reference level with the congruent condition ( $c t l \_c g$ ) and with the incongruent condition (ctl_incg). Likewise, the spatial word category was set as the reference level for word category, and two treatment levels contrasting the reference level with the emotion condition (spt_emt) and with the social condition (spt_soc) were created.

Models A mixed-effects regression model and a linear mixed-effects model were performed for accuracy and reaction time, respectively, using lme4 [15] and the ImerTest package [16] in R [17]. Both models included aforementioned four treatment levels as fixed effects, and participants and items as random intercepts. The model structure is summarized below:

## 3. Results

Accuracy The mean accuracy for each congruency condition across the three word categories is visualized in Figure 3. Figures 3 and 4 use these abbreviations: $\mathrm{InCg}=$ incongruent condition; $\mathrm{Ctl}=$ control condition; $\mathrm{Cg}=$ congruent condition; error bars represent standard errors.

Figure 2. Mean accuracy

A mixed-effects regression model revealed significantly lower accuracy in the control condition for social words $(M=81.8 \%, S D=18.8 \%)$ than for spatial words $(M=87.9 \%, S D=14.1 \%)[\beta=-0.50, S E=0.24, z$ $=-2.07, p=.04]$. A power analysis using the simr package [18] with 500 simulations indicated a power of $52.4 \%$ $(47.9 \%-56.9 \%)$. No other significant effects were found.

Reaction Time The distribution of reaction times for each congruency condition across the three word categories is plotted in Figure 4. The $\mathrm{x}$ in the boxplots represents mean reaction time. A linear mixed-effects model revealed no significant effects.

Figure 3. Distribution of reaction time

## 4. Discussion

This study tested two hypotheses: (i) visual perception of vertical motions improves the learning of words that activate up/down spatial concepts either metaphorically (emotion and social words) or directly (spatial words), and (ii) the degree of learning enhancement effect does not change for spatial, emotion, and social words.

The results did not support the first hypothesis, as there was no significant effect of perceiving congruent vertical motions on either accuracy or reaction times. We speculate two possible explanations.

First, unlike previous studies in which participants completed a task with only one congruency condition (between-subject design) [10] or in three blocks, each associated with one condition (within-subject block design) [11], we used a within-subject randomized block design in which each block included all three conditions. Thus, for instance, participants learned one word in the congruent condition and the next word in the incongruent condition, which may have led to effects that interfered with each other, making the effects less powerful. Supporting this explanation, the mean accuracy for the congruent condition was higher than that for the incongruent condition regardless of word category.

Second, reanalyzing the data from the previous studies using a similar model as in the present study, we did not find significant effects of congruent motions $(\beta=0.63$, $S E$ $=0.44, z=1.42, p=0.16$ for $[10] ; \beta=0.50, S E=0.27, z$ $=1.90, p=0.06$ for [11]). Also, the statistical power (32.0\% for [10]; 48.2\% for [11]) was inadequate [18-19]. These findings suggest that a between-subject design might be more suitable for investigating effects of visually perceiving congruent or incongruent vertical motions.

As for the second hypothesis, we found mixed results. Although the difference was not statistically significant, the mean accuracy was higher for the congruent condition than for the incongruent condition, suggesting a learning enhancement (interference) effect of perceiving meaning congruent (incongruent) vertical motions. However, this pattern was less prominent for spatial words. This implies that perceiving meaning-congruent/incongruent motions improves/interferes with word learning only when the mapping between the target meaning and the domain of spatial orientations is metaphorical, but more research is needed to further our understanding of this possibility. Lastly, significantly lower accuracy in the control condition for social words than spatial words might indicate that social words are more difficult to learn.

## Acknow ledgement

This study is supported by KAKENHI \#19H01263

(PI: Manami Sato).

## References

[1] Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577-660. https://doi.org/10.1017/S0140525X99002149

[2] Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59(1), 617-645.

https://doi.org/10.1146/annurev.psych.59.103006.0 93639

[3] Bergen, B. K. (2015). Embodiment. In E. Dabrowska \& D. Divjak (Eds.), Handbook of cognitive linguistics (pp. 10-30). Walter de Gruyter GmbH \& Co KG.

[4] Lakoff, G., \& Johnson, M. L. (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. Basic Books.

[5] Shapiro, L. A. (2019). Embodied cognition (2nd ed.). Routledge/Taylor \& Francis Group.

[6] Stanfield, R. A., \& Zwaan, R. A. (2001). The effect of implied orientation derived from verbal context on picture recognition. Psychological Science, 12(2), 153-156. https://doi.org/10.1111/1467-9280.00326

[7] Wilson, A. D., \& Golonka, S. (2013). Embodied cognition is not what you think it is. Frontiers in Psychology, 4, 58.

https://doi.org/10.3389/fpsyg.2013.00058

[8] Lakoff, G., \& Johnson, M. (2003). Metaphors we live by. University of Chicago Press.

[9] Casasanto, D., \& Dijkstra, K. (2010). Motor action and emotional memory. Cognition, 115(1), 179-185. https://doi.org/10.1016/j.cognition.2009.11.002

[10] Casasanto, D., \& de Bruin, A. (2019). Metaphors we learn by: Directed motor action improves word learning. Cognition, 182, 177-183. https://doi.org/10.1016/j.cognition.2018.09.015

[11] Akamine, S., Omine, A., Kohatsu, T., Niikuni, K., \& Sato, M. (2021). Visual perception of vertical motions improves valence word learning. Conference handbook for the 22nd Annual
International Conference of Japanese Society for Language Sciences (JSLS22), 79-82.

[12] Bradley, M. M., \& Lang, P. J. (1999). Affective norms for English words (ANEW): Instruction manual and affective ratings [Technical report C-1]. The Center for Research in Psychophysiology, University of Florida.

[13] Stoet, G. (2010). PsyToolkit: A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 10961104. https://doi.org/10.3758/BRM.42.4.1096

[14] Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.

https://doi.org/10.1177/0098628316677643

[15] Bates, D., Mächler, M., Bolker, B. M., \& Walker, S. C. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48. https://doi.org/10.18637/jss.v067.i01

[16] Kuznetsova, A., Brockhoff, P. B., \& Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1-26. https://doi.org/10.18637/jss.v082.i13

[17] R Core Team. (2020). R: A language and environment for statistical computing. $\mathrm{R}$ Foundation for Statistical Computing.

https://www.r-project.org/

[18] Green, P., \& MacLeod, C. J. (2016). SIMR: An R package for power analysis of generalized linear mixed models by simulation. Methods in Ecology and Evolution, 7(4), 493-498. https://doi.org/10.1111/2041-210X. 12504

[19] Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for $\mathrm{t}$-tests and ANOVAs. Frontiers in Psychology, 4.

https://www.frontiersin.org/articles/10.3389/fpsyg. 2013.00863