# Genre-based Character Network Analysis and Emotion Sequence Analysis for Manga

Megha Sharma $^{1}$ Yoshimasa Tsuruoka $^{1}$
${ }^{1}$ The University of Tokyo
\{meghas, yoshimasa-tsuruoka\}@g.ecc.u-tokyo.ac.jp
}

\begin{abstract}
This paper explores the relationship between emotion sequences and character network graphs from Japanese comics to explore trends among different genres. We propose a framework to extract emotion sequences from the manga, and analyse whether significant correlations exist between temporal pages and emotions of the manga as well as between emotion variables. We also detect the peaks and valleys to find correlations with genres. We build social network graphs from the characters in pages to analyse the structural relationship across genres and extract features from the graphs and report on the trends across genres. Our results suggest that some preliminary trends exist in the current scope of the study.
\end{abstract
## 1 Introduction

Japanese comics, manga, are a multi-media art form where visual cues (drawings) and textual cues (dialogues) are used to create a complete story [1]. Additional visual elements such as style and onomatopoeia are used to enhance the reading experience. Manga analysis allows us to understand about the content and organisation of the manga through a social and reflective lens. Manga analysis not only benefits a social interpretation of multi-media, but it also supports applications such as manga translation [2], in-painting [3], and sketch-to-manga conversion [4].

Existing studies on manga analysis show that societal influence plays an important role in the development of manga. Research on gender-based and genre-based division in dialogues reveal mangas are influenced by dialogue patterns that mimic societal usage over the years [5]. In fact, Sugishita and Masuda [6] argue that manga story-lines exhibit character networks that reflect empirical social networks in society.
In this paper, we aim to analyse manga characters and content through a lens of genre. Our contributions are two-fold. We view our work as an extension of Sugishita and Masuda's [6] work by extending the character network analysis to recognise genre-based patterns. Moreover, we also conduct a brief emotion sequence analysis based on genres. Our results find preliminary trends in the sequences and character graphs.

## 2 Method

We short-listed 46 books from the Manga109 dataset [7] [8], which is annotated with face, dialogue and body bounding boxes. There are 12 genres defined in the dataset. We use four books from each genre except for horror where only two books are available. For character network analysis we only use two books from Four-Frame Cartoon (FFC) genre due to available data and annotation conflicts. We approach the analysis using two methods: Emotion sequence analysis and character network analysis. The former utilises the continuous sequence of emotions detected using the character faces and dialogues. The latter builds character networks of the characters appearing in manga and analyses the resulting graphs. In this section, the details of both methods are explained.

## 2.1 Emotion Sequence Analysis

We define emotions using Russell's four quadrant theory that maps emotions as a function of arousal (intensity) and valence (positivity) [9]. Emotion is detected using the dialogues (text) of the manga page as well as the character faces. To do so, a face emotion detection model and a text emotion detection model are trained. The face emotion detection model is trained on a dataset of 509 manga character faces associated with 11 emotion classes ${ }^{1)}$. An


Figure 1 Emotion sequence for manga titled "Paraiso Road" by Kanno Hiroshi. The emotions are normalised from 1 to 5

EfficientNet model [10] with a custom classification layer achieves a validation accuracy of $77.7 \%$. We convert the emotion categories to valence and arousal values based on the NRC-VAD lexicon[11]. We use a pre-trained JapaneseBERT $^{2}$ model for text emotion detection. We fine-tune the model on a dataset of 30,000 social media blog posts [12]. The model achieves an accuracy of $61 \%$ on sentiment (valence) detection. A case study of emotion sequences is shown in fig. 1 , where the normalised emotion variables of arousal and valence are plotted. The plots suggest spikes in arousal at three peaks, with a considerable pace change towards the end of the book at page 100. The sharp change in arousal and valence boosts the emotion to a high point, which can signal a happy end to the story line. This is reflected as the story reaches a positive pay-off in the final few pages. Qualitative analysis of emotion sequences can help understand the emotional arc of the story-line and identify climactic points (points of sharp peaks or valleys).

We extract features from the emotion sequences that will be used to understand behaviour by genre. These features can be categorised into three: detecting (1) temporal relations, (2) manga feature relations, and (3) peaks and valleys. For temporal relations, we calculate the Kendall tau correlation coefficient between temporal (pages) and other continuous manga features such as arousal, valence and text length. We also compute Pearson correlation among the three manga features. Finally, we keep track of number of peaks and valleys detected for each book in both arousal and valence sequences. The peaks and valleys are

2) huggingface.co/minutillamolinara/bert-japanese_finetunedsentiment-analysis defined for emotions above or below a threshold of four and two respectively when normalised on a scale of one to five. The statistics of the results of all features can be found in appendix A.1. These results are made only using the statistically significant coefficients to avoid noise from insignificant correlations.

## 2.2 Character Network Analysis

From the analysis by Sugishita and Masuda [6], we take a social network approach to our manga analysis by genre. Hence, we create temporal bipartite graphs where one layer of nodes are the pages and the other are the characters. An edge between the character and the page exists if the character exists on the page. Using this graph, we can project it on a space of only the character nodes to capture the character network. Examples of extracted character networks are shown in fig. 2 - fig. 4. As seen in the figures, the character graphs can vary from size and structure. Complete graphs show equal distribution of character interactions through the manga, while star-like graph show a protagonist-driven story. On the other hand, subgroups may exist as seen in fig. 4 despite a well-connected "protagonist" (node with highest degree). These might suggest such stories have sub-plots that feature different characters. Our goal is to analyse these properties to detect any trends between the genre of these novels.

Similar to emotion sequence analysis, we extract features from the graphs to study for the discussion. We extract the number of nodes and edges as $N$ and $M$ respectively. We also measure the average degree of the graph as $K$ and the coefficient of variation of the degree as $K C V$. The latter measures the variation trends in the degree of all nodes in the graph by taking the ratio of the standard deviation to the mean. We also measure the degree of the "protagonist", $K P . K P$ is also normalised by dividing by the rest of the number of nodes. Degree assortativity, $R$, measures whether nodes with similar degrees are closer to each other or further apart [13]. Clustering coefficient, $C$, measures the extent of clustering as a triangle in the graph [14].

## 3 Discussion

## 3.1 Emotion Sequence Analysis

We have plotted the average temporal relation coefficients in fig. 5 for each of the 12 genres. Science fiction

Figure 2 A completely connected social graph from OL Lunch by Sanri Youko.

Figure 3 A star-like social graph from Touyou Kidan by Tojo Miho.

Figure 4 A social graph with sub-groups from Tsubasa No Kioku by Sato Harumi. had no statistically significant results in this step. Apart from horror, all genres exhibit mostly a negative correlation with time. This might suggest that the manga has decreasingly lower arousal and valence with the turn of the pages. However, the relationship is not very strong, ranging from 0 to -0.17 at best. Animal genre has the most negative correlation between the dialogue text length and the page number, suggesting that the dialogues get shorter with the story. On the other hand, Suspense and Historical drama both show a positive correlation between text length and page number. Historical drama in particular is the longest manga on average, with a text character length of over $28 \mathrm{~K}$. Horror has the most positive correlation with valence and arousal. However, it should be noted that these significant values are taken from only one book in the genre as the other has no significant relation. This highlights a potential limitation of the study, the limited books used here act as a barrier to the understand the performance across genres.

In particular, we found a very strong correlation between arousal and valence across all genres. The lowest correlation is that of 0.7 (Animal genre), achieving up to 0.9 (FFC genre). This strong correlation seems to point the data mostly stays in the first and third quadrant of Russell's four quadrants. We believe this narrow view of emotion is reflective of the capacity of the emotion detection models rather than the actual manga emotion content. All genres also have a positive correlation between emotion variables (arousal, valence) and text length. This relationship is the lowest for Love Romance and Romantic Comedy and highest for FFC, implying that romantic genres tend to associate little arousal growth with text length of the manga. The relationship between valence and text length remains stable between 0.6 to 0.8 across all genres.

Figure 5 Temporal correlations of emotion sequences by genre. Empty bars suggest no statistically significant results were observed.

Finally, we look into the content of the valence and arousal values across pages using peak and valley detection. In general, the percentage difference between peak and valleys is over $97 \%$, implying valleys are far more common than peaks across genres. In particular, suspense genre has the lowest number of arousal valleys, suggesting that arousal is not usually on the lower end, while Animal has the highest number of arousal valleys, suggesting that arousal is usually on the lower end. Humour and Love Romance also have the highest number of valence valleys, suggesting sentimental lows in the story.

In general, we found that arousal, valence, and text length do not hold significant differences among genre. One reason for such results could be the limited data and model performance. Moreover, manga is more generally differentiated by intended audience such as Shoujo (manga for girls) and Shounen (manga for boys) are more common genre descriptions for Japanese comics [5]. However, we leave the analysis as per gendered genres as future work as it requires simultaneous studies of gender and genre differentiation.

## 3.2 Character Network Analysis

The statistical results from character network graphs is available in table 1. On average, there are 28 characters defined for a manga. However, the variety of network size varies among genres. Our results find that Historical Drama has the largest number of characters and edges on average, which falls in-line with the text length of the books. Four-Frame Cartoons (FFC) tend to have the smallest number of characters and edges. An example graph fig. 2 shows how FFC genre can be different from other genres seen in fig. 4 and 3. This is also confirmed by the low $K C V$ we find for FFC, which indicates that the degree is more homogeneous than diverse, implying a small number of characters interact with similar number of characters throughout the manga.

The coefficient of variation of the degree of graph lies between 0.00 (completely connected graph) to 1.15 (high variation in the graph). On average, $K C V$ takes a value of 0.62 , with a majority of manga falling under 1 . This is consistent with the results by Sugishita and Masuda [6] where smaller social networks, as seen in manga, exhibit similar characteristics. Historical drama achieves the highest graph degree on average, while animal genre has the lowest. These results are intuitively in-line with results from $N$ and $M$, with a positive correlation with both. The normalised degree of the protagonist on average is 0.84 , which shows high level of connectivity in manga social networks. This could suggest, most manga are protagonist driven, a similar conclusion derived in [6]. Romantic Comedy has the significantly lowest $K P$ among the genres at 0.6 , while other genres are all above 0.75 . On structural analysis of the character graphs, it is evident that Romantic Comedy is more likely to exhibit graphs with sub-groups of characters similar to fig. 4. Whether these exist due to sub-plots can not be verified from graph structure and emotion sequence alone. Detailed results are available in appendix A.2.

As seen in fig. 6, character networks are all dissassortative, which means that nodes with dissimilar degrees are closer to each other. This interesting result is also exhibited in [6]. We find that horror and Four-frame cartoons exhibit the lowest dissassortativeness. This is due to the particular well-connected graphs seen in both genres, where the story is not as protagonist-centered as other genres. On the other

Figure $6 R, C$ values for each genre based on the character network graphs. Manga exhibits disassortative networks

hand, all genres exhibit a high level of clustering. These clustering coefficients are largely similar, with an average of 0.83 across all books. This implies the graphs exhibit a large number of triangles.

In general, character network analysis reveals some interesting trends across genres, which can be analysed further with more books and longer volumes.

Table 1 Statistical results for all genres for character network analysis. $N$ is for number of nodes, $M$ is for number of edges, $K$ is for degree of the graph, $K C V$ is the coefficient of variation for the degree, $K P$, is the degree of the "protagonist" of the story, $R$ is the degree assortavity and $C$ is the clustering coefficient

## 4 Future Work and Conclusion

This paper studies the content of manga in different genres through two lenses: emotional and social. Our results discuss some interesting trends revealed in the emotion sequence analysis and the character network analysis. However, these results are based on a limited set of data and requires further investigation to achieve a more substantial study of manga analysis. Hence, we propose the following future directions. An important bottleneck for emotion sequence analysis are the performances of the emotion recognition models, hence, we plan to experiment with different modeling architectures to improve emotion recognition in text and visual cues. We also propose deeper sequence analysis, especially inspired from bio-informatics. Solomon et al. [15] used such sequence analysis techniques to detect motifs in conversations. To conclude, our results invite a further analysis of manga through various lenses.

## References

[1] Toni Johnson-Woods. Manga: An anthology of global and cultural perspectives. Bloomsbury Publishing USA, 2010.

[2] Ryota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, and Yusuke Matsui. Towards fully automated manga translation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, pp. 12998-13008, 2021.

[3] Minshan Xie, Menghan Xia, Xueting Liu, Chengze Li, and Tien-Tsin Wong. Seamless manga inpainting with semantics awareness. ACM Transactions on Graphics (TOG), Vol. 40, No. 4, pp. 1-11, 2021.

[4] Ziyuan Liu, Ye Ding, Qi Wan, Zhiyang He, and Qing Zhang. Generating manga images from sketch via gan. 2022.

[5] Giancarla Unser-Schutz. Influential or influenced? the relationship between genre, gender and language in manga. Gender and Language, Vol. 9, No. 2, pp. 223-254, 2015.

[6] Kashin Sugishita and Naoki Masuda. Social network analysis of manga: similarities to real-world social networks and trends over decades. Applied Network Science, Vol. 8, No. 1, p. 79, 2023.

[7] Kiyoharu Aizawa, Azuma Fujimoto, Atsushi Otsubo, Toru Ogawa, Yusuke Matsui, Koki Tsubota, and Hikaru Ikuta. Building a manga dataset "manga109" with annotations for multimedia applications. IEEE MultiMedia, Vol. 27, No. 2, pp. 8-18, 2020.

[8] Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, Vol. 76, No. 20, pp. 21811-21838, 2017.

[9] James A Russell. A circumplex model of affect. Journal of personality and social psychology, Vol. 39, No. 6 , p. 1161, 1980.

[10] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp. 61056114. PMLR, 2019

[11] Saif Mohammad. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 174-184, 2018.

[12] Haruya Suzuki, Yuto Miyauchi, Kazuki Akiyama, Tomoyuki Kajiwara, Takashi Ninomiya, Noriko Takemura, Yuta Nakashima, and Hajime Nagahara. A japanese dataset for subjective and objective sentiment polarity classification in micro blog domain. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 7022-7028, 2022.

[13] Mark EJ Newman. Assortative mixing in networks. Physical review letters, Vol. 89, No. 20, p. 208701, 2002.

[14] Duncan J Watts and Steven H Strogatz. Collective dynamics of 'small-world' networks. nature, Vol. 393, No. 6684, pp. 440-442, 1998

[15] Denise Haunani Solomon, Susanne Jones, Miriam Brin- berg, Graham D Bodie, and Nilam Ram. Using sequence analysis to identify conversational motifs in supportive interactions. Journal of social and Personal Relationships, Vol. 39, No. 10, pp. 3155-3179, 2022.

## A Appendix

## A. 1 Statistic Results from Emotion Sequence Analysis

Table 2 Statistical results across all genres. page_arousal, page_valence, page_text, arousal_text, arousal_valence, valence_text are correlation coefficients between both variables. valence_peaks, valence_valleys, arousal_peaks, arousal_valleys are number of peaks and valleys for valence and arousal respectively.

## A. 2 Genre-based Comparison of Character Network Graph Features

Figure 7 Number of nodes and edges divided by genre.

Figure 8 Degree $(K)$, Coefficient of variation of degree $(K C V)$ and protagonist normalised degree $(K P)$ divided by genre.