Tohoku University Cognitive Neuroscience Application Center

What mechanisms support reading comprehension in a second language (L2), and how does multimodal information—such as simultaneous audio and text—enhance understanding? Moreover, how do learners with higher working memory make use of this information? To answer these questions, we conducted an fMRI study in collaboration with Professor Kajiura from Nagoya City University, presenting our findings at the Society for the Neurobiology of Language conference held in Brisbane, Australia, from October 24-26, 2024.
In this study, Japanese university students processed information in their native language (L1) and L2 under three conditions: audio-only, text-only, and simultaneous audio-text input. We observed their brain activity and examined the roles of working memory and attention inhibition abilities. The results showed that, during L2 reading, brain regions involved in integrating audio and visual information (such as the superior temporal gyrus and angular gyrus) were more active than in L1 reading, suggesting that multimodal input supports comprehension in L2. Additionally, learners with higher working memory showed stronger integration of information, indicating that personalized multimodal strategies could be especially effective for L2 learning. These findings highlight the potential of multimodal approaches in second language acquisition, attracting considerable interest from conference attendees. (Jeong)

Society for the Neurobiology of Language (neurolang.org)

Mayumi Kajiura, Chunlin Liu, Diego Dardon, Motoaki Sugiura, Hyeonjeong Jeong
Exploring Brain Activity and Multimodal Input: Effects on First and Second Language Learning and the Role of Working Memory
Society for the Neurobiology of Language, 16th Annual Meeting, Brisbane, Australia【 Poster 】