Musical interventions, both instrumental and with lyrics, have long been discussed as affecting language acquisition, specifically second language (L2) acquisition (Ludke et al, 2014). Indeed, prior research highlights its potential benefits for pronunciation, prosody, vocabulary, listening comprehension, memory, and learner motivation (Patel et al. 2011). Although musical interventions have shown promise in L2 acquisition, existing research treats all interventions as a relatively unified category, and little is known about which intervention types or specific musical features are the most effective for particular linguistic aspects and learner profiles.  Hence, we propose a conceptual framework for adaptive singing-based L2 interventions that integrates music, cognition, learner profiles, and multimodal generative AI.

The  project is situated at the intersection of musical interventions for language learning, cognitive and neural mechanisms linking the processing of music and language , and AI-supported personalization of interventions. Rather than presenting or evaluating a specific generative AI system, the paper outlines the conceptual and methodological foundation for a research-informed approach to AI-supported musical, especially singing-based, interventions in L2 acquisition. The central questions guiding the project are: 

1.     Can musical interventions be defined and modelled

 a) to target specific aspects of L2 acquisition and 

 b) in relation to a particular learner profile?  

2. How can AI-supported personalization architectures be designed to adapt musical L2 interventions to individual learner profiles and linguistic needs?

Age of acquisition (AoA) can explain much of the variation in L2 acquisition (e.g., Abrahamsson, 2018; Abutalebi & Clahsen, 2018; Weber-Fox & Neville, 1996, 2001). Especially in L2 production (i.e., pronunciation) and processing of phonology (Archila-Suerte et al., 2015). Other variables that relate to L2 proficiency includes socioeconomic status (SES) and cross linguistic influence (CLI) (Andersson & Newman, 2022). More specifically, SES background typically relates to vocabulary acquisition that in turn predicts development of syntax (Weislader & Fernald, 2013), while CLI relates to all aspects of the language (phonology, vocabulary, syntax, and morphosyntax). That is, when L2 is similar to the first language (L1) the particular shared aspect is easier to acquire, and processing of the aspects will be more native like than when non shared. Accordingly, if a learner profile is known (e.g., AoA, SES, and CLI) then some challenges could be predicted.  

One  reason for musical interventions to affect L2 acquisition is the shared underlying neuronal mechanisms for factors such as, rhythm, temporal prediction, prosody, auditory pattern processing, and vocal-motor integration. Indeed, converging evidence suggests partial overlap between neural systems underlying music and language processing, particularly in domains related to these factors  (e.g., Gluschko et al., 2016; Patel A. 2011). Other reasons are the repetition of vocabulary in music and that music provides an enhanced context for a particular word (e.g., Chen, 2020). Singing, in particular, occupies a productive position between music and language acquisition as it combines melody, rhythm, articulation, repetition, affect, and verbal meaning. Accordingly, singing shows a strong potential for affecting L2 proficiency. However, interventions may have different outcomes depending on the specific learner. Indeed, interventions may need to be personalized according to the learner profile including musical expertise.

Recent developments in multimodal generative AI have made it increasingly feasible to generate music and lyrics dynamically (Yu et al., 2024). These developments create new possibilities for interventions that are not only engaging but also targeted to specific challenges predicted from a learner profile. 

Prior studies have examined a range of interventions including singing, rhythm-based training, instrumental music, musical listening, computer-based musical exercises, and melody-supported memorization. These interventions may plausibly support different aspects of L2 acquisition, such as phonology and prosody, vocabulary, syntax, and morpho syntax. However, the field remains fragmented, and it is not yet clear which types of interventions are most suitable for which linguistic aspects.

To address this problem, the project proposes a framework that brings together findings from L2 acquisition, music cognition, and artificial intelligence. The first aim is to identify how specific musical features and intervention types relate to particular aspects of L2 acquisition. For example, rhythm and metrical structure may be especially relevant for prosody, stress patterns, and fluency; melody and repetition may support vocabulary retention and formulaic language; singing may support pronunciation by combining auditory perception, motor production, and affective engagement. The second aim is to relate these intervention types to learner profiles, including L1, AoA, musical expertise, L2 exposure, self-reported L2 proficiency, and potentially cognitive or clinical factors such as dyslexia, ADHD, or a language disorder.

The paper outlines a conceptual workflow for personalized musical intervention design. The workflow begins with an initial screening instrument in order to create a learner profile, as well as user preferences regarding music. This information is then mapped onto a set of empirically grounded intervention categories targeting specific linguistic aspects. Within a closed-loop adaptive system, multimodal generative AI can subsequently be used to produce the intervention.

The contribution of the project is its attempt to connect theoretical knowledge about the relationship between music, singing, and language with practical applications in language education and rehabilitation. By combining  learner characteristics  and personalized music interventions through AI, the project moves beyond general claims that music supports language acquisition. Instead, it seeks to specify how, for whom, and under what conditions musical interventions may support particular aspects of L2 acquisition and development. In doing so, we offer a starting point for future empirical studies and for the development of adaptive, evidence-informed tools for music-based language interventions. 

Reference list

Abrahamsson, N. (2018). But first, let's think again! Bilingualism: Language and Cognition, 21(5), 906-907. https://doi.org/10.1017/S1366728918000251  

Abutalebi, J., & Clahsen, H. (2018). Critical periods for language acquisition: new insights with particular reference to bilingualism research. Bilingualism: Language and Cognition, 21(5), 883-885. https://doi.org/10.1017/S1366728918001025  

Andersson, A., & Newman, A. (2022). The roles of age of acquisition, proficiency, and first language on second language processing. In A. Newman & G. Grossi (Eds.), Changing Brains: Essays in Honor of Helen J. Neville (pp. 57-77). Taylor & Francis. https://doi.org/10.4324/9780429342356  

Archila-Suerte, P., Zevin, J., & Hernandez, A. E. (2015). The effect of age of acquisition, socioeducational status, and proficiency on the neural processing of second language speech sounds. Brain and Language, 141, 35-49. https://doi.org/https://doi.org/10.1016/j.bandl.2014.11.005  

Chen, I. S. J. (2020). Music as a Mnemonic Device for Foreign Vocabulary Learning. English Teaching & Learning, 44(4), 377-395. https://doi.org/10.1007/s42321-020-00049-z  

Glushko, A., Steinhauer, K., DePriest, J., & Koelsch, S. (2016). Neurophysiological correlates of musical and prosodic phrasing: shared processing mechanisms and effects of musical expertise. PLoS One, 11(5), e0155300. https://doi.org/10.1371/journal.pone.0155300  

Ludke, K. M., Ferreira, F., & Overy, K. (2014). Singing can facilitate foreign language learning. Memory & Cognition, 42(1), 41–52. https://doi.org/10.3758/s13421-013-0342-5

Patel, A. D. (2011). Why would musical training benefit the neural encoding of speech? The OPERA hypothesis. Frontiers in psychology, 2, 142. 

Weber-Fox, C., & Neville, H. J. (1996). Maturational constraints on functional specializations for language processing: ERP and behavioral evidence in bilingual speakers. Journal of Cognitive Neuroscience, 8(3), 231-256. https://doi.org/10.1162/jocn.1996.8.3.231  

Weber-Fox, C., & Neville, H. J. (2001). Sensitive periods differentiate processing of open- and closed-class words: An ERP study of bilinguals. Journal of Speech, Language, & Hearing Research, 44(6), 1338-1353. https://doi.org/10.1044/1092-4388(2001/104)  

Weisleder, A., & Fernald, A. (2013). Talking to children matters: early language experience strengthens processing and builds vocabulary. Psychological Science, 24(11), 2143-2152. https://doi.org/10.1177/0956797613488145  

Yu, J., Wu, S., Lu, G. et al. Suno: potential, prospects, and trends. Front Inform Technol Electron Eng 25, 1025–1030 (2024). https://doi-org.proxy.lnu.se/10.1631/FITEE.2400299