https://orcid.org/0009-0007-9704-1931
Figures
Abstract
This study aimed to investigate how neural activation patterns differ between pianists and non-pianist musicians during piano performance tasks, using an LSTM-Autoencoder model applied to fNIRS data. A total of 22 participants, comprising both piano-majors and non-piano-majors, were involved in this study. Each participant’s performance data, collected during Score-based playing and improvisation tasks, was analyzed using a Long Short-Term Memory (LSTM) Autoencoder model. Reconstruction errors in specific brain channels, measured through Functional Near-Infrared Spectroscopy (fNIRS), showed group-level patterns—particularly in channels 1 and 15 (app. BA 45L, BA 45R)—but these differences did not reach statistical significance in inferential tests. These findings are therefore reported as exploratory and suggest distinct neural engagement patterns associated with varying levels of musical expertise.
Citation: Kang HJ, Lee S-c, Hwang Y, Ham J, Park M, Park K-m (2025) AI based comparison of prefrontal cortex activity between piano-majors and non-piano-major musicians during score-based playing and Motif-improvisation. PLoS One 20(12): e0337458. https://doi.org/10.1371/journal.pone.0337458
Editor: Noman Naseer, Air University, PAKISTAN
Received: March 20, 2025; Accepted: November 7, 2025; Published: December 26, 2025
Copyright: © 2025 Kang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All de-identified data underlying the findings of this study (including preprocessed channel-wise fNIRS time series, task markers, channel mapping, and LSTM-AE–derived ROI indices), together with the analysis code and documentation, are publicly available at https://github.com/KmPark88/PianoPlaying. The repository includes a README and scripts to reproduce Fig 5 and Table 7 .
Funding: This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Education (MOE) (RS-2025-25400034) and by the Korea Institute of Energy Technology Evaluation and Planning (KETEP) grant funded by the Ministry of Trade, Industry and Energy (MOTIE) (RS-2024-00394769).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Musical improvisation is widely regarded as one of the most highly creative forms of musical performance, involving the spontaneous generation of novel musical sequences in real-time [1,2]. Prior neuroimaging work indicates large-scale involvement of prefrontal, premotor, and temporal cortices and, in many cases, relative downregulation of dorsolateral prefrontal cortex (DLPFC) alongside increased medial prefrontal engagement during improvisation—patterns interpreted as reduced top-down monitoring that facilitate creative flow [3–7]. However, most prior studies have relied on block-averaged contrasts or general linear modeling, analytical choices that may obscure rapid, transient neural dynamics characteristic of live performance, particularly in highly trained musicians [3,8].
Expert piano-majors—those who have received formal instruction in secondary education or earlier and continuously practiced their instrument—show pronounced neuroplastic changes in auditory–motor integration and cortical representation, as demonstrated by Bangert & Altenmüller [9] and Bangert et al. [10]. However, the majority of existing research has focused on broad contrasts between musicians and non-musicians or employed block-averaged analyses that obscure rapid, expertise-specific neural fluctuations [11,12]. Critically, no study has yet compared the fine-grained, time-resolved neural dynamics of expert piano-majors versus non-piano-major musicians during motif-based improvisation, a form of improvisation centered on the spontaneous elaboration of short, thematically significant musical fragments.
Functional near-infrared spectroscopy (fNIRS) offers a practical alternative, allowing continuous monitoring of cortical hemodynamics during naturalistic piano performance Nonetheless, its application to musical creativity remains limited and mostly exploratory, as previous fNIRS studies predominantly rely on traditional statistical methods that average signals over extended intervals, thus failing to capture rapid temporal dynamics critical for understanding live creative processes [13–16]. While prior studies have provided valuable insights into musical creativity using block-averaged statistical methods, these approaches are inherently limited in capturing rapid, transient neural dynamics that characterize live performance. Although deep learning techniques have recently been applied to EEG and fMRI data to explore neural patterns, including aspects of musical creativity [17,18] such methods have not yet been applied to fNIRS recordings during real-time musical improvisation. This gap is significant because fNIRS allows for naturalistic, continuous monitoring of cortical hemodynamics, making it uniquely suited for capturing fine-grained temporal patterns that traditional statistical analyses cannot resolve. By leveraging a sliding-window LSTM-AutoEncoder, our study addresses this critical limitation, enabling autonomous detection of subtle, time-resolved neural dynamics and highlighting the novelty and necessity of an AI-driven time-series approach for understanding musical creativity.
To address this gap, we introduce an unsupervised sliding-window LSTM-AutoEncoder applied to high-resolution fNIRS time-series data recorded from 15 channels of the NIRSIT Lite system. Using reconstruction errors as an index of neural novelty, this approach autonomously detects subtle, transient patterns without requiring labeled data, allowing sensitive identification of expertise- and task-dependent dynamics.
The objective of this study is to investigate neural dynamics in expert piano-majors, building on the foundational work of Kang [19]. Specifically, we apply an unsupervised LSTM-AutoEncoder model for time-series neural data analysis to compare motif-improvisation with matched score-based playing, to identify subtle neural deviations overlooked by conventional block-averaged methods. This approach advances both the neuroscientific understanding of musical creativity and the development of neuro-inspired AI models emulating human creative cognition.
Literature review
Research in neuroscience has particularly illuminated how various regions of the brain contribute to this complex task. Among these, the prefrontal cortex (PFC) has consistently been identified as a central hub, because it integrates executive control, working memory, and self-monitoring—functions that are indispensable for creative musical performance. Highlighting this pivotal role provides the rationale for beginning our literature review with the PFC. In particular, previous studies have indicated the importance of PFC in creative playing [4–6,19–22], and therefore this study will primarily explore the PFC regions.
Dorsolateral Prefrontal Cortex (DLPFC)
The DLPFC is central to executive functions, including working memory, decision-making, and cognitive control. Research by Limb and Braun [4], and Jo and Kang [22] showed a decrease in DLPFC activity during improvisation, associated with reduced self-monitoring and enhanced creative flow. This decrease facilitates a state of cognitive flexibility, allowing musicians to generate spontaneous musical ideas without the constraints of over-monitoring their performance. The transient hypofrontality theory proposed by Dietrich and Kanso [23] further supports this by suggesting that creative states involve a temporary downregulation of prefrontal cortex activity. Ferreri et al. [24] found a decrease in DLPFC while listening to music compared to silence.
Ventrolateral Prefrontal Cortex (VLPFC)
The VLPFC is implicated in processing complex, novel information and is crucial for adaptive behaviors. Located in the frontal lobes of the brain, primarily involved in cognitive functions like processing language and managing speech production. The VLPFC contributes broadly to language comprehension and is involved in higher-order cognitive processes, such as reasoning, planning, and problem-solving. Previous studies have shown that the ventrolateral prefrontal cortex (VLPFC) plays a critical role in integrating diverse information during creative tasks such as musical improvisation. This region is particularly active when musicians must suppress habitual responses and generate novel solutions, which are essential for high-level improvisation [5,7].
Inferior Frontal Gyrus (IFG) and Broca’s Area
The Inferior Frontal Gyrus (IFG) is part of the VLPFC and includes areas such as Broca’s area, specifically involved in language production and processing, illustrating the specific roles these areas play within the broader prefrontal cortex functionalities [23]. Within the IFG, Broca’s area, predominantly located in the left hemisphere and including Brodmann areas 44 and 45, is critically involved in language production, underlining the region’s importance in speech articulation and the complex motor functions required for producing speech. The various studies referenced in the provided citations confirm and expand upon this idea by highlighting the involvement of Broca’s area (specifically the inferior frontal gyrus) in both language processing and musical creativity [25–27]. During improvised rock guitar performances, Tachibana [5] demonstrated increased activity in Broca’s area and its right hemisphere homologue. This activity suggests that Broca’s area is involved in managing complex, non-verbal auditory processing during musical improvisation, similar to how it functions in language syntax. This highlights the overlap between language and music in cognitive processing, where both domains require the integration of complex, structured sequences.
Methodology
Participants
As shown as Table 1, participants in this study were a group of 22 classically trained musicians, officially majoring in music institutes such as music colleges or conservatoires from higher than high school level. They were recruited through targeted announcements on music college websites and societies focused on piano pedagogy. This recruitment strategy ensured that all participants possessed a strong foundation in classical music, allowing for an in-depth analysis of neurophysiological aspects among musicians with varying skill levels. The cohort consisted of 4 males and 18 females, with ages ranging from 19 to 58 years (M = 35.5, SD = 9.99). Participants’ years of piano training varied from 1 to 50 years (M = 16.51, SD = 10.23). Sixteen participants were piano-majors, while the remaining six were non-piano majors. As shown as Table 2, for piano-majors (n =16), the mean age was 35.69 years (SD = 8.87), with piano training ranging from 10 to 50 years (M = 21.4, SD = 10.81). As shown as Table 3, non-piano-major musicians (n =6) had an average age of 35 years (SD = 14.25), with piano training ranging from 10 to 50 years (M = 5.33, SD = 3.50). In the study group of non-piano-major musicians (non-piano-majors), there are 4 females and 2 males. The ages range from 19 to 51 years, with an average age of 35 years and a standard deviation of approximately 14.25. The years of piano training range from 1 to 10, with an average of approximately 5.33 years and a standard deviation of about 3.50.
Experimental design
A 60-minute workshop was conducted by a professional improvisation instructor, who led all participants through exercises focusing on arpeggio-based improvisation, thematic improvisation, and emotional expression. Previous research indicates that shorter sessions may not provide enough time for classical musicians, often less experienced with improvisation—to overcome anxiety and build creative confidence [28]. The workshop aimed to familiarize participants with improvisation techniques and reduce performance anxiety. We tested score-based playing, motif improvisation, and free improvisation in random order. In this study, we performed a comparative analysis only between score-based playing and motif improvisation, while previous studies have demonstrated significant neural differences between score-based playing and free improvisation [19,22]. Table 4 summarizes the timing of task and rest periods. Rest periods were inserted between experimental tasks to allow hemodynamic activity to return to baseline, ensuring a clear contrast between task-induced activations and the neutral state [24,29].
Score-based playing.
Score-based playing refers to performance practice based on the faithful realization of notated musical scores. We allowed participants to practice without memorization for 3 days and experiment while playing with the score.These functions as a baseline to compare with motif-improvisation [30,31]. We selected Muzio Clementi’s Sonatina in C Major for both piano-majors and non-piano-majors, considering its accessible classical sonata form and 4/4-time signature. The piece was suitable for mastery within a three-day preparation period, allowing participants to familiarize themselves with the music and focus on performance without the cognitive load of memorization. This preparation time struck a balance between familiarity and tension, avoiding sight-reading, which could introduce anxiety [32], or memorization, which could lead to over-familiarity and hinder improvisational focus [33].
Motif-improvisation.
Motif refers to a short musical fragment or sequence that possesses thematic significance and symbolic meaning within a compositional context. It functions as a foundational unit that encapsulates the identity of a musical work. Motif-Improvisation, therefore, denotes a performance practice in which a single motif is initially presented and subsequently developed through spontaneous, creative elaboration in real time [34].
Motif-based improvisation was designed to provide a semi-structured creative task that bridges fully composed performance and free improvisation. This format avoids the confounds introduced by large variations in tempo, meter, and harmonic language inherent in jazz or free improvisation, which can significantly alter both motor execution and neural dynamics [16,35]. Although constrained, motif-based improvisation engages key cognitive processes such as auditory imagery, predictive timing, and hierarchical sequencing, which are known to activate prefrontal and language-related brain areas, including Broca’s area [36,37]. The first two bars of the Score-based playing piece serve as the motif material for improvisation. This design was inspired by classical thematic development practices and adapted to be more accessible for non-improvisers. Using C Major and a right-hand–focused motif reduces technical complexity, thereby allowing performers to focus more on creative transformation and expressive intent rather than motor accuracy. This approach aligns with findings that constrained improvisation can still evoke complex neural and creative processes, especially in classically trained musicians unfamiliar with improvisation idioms [38].
As shown in Fig 1, for Score-based playing, participants play through for 60 seconds. For Motif-Improvisation, participants play the first two bars and keep playing improvisationally for 60 seconds.
fNIRS data
Fig 2 (a) represents the functional division of PFC where the channel of NIRSIT Lite measures. (b) is approximately on the shown brain surface [39]. Measurements will be taken from approximately 15 channels from Brodamann Area (BA) 10, 46, 45. (c) shows Brodamann’s area from lateral view of the brain including the orbitofrontal cortex (BA 11, purple), ventrolateral prefrontal cortex (VLPFC, BA 44/45/47, pink), inferior frontal junction (IFJ, gray), dorsolateral prefrontal cortex (DLPFC, BA 9/46, light blue), anterior prefrontal cortex (BA 10, dark blue), and dorsal premotor cortex (BA 6, dark green) [28,40]. fNIRS and MIDI recordings were conducted using the NIRSIT LITE of OBELAB, Inc and Logic Pro. During the fNIRS scans, the participants played a YAMAHA P125 keyboard with 88 keys. MIDI data was collected simultaneously. During scanning, the participants’ performance on the keyboard was recorded with their consent. Auditory feedback from the piano was provided to the participants through speakers. They were instructed not to move their heads during the playing. During all three conditions, participants were instructed to direct their gaze forward. We collected neural data using the fNIRS system, focusing on the PFC, known for its involvement in cognitive flexibility, decision-making, and creative processes [28]. Functional near-infrared spectroscopy (fNIRS) is a non-invasive technique ideal for studying PFC activity in musicians, as it effectively captures hemodynamic responses and allows for brain activity measurement in naturalistic settings, thus enhancing ecological validity [24,41]. Its sensitivity to cognitive control and creativity makes it particularly useful for investigating the neural mechanisms underlying musical improvisation and performance [24,42,43]. Data were recorded at a sampling rate of 8 Hz (8 data points per second). Hemodynamic responses were assessed using changes in oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) concentrations, filtered in real-time to remove motion artifacts. These physiological signals were synchronized with MIDI performance data, ensuring alignment between brain activity and musical execution. For real-time fNIRS data processing, we used OBELAB’s proprietary software, designed specifically for the NIRSIT LITE system. This software allowed for the efficient acquisition and processing of neural data, providing seamless integration with MIDI data to ensure precise synchronization between brain activity and musical performance [15,39,44–46].
Data set
An LSTM-AutoEncoder model is used to facilitate this comparison, wherein the model is trained on the outcomes of Score-based playing, and the reconstruction error for Motif-Improvisation is subsequently measured. This methodology identifies channels exhibiting anomalies, with elevated reconstruction errors indicating the presence of variations in the data patterns. Due to the varying lengths of the collected data, all data samples were adjusted to a uniform length of 1800 time points(steps) for (the purpose of) training.
Neural network model structure
Previous studies have demonstrated that LSTM-AutoEncoder models generally outperform standard AutoEncoder models when dealing with large volumes of temporal data, particularly in anomaly detection tasks [17,28,47]. This is due to LSTM’s ability to capture long-term dependencies in time series data [48]. In this study, we also leverage the advantages of the LSTM-AutoEncoder to implement a model suitable for the characteristics of time series data. Traditional statistical methods, such as mean signal comparison or GLM-based analysis, have been widely used in fNIRS studies to evaluate brain activation differences across conditions [49]. However, these approaches often rely on assumptions of stationarity and may fail to capture nonlinear temporal dependencies or subtle individual variability, especially in dynamic tasks like musical improvisation. In contrast, deep learning models such as LSTM-AutoEncoders can learn complex temporal patterns directly from data without requiring handcrafted features [50,51], making them particularly suitable for identifying latent deviations in neural activity. The architecture of the LSTM-AutoEncoder utilized in this research is illustrated in Fig 3. Considering the temporal nature of the collected data, the LSTM-AutoEncoder model was implemented by combining Long Short-Term Memory (LSTM), which is suitable for time series data learning, with the AutoEncoder, a model designed for unsupervised learning [23,25,52–54]. The AutoEncoder reduces the input data via the encoder section and reconstructs it using the decoder section. The discrepancy between the reconstructed data and the original data is called the reconstruction error. Leveraging these characteristics, AutoEncoder models are utilized in anomaly detection to identify patterns that differ from the training data [55–57]. In the context of neuroimaging, recent studies have demonstrated the utility of LSTM-based AutoEncoder frameworks in detecting abnormal temporal patterns in EEG and fNIRS signals, outperforming conventional methods in sensitivity and temporal resolution [58–60]. Our approach builds on these findings by applying an LSTM-AutoEncoder to identify neural deviations during improvisational tasks relative to baseline (score-playing) conditions. The AutoEncoder was first trained on Task A (Score-Playing) data to establish a normal baseline. Task B (Improvisation) data were then tested for deviations using reconstruction error analysis. In the encoder section of Fig 3, the input data is processed sequentially through LSTM layers, which consist of 64 and 32 units and utilize ReLU as the activation function. The data is then compressed into a latent space that reflects low-dimensional compressed data through Dense layers with 32 and 16 units. The data that has been compressed by the encoder is restored through the decoder. It is transformed into an appropriate sequence length for the decoder using the RepeatVector and subsequently processed through a Dense layer. Finally, it is reconstructed through the LSTM layer with 32 units, and the reconstruction error is assessed.
Experimental results
This study utilized the Python programming language for data preprocessing and model implementation, while the TensorFlow framework was employed for training and optimizing deep learning models. Designed for efficient neural network computations, TensorFlow provides GPU acceleration and high scalability, which contributed to improving the efficiency of model training and evaluation during the experiments. This enabled the effective exploration of various neural network architectures and facilitated performance comparisons throughout the study
Hyperparameters.
The hyperparameter values used in the learning model are shown in Table 5. In this study, mean square error (MSE) was used as the loss function. MSE is a metric that measures the mean squared difference between the predicted value and the actual value of the model, as defined in (1).
Summary of the hyperparameters used to train the LSTM-AutoEncoder model.
Mean Squared Error (MSE) is particularly sensitive to large discrepancies, effectively emphasizing substantial differences between predicted and actual values. This characteristic is instrumental in enhancing overall predictive accuracy by minimizing significant errors within the model. Furthermore, MSE’s differentiable nature is advantageous for optimization algorithms, such as gradient descent, facilitating smoother convergence. Consequently, MSE is a suitable loss function for a variety of machine learning models, including those used in regression problems and neural network training. The experiments were conducted five times for each of the window sizes 15, 20, and 25, and the results were averaged and summarized in Table 6. Among these, the window size of 20 exhibited the lowest reconstruction errors, with the train and test reconstruction errors being approximately 0.03 and 0.09, respectively. Therefore, this window size was used in the subsequent analysis.
Training results.
As a result of training on Task A for all participants, it was observed that both the training loss and validation loss converged, as illustrated in Fig 4. As shown in Table 7, the relatively high training loss is approximately 0.039, while the validation loss is about 0.245. This may be attributed to the complexity of the data.
Analysis results.
As shown in Table 8, when the window size was set to 20, reconstruction errors of approximately 0.25 and 0.18 were observed in channels 1 and 15, respectively.
Average Channel-wise reconstruction error for the entire dataset.
As an additional analysis of Result 1, we examined the differences in channels between piano-majors and non-piano-majors. As shown in Table 9, with a window size of 20, significant reconstruction errors were observed in channels 1 and 15 for piano-majors. In contrast, for non-piano-majors, reconstruction errors of approximately 0.17 and 0.135 were observed in channels 2 and 5, respectively.
Average Channel-wise reconstruction error of data from pianists and non-pianists for Task A and B.
Discussion
Comprehensive analysis
In Fig 5(a), the LSTM-AutoEncoder model trained on Task A with a window size of 20 showed the largest discrepancies in channels 1 and 15 for pianists’ Task B. In contrast, Fig 5(b) displays the average reconstruction errors for non-pianists, showing relatively large discrepancies in channels 2 and 5. However, due to the insufficient data for non-pianists, the results were highly variable, making it difficult to draw definitive conclusions. Fig 5(c) provides a visual comparison of the average reconstruction errors across different channels for both pianists and non-pianists. As shown in the figure, channels 1 and 15 exhibited differences greater than 0.10. Nevertheless, inferential tests conducted on an a priori BA45 ROI (channels 1 and 15) with robust summaries (median log-MSE, trimmed mean, and 90th percentile) did not reach statistical significance (Welch’s t-test, Mann–Whitney U, and label-permutation; all p > 0.28), and effect sizes were small-to-moderate. On the other hand, the red dots indicate that the average reconstruction errors for non-pianists were larger, though the differences were not as pronounced. These descriptive patterns should therefore be interpreted as exploratory.
(a) Average for pianists, (b) average for non-pianists, (c) overall average across all participants.
To formally assess the descriptive patterns in Fig 5, we (channels 1 and 15) used three robust summaries of the LSTM-AutoEncoder reconstruction error (median log-MSE, 10% trimmed mean log-MSE, and 90th-percentile log-MSE). None of the tests reached statistical significance at the group level (Welch’s t, Mann–Whitney U, and label-permutation with 10,000 iterations; all p > 0.28). For the most discriminative summary (90th-percentile log-MSE), we observed p = 0.283 (Welch’s t), p = 0.494 (Mann–Whitney), and permutation p = 0.297, with small-to-moderate effect sizes (Cohen’s d = 0.521; Cliff’s ; pianists n = 16, non-pianists n = 6). These findings indicate that the visually apparent discrepancies—particularly at channels 1 and 15—should be interpreted as exploratory trends rather than conclusive group effects, likely reflecting limited power due to the small, imbalanced sample and inter-subject variability in fNIRS signals.
Finding of neural mechanisms in improvisation
The elevated reconstruction errors observed in Channel 1 (BA45R) during improvisation (although not statistically significant in our inferential analyses) suggest a heightened neural engagement in tasks that require cognitive flexibility, motor inhibition, and action updating—functions strongly associated with the right VLPFC. This area’s involvement in improvisation highlights its role in managing the unpredictability and spontaneity inherent in creative tasks. Similarly, the findings in Channel 15 (BA45L) indicate that Broca’s area, traditionally known for its role in language production, is also critically engaged during musical improvisation. We note that these interpretations are exploratory given the non-significant statistical tests, but they suggest that the left IFG plays a pivotal role in the creative manipulation of musical elements, aligning with its known functions in syntactic processing and complex cognitive tasks. Interestingly, these findings support a growing body of research suggesting that improvisation, whether in music or language, involves complex and dynamic neural processes across both hemispheres of the brain, fostering the flexibility and adaptability essential for creative expression. Our findings align with those of Tachibana [5], who also observed significant activation in BA45L and BA45R during musical improvisation, particularly in improvisation as compared to more formulaic playing. The above study finds the averaged hemodynamic responses in the BA45 region during Improv and Formulaic task conditions.
The reason why piano-majors tended to show more consistent (though not statistically significant) neural response, especially in the context of creative musical tasks, can be attributed to their extensive training. Piano-majors undergo rigorous practice that fine-tunes their motor and cognitive skills, particularly those involving the integration of complex motor sequences and auditory feedback. This training likely enhances the connectivity and efficiency of neural networks associated with both musical and linguistic processing, making them more adept at engaging these regions during improvisation. This is in contrast to non-piano-majors, whose neural engagement in these regions was less consistent, likely due to their varied primary instruments, limited familiarity with the piano, and the small sample size. These factors introduce confounds that require further exploration, such as how differences in instrument familiarity influence the engagement of the prefrontal cortex in creative musical tasks. These factors introduce biases that make it challenging to detect stable neural responses during improvisation among non-piano-majors. The patterning in Channels 1 and 15, particularly in piano-majors, suggest a potential relationship between symmetrical brain structures and their connectivity. Since both Channel 1 and Channel 15 are located in the triangular part of the IFG, they likely share similar functional responsibilities, primarily related to language processing, both verbal and non-verbal, as well as cognitive control. Although they are located symmetrically on opposite hemispheres, their functional contributions could be asymmetrical. The left IFG (Channel 15) is more involved in language production and complex verbal tasks, while the right IFG (Channel 1) contributes to broader cognitive functions, including non-verbal communication and possibly emotional processing Studies have shown that high-level cognitive functions often require the coordination of both hemispheres [5,23]. The symmetrical activity might reflect the brain’s need to integrate various forms of information—such as timing, rhythm, and melody in the case of music—requiring efficient communication between the hemispheres. This is particularly true for tasks like improvisation, which involve both planned and spontaneous elements. This difference appears to be due to brain plasticity. Consistent practice has been shown to alter brain activity in both healthy individuals and patients with brain damage or neurological conditions [61–63]. More specifically, various studies have explored the effects of musical training on both structural and functional changes in the brain, highlighting its role in enhancing brain plasticity. These studies show that musical training increases connectivity between brain regions, improves linguistic and auditory abilities, and has positive effects on cognitive and neurophysiological functions [64–68]. Despite the unfamiliarity classical musicians may have with improvisational playing, training as a professional piano-major still helps bridge the gap between non-piano-major musicians during motif variation.
Conclusions
This study explored neural activity patterns between motif improvisation and score performance using an AI-based autoencoding method. The error-based perspective may reveal patterns not readily captured by simple activation/deactivation analyses. However, at the group level we did not observe statistically significant differences under a priori ROI analyses with robust summaries and permutation testing; effect sizes were small to moderate. Given that music is a temporal art, it is natural for neural activity to fluctuate dynamically in response to musical features, and the error detection approach may provide a more nuanced analysis than a simple comparison of activation. Descriptively, pianists tended to show higher reconstruction errors in the right and left IFG triangularis regions (BA45R, BA45L), but these trends did not reach statistical significance in our dataset. In contrast, non-pianists showed greater variability, limiting interpretability. Accordingly, our findings are exploratory and should be interpreted with caution given the small and imbalanced sample and inter-subject variability in fNIRS signals. This work offers a complementary perspective on neural differences in creative musical tasks via error detection, and future research should validate these observations using within-subject designs, larger and balanced samples, and complementary baseline models.
References
- 1.
Pressing J. Improvisation: methods and models. In: Sloboda JA, editor. Generative processes in music. Oxford: Oxford University Press; 1988. p. 129–78.
- 2.
Berliner PF. Thinking in Jazz: the infinite art of improvisation. Chicago: University of Chicago Press; 1994.
- 3. Berkowitz AL, Ansari D. Generation of novel motor sequences: the neural correlates of musical improvisation. Neuroimage. 2008;41(2):535–43. pmid:18420426
- 4. Limb CJ, Braun AR. Neural substrates of spontaneous musical performance: an fMRI study of Jazz improvisation. PLoS ONE. 2008;3(2):e1679.
- 5. Tachibana A, Noah JA, Ono Y, Taguchi D, Ueda S. Prefrontal activation related to spontaneous creativity with rock music improvisation: a functional near-infrared spectroscopy study. Sci Rep. 2019;9(1):16044. pmid:31690744
- 6. Pinho AL, de Manzano Ö, Fransson P, Eriksson H, Ullén F. Connecting to create: expertise in musical improvisation is associated with increased functional connectivity between premotor and prefrontal areas. J Neurosci. 2014;34(18):6156–63. pmid:24790186
- 7. Beaty RE. The neuroscience of musical improvisation. Neurosci Biobehav Rev. 2015;51:108–17. pmid:25601088
- 8. Curzel F, Tillmann B, Ferreri L. Lights on music cognition: a systematic and critical review of fNIRS applications and future perspectives. Brain Cogn. 2024;180:106200. pmid:38908228
- 9. Bangert M, Altenmüller EO. Mapping perception to action in piano practice: a longitudinal DC-EEG study. BMC Neurosci. 2003;4:26. pmid:14575529
- 10. Bangert M, Peschel T, Schlaug G, Rotte M, Drescher D, Hinrichs H, et al. Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction. Neuroimage. 2006;30(3):917–26. pmid:16380270
- 11. Schlaug G, Jäncke L, Huang Y, Staiger JF, Steinmetz H. Increased corpus callosum size in musicians. Neuropsychologia. 1995;33(8):1047–55. pmid:8524453
- 12. Bengtsson SL, et al. Listening to rhythms activates motor and premotor cortices. Cerebral Cortex. 2009;19(3):703–13.
- 13. Ding K, Li J, Li X, Li H. Understanding the effect of listening to music, playing music, and singing on brain function: a scoping review of fNIRS studies. Brain Sci. 2024;14(8):751. pmid:39199446
- 14. Pinti P, Aichelburg C, Gilbert S, Hamilton A, Hirsch J, Burgess P, et al. A review on the use of wearable functional near-infrared spectroscopy in naturalistic environments. Jpn Psychol Res. 2018;60(4):347–73. pmid:30643322
- 15. Quaresima V, Ferrari M. Functional Near-Infrared Spectroscopy (fNIRS) for assessing cerebral cortex function during human behavior in natural/social situations: a concise review. Organizational Research Methods. 2016;22(1):46–68.
- 16. Chiarelli AM, Zappasodi F, Di Pompeo F, Merla A. Simultaneous functional near-infrared spectroscopy and electroencephalography for monitoring of human brain activity and oxygenation: a review. Neurophotonics. 2017;4(4):041411. pmid:28840162
- 17. Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp. 2017;38(11):5391–420. pmid:28782865
- 18. Halkiopoulos C, Gkintoni E, Aroutzidis A, Antonopoulou H. Advances in neuroimaging and deep learning for emotion detection: a systematic review of cognitive neuroscience and algorithmic innovations. Diagnostics (Basel). 2025;15(4):456. pmid:40002607
- 19.
Kang HJ. Prefrontal cortex brain activity during score playing versus improvisation: a comparative analysis and influencing factors. Seoul, South Korea; 2024.
- 20. Bashwiner D, Bacon D. Musical creativity and the motor system. Current Opinion in Behavioral Sciences. 2019;27:146–53.
- 21. López-González M, Limb CJ. Musical creativity and the brain. Cerebrum: The Dana Forum on Brain Science. 2012;:2.
- 22. Jo K-R, Kang H-J. Comparative analysis on prefrontal cortex activation during piano improvisation and interpretive piano performance using fNIRS. KOSSCOM. 2023;12(7):239–59.
- 23. Dietrich A, Kanso R. A review of EEG, ERP, and neuroimaging studies of creativity and insight.. Psychological Bulletin. 2010;136(5):822–48.
- 24. Ferreri L, Bigand E, Perrey S, Bugaiska A. The promise of Near-Infrared Spectroscopy (NIRS) for psychological research: a brief review. Année psy. 2014;114(03):537–69.
- 25. Ferrari M, Quaresima V. A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. Neuroimage. 2012;63(2):921–35. pmid:22510258
- 26. Kunert R, Willems RM, Casasanto D, Patel AD, Hagoort P. Music and language syntax interact in Broca’s Area: an fMRI study. PLoS One. 2015;10(11):e0141069. pmid:26536026
- 27.
Dhakal K. Brain functional and structural networks underpinning musical creativity.
- 28. Dolan D, Jensen HJ, Mediano PAM, Molina-Solana M, Rajpal H, Rosas F, et al. The improvisational state of mind: a multidisciplinary study of an improvisatory approach to classical music repertoire performance. Front Psychol. 2018;9:1341. pmid:30319469
- 29. Plichta MM, Herrmann MJ, Baehne CG, Ehlis A-C, Richter MM, Pauli P, et al. Event-related functional near-infrared spectroscopy (fNIRS): are the measurements reliable?. Neuroimage. 2006;31(1):116–24. pmid:16446104
- 30. Schön D, Anton JL, Roth M, Besson M. An fMRI study of music sight-reading. Neuroreport. 2002;13(17):2285–9. pmid:12488812
- 31. Mateos-Moreno D, Erlanson E. Should improvisation be regularly included in music lessons? A single-case quasi-experimental study exploring the differences in the electrical activity of the brain between musical improvisation and sight-reading. Education Sciences. 2023;13(2):191.
- 32.
Koelsch S. Neural correlates of music perception. In: Peretz I, Zatorre R, editors. The cognitive neuroscience of music. Oxford, UK: Oxford University Press; 2013. p. 115–34.
- 33.
Loui P. Temporal aspects of musical expectancy and creativity in improvisation. Performing time. Oxford University PressOxford; 2023. p. 283–96. https://doi.org/10.1093/oso/9780192896254.003.0028
- 34. Ezegwu EO, Onyebuchi NJ. Development of motifs in vocal and instrumental sections of Igbo popular music: an analysis of selected songs. International Journal of Arts and Humanities. 2021;10(4):1–15.
- 35. Beaty RE, Benedek M, Kaufman SB, Silvia PJ. Default and executive network coupling supports creative idea production. Sci Rep. 2015;5:10964. pmid:26084037
- 36. Brown S, Martinez MJ, Parsons LM. Music and language side by side in the brain: a PET study of the generation of melodies and sentences. Eur J Neurosci. 2006;23(10):2791–803. pmid:16817882
- 37. Liu S, Chow HM, Xu Y, Erkkinen MG, Swett KE, Eagle MW, et al. Neural correlates of lyrical improvisation: an FMRI study of freestyle rap. Sci Rep. 2012;2:834. pmid:23155479
- 38. Bashwiner DM, Wertz CJ, Flores RA, Jung RE. Musical creativity “Revealed” in brain structure: interplay between motor, default mode, and limbic networks. Sci Rep. 2016;6:20482. pmid:26888383
- 39.
OBELAB. NIRSIT channel information. [cited 2023 Oct 1]. https://www.obelab.com/info/notice.php
- 40. Hoshi Y, Tamura M. Dynamic multichannel near-infrared optical imaging of human brain activity. Optics Letters. 1993;18(15):954–6.
- 41. Ehlis A-C, Schneider S, Dresler T, Fallgatter AJ. Application of functional near-infrared spectroscopy in psychiatry. Neuroimage. 2014;85 Pt 1:478–88. pmid:23578578
- 42. Loui P. Rapid and flexible creativity in musical improvisation: review and a model. Ann N Y Acad Sci. 2018:10.1111/nyas.13628. pmid:29577331
- 43. Pinho AL, de Manzano Ö, Fransson P, Eriksson H, Ullén F. Connecting to create: expertise in musical improvisation is associated with increased functional connectivity between premotor and prefrontal areas. J Neurosci. 2014;34(18):6156–63. pmid:24790186
- 44. Rahman JS, Caldwell S, Jones R, Gedeon T. Brain melody interaction: understanding effects of music on cerebral hemodynamic responses. MTI. 2022;6(5):35.
- 45. Zaaimi B, Clay R, Shivakumar A, Rickert J, Jackson A. Brain-responsive music enables non-invasive, targeted and unobtrusive neurostimulation. bioRxiv. 2024:2024–07.
- 46. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, et al. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage. 2002;15(1):273–89. pmid:11771995
- 47. Aswolinskiy H, Wermter J. Weakly supervised learning with time-series neural data: a review. Neurocomputing. 2019;356:171–85.
- 48. Blázquez-García A, Conde A, Mori U, Lozano JA. A review on outlier/anomaly detection in time series data. ACM Comput Surv. 2021;54(3):1–33.
- 49. Tak S, Ye JC. Statistical analysis of fNIRS data: a comprehensive review. Neuroimage. 2014;85 Pt 1:72–91. pmid:23774396
- 50. Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J. Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng. 2019;16(5):051001. pmid:31151119
- 51.
Bashivan P, Rish I, Yeasin M, Codella N. Learning representations from EEG with deep recurrent-convolutional neural networks. 2015. https://arxiv.org/abs/1511.06448
- 52. Zorick M, Mandelkern LC. Multivariate unsupervised analysis of resting-state EEG using entropy and complexity measures. Brain Research. 2013;1492:45–52.
- 53. Nagel IE, Schumacher EH, Goebel R, D’Esposito M. Functional MRI investigation of verbal selection mechanisms in lateral prefrontal cortex. Neuroimage. 2008;43(4):801–7. pmid:18692142
- 54. Cheng S, Qiu X, Li S, Mo L, Xu F, Zhang D. Different roles of the left and right ventrolateral prefrontal cortex in cognitive reappraisal: an online transcranial magnetic stimulation study. Front Hum Neurosci. 2022;16:928077. pmid:35754771
- 55.
An J, Cho S. Variational autoencoder based anomaly detection using reconstruction probability. In: Special Lecture on IE. 2015. p. 1–18.
- 56.
Malhotra P, Vig L, Shroff G, Agarwal P. Long short-term memory networks for anomaly detection in time series. In: Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium, 2015. 89.
- 57.
Said Elsayed M, Le-Khac N-A, Dev S, Jurcut AD. Network anomaly detection using LSTM based autoencoder. In: Proceedings of the 16th ACM Symposium on QoS and Security for Wireless and Mobile Networks. 2020. p. 37–45. https://doi.org/10.1145/3416013.3426457
- 58.
Elessawy RH, Eldawlatly S, Abbas HM. A long short-term memory autoencoder approach for EEG motor imagery classification. In: Proceedings of the 2020 International Conference on Computation, Automation and Knowledge Management (ICCAKM). 2020. p. 79–84.
- 59. Liu Y, Yang F, Wu B. Compression of EEG signals with the LSTM-autoencoder via domain adaptation approach. Comput Methods Biomech Biomed Engin. 2024;:1–14. pmid:38686789
- 60. Radaideh MI, Pappas C, Walden J, Lu D, Vidyaratne L, Britton T, et al. Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders. Digital Signal Processing. 2022;130:103704.
- 61. Fontaine RG, Yang C, Dodge KA, Pettit GS, Bates JE. Development of Response Evaluation and Decision (RED) and antisocial behavior in childhood and adolescence. Dev Psychol. 2009;45(2):447–59. pmid:19271830
- 62. Moransard M, Dann A, Staszewski O, Fontana A, Prinz M, Suter T. NG2 expressed by macrophages and oligodendrocyte precursor cells is dispensable in experimental autoimmune encephalomyelitis. Brain. 2011;134(Pt 5):1315–30. pmid:21596769
- 63. Buchheim A, Labek K, Walter S, Viviani R. A clinical case study of a psychoanalytic psychotherapy monitored with functional neuroimaging. Front Hum Neurosci. 2013;7:677. pmid:24167481
- 64. Herholz SC, Zatorre RJ. Musical training as a framework for brain plasticity: behavior, function, and structure. Neuron. 2012;76(3):498–508.
- 65. Glück J, König S, Naschenweng K, Redzanowski U, Dorner L, Straßer I, et al. How to measure wisdom: content, reliability, and validity of five measures. Front Psychol. 2013;4:405. pmid:23874310
- 66. Drapeau J, Gosselin N, Gagnon L, Peretz I, Lorrain D. Emotional recognition from face, voice, and music in Dementia of the Alzheimer type. Annals of the New York Academy of Sciences. 2009;1169(1):342–5.
- 67. McKellar CE, Shatz CJ. Synaptogenesis in purified cortical subplate neurons. Cerebral Cortex. 2008;19(8):1723–37.
- 68. Skoe E, Kraus N. A little goes a long way: how the adult brain is shaped by musical training in childhood. J Neurosci. 2012;32(34):11507–10. pmid:22915097