Brain-to-brain communication during musical improvisation: a performance case study

Brain synchrony assessment through hyperscanning

Hyperscanning refers to the synchronous recording of brain activity from more than one individual simultaneously, and has been implemented to study dynamic similarities or differences between the brain signals of multiple participants engaged in interactive or social tasks.¹⁵ Such an approach holds promise in understanding the nature of cognitive traits during social interactions.¹⁵ Recent hyperscanning studies have documented traces of shared cognition, emergent during moments of social interaction, collaboration, competition, and in educational settings.^16,17 The study of neural synchrony between individuals provides an insight into human connectedness and may aid in the development of treatments for social cognition disorders such as autism.¹⁸ A desired outcome of hyperscanning is the development of neural biomarkers that track in real-time the quality or strength of shared cognitive states such as brain-to-brain communication, shared attention, message conveying, and high engagement during human interactions.

Indeed, recent studies on human interactions have analyzed shared brain dynamics during teamwork tasks,¹⁹ and cooperative/competitive interactions.¹⁶ It has been reported that neural synchronization increases when participants are interacting in cooperation, and it reduces when they are competing against each other. A hyperscanning study allowed the quantification of the synchronization between brain signals of infants and adults during gaze interactions, showing increased neural coupling during direct eye-contact.²⁰ Neural coupling between humans has also been associated to the degree of mutual pro-sociality, where higher synchronization reflects stronger social relationships,²¹ and likeliness of interpersonal bonding.²² Considering the aforementioned studies, by analyzing inter-brain activity, hyperscanning offers a quantitative assessment of the strength and quality of different types of social interactions.²³

Regarding neural synchrony metrics, among the most common are coherence,¹⁷ phase coherence,¹⁶ phase locking value (PLV) and phase locking index (PLI),¹⁵ Granger causality,²⁰ correlation,²¹ wavelet transform coherence (WTC),²² graph theory, and partial directed coherence (PDC).²³ Bispectrum is another, more recent, metric in hyperscanning literature,^19,24 and offers insights on temporal, spatial and spectral levels. The bispectrum of a signal is a high order spectra that reflects the degree of temporal synchronization and phase coupling between two time series at different frequencies.²⁵ The bispectrum offers additional insight when compared to other neural synchrony metrics, as it provides a more complete intuition on phase coupling,¹⁹ resonance, temporal synchronization and non-linear interactions²⁶ between any analyzed signal pair²⁵; rather than only focusing on phase-coupling or correlation. Moreover, bispectrum is feasible to implement in real-time applications, which makes it more attractive for closed loop brain-computer interface applications.¹⁹

The estimation of temporal synchronous activity of brain areas, within and across brains, at different frequencies is critical for understanding social interaction in various contexts, including musical interaction as shown in this study. Several approaches to quantifying these neural interactions have been proposed. The bispectrum method is typically used to detect nonlinear interactions and identify cross-frequency interactions in the EEG signals acquired during various cognitive states.²⁷ It has the added advantage that it reduces Gaussian noise as Gaussian processes have vanishing bispectra.²⁸ PLV (a popular algorithm used in hyperscanning studies) quantifies the strength of phase coupling; however, it yields false positive correlations in the presence of source signal mixing due to instantaneous field spread and volume conduction in EEG recordings.²⁹ Amplitude-correlation based connectivity measures, which are insensitive to zero-lag correlations, may still be affected by spurious interactions due to field spread in the vicinity of true interactions.²⁹ Thus, the bispectrum is deemed to provide the best quantification approach to nonlinear cross-frequency interactions in EEG.

Musical improvisation as a window to study Brain-to-brain synchrony

Studies on intra and inter neural synchrony between pairs of guitarists during musical improvisation have shown dynamical networks that connect different brain regions, depending on the situation and/or expectations, with involvement of the fronto-parietal region, as well as the somatosensory, auditory, and visual cortices.^30,31 The analysis of such obtained networks can be used to study the temporal dynamics of these interactions and providing a neurophysiological interpretation of the observed behavior. Considering the rich and complex interchange of cognitive processes necessary during collaborative artistic performances, its study using the hyperscanning approach is a valid approach to explore the shared neural cognitive traces that emerge from these interactions.^32,33

Collaborative, free musical production (improvisation) is a complex and rich form of social interaction,³⁴ it has also been described as a continuous process of generation and transformation of musical interaction,³⁵ and offers an interesting object of study for hyperscanning. Similarities between music and language have been observed previously in terms of the social interaction they entail; as described in,³⁶ a jazz improvisation can be interpreted as a conversation, and a good improvisation as a complex, meaningful conversation. Over recent years, there has been a growing interest in the study of improvisational, freely-moving, collaborative musical production in live performance settings.²³ Hyperscanning in the musical context can allow the observation of neural traits of dynamic processes. For example the patterns between musicians’ brain activity when performing cooperatively or not, as it has been reported that such actions create differences in their peripersonal space,³⁷ and in the rhythmical alignment of the overall performance.³⁸

This process of improvised production can be perceived as a creative act of communication: one that is complex, nuanced, and technical, integrating simultaneous cognitive processes together in real time. Musical improvisation involves complex but rapid interactions of several components, including the generation and evaluation of melodic, harmonic, and rhythmic pattern ideas on a fast time-scale within a performance.³⁹ Mobile brain-body (MoBI) imaging provides the tools for analyzing neural patterns in real-time for freely-moving participants,^1,2,40 with hyperscanning techniques that provide an experimental approach to assess non-verbal communication in musical performance.³² During an improvised performance, musicians interact with each other, making use of different skills such as creativity,⁴¹ emotional expression and perception,³⁴ self-organization,⁴² memory retrieval and procedural memory,⁴³ and integration of visual and auditory stimuli with complex and precise motor coordination.^44,45 Musical improvisation can also be considered as an 'in the moment' composition, where improvisers are able to generate immediate responses to the present musical environment, to manipulate ideas and sequences in a spontaneous manner.⁴⁶

Brain-to-brain synchrony between musicians in free-jazz improvisation

Among many music styles, this study is centered on jazz, which takes roots from a mixture of afrological (african-american), and eurological (classical and contemporary) music.⁴⁷ More specifically, the performance of “free jazz” is of interest to this study, due to its unique characteristics that convey freedom to the perfomers. It is not based on pre-defined musical components (e.g. melody, rythm and harmony), pushes boundaries of improvisational norms, avoids the existence of shared musical frameworks; and allow musicians to generate temporal coherent pieces using improvisation as its major element.⁴⁷

In a theoretical model to study group jazz improvisation, Biasutti and Frezza⁴⁸ identify the processes that are essential for creative musical improvisation: anticipation, use of repertoire, emotive communication, feedback, and flow. In Wopereis et al.,⁴⁹ 26 expert musicians provided statements about musical improvisation in two 10-min individual brainstorm sessions. The statements resulted in a 7-cluster concept map, with self-regulation as the central concept, and affect, risk-taking, ideal, basic skills, responsivity, and creation, as constituent concepts for improvisational expertise. Specifically for collaborative improvisation, monitoring, feedback, and evaluation must be performed in association with other musicians, with both generative and communicative attentional demands.⁵⁰ Another study on jazz improvisation remarks that shared intentions emerge on the fly, and their presence fosters acoustic and temporal coordination, as well as improving the quality of the performance, as perceived by the performers and listeners.¹³ Time perception is likewise important during improvisation, as noted by Refs. 51, 52 where theta band power was found to increase when time was perceived more precisely by an expert musician during a free jazz improvisation under ecological settings.

Recently, the predictive coding of music (PCM) model has been introduced to model how listeners form expectations which may be fulfilled or not, through perception, action, emotion and, over time, learning.⁵³ Under this model, musical interaction is guided by mutual reduction of prediction errors, evidenced by alpha-band intrabrain neural synchronization (phase-locking analysis) in a right-lateralized temporoparietal network, with a higher occurrence probability in mutually adapting dyads than in leader-leader dyads.⁵⁴ These models of music improvisation highlight the centrality of anticipation, self-regulation, generation and evaluation, with feedback and communication in joint performance.

Research aims of this case study

This article aims to contribute to the understanding of brain-to-brain communication during a collaborative musical improvisation between jazz musicians. In this case, brain-to-brain communication can be understood as the existent non-verbal communication (e.g., anticipation, planning, and actions such as taking turns) between dyads of participants (jazz musicians improvising),⁵⁵ that can be assessed by quantifying and analyzing the synchrony between their brain signals, recorded simultaneously using the hyperscanning approach.⁵⁶ A jazz performance incorporates each of the five elements of musical improvisation: anticipation, feedback, use of previous repertoire, emotive communication, and coordinated flow.⁴⁸ Moreover, in a free jazz performance, as described in,⁵⁷ a continuous process of evaluation is present, where musicians can decide to maintain or change the current theme; initiate or respond to a change; and to adopt, augment, or contrast a given idea.

While improvising, musicians can elaborate over (but are not constrained to) a composition’s underlying chord structure⁵⁸ and theme, with variations that incorporate multiple derivations from instantaneous decisions in real-world practice.⁵⁹ An important aspect of jazz performance and proficiency lies in the embodied cognition and motor memory.⁶⁰ However, most neuroimaging studies on musical improvisation have used functional magnetic resonance imaging (fMRI).^39,50 Lying down in an fMRI scanner alters spatial and visual perception,⁶¹ and restricts body movement, which limits the capability of fMRI studies to observe realistic musical performance given the importance of embodied cognition in the task (due to the continuous retrieval and processing of spatial, auditory, visual and somatosensory information).⁶⁰ Because mobile electroencephalography (EEG) does not impose movement constraints, (as compared to the ones imposed when lying down in an fMRI scan), and subsequently allows participants to naturally engage in creative production with minimal, instrumentational constraint, it may afford advantages in studying musical improvisation.^62,63

The current study examines the neural correlates of brain-to-brain communication of jazz musicians during collaborative musical improvisation through hyperscanning; and addresses the limited body of knowledge on collaborative musical improvisation in an ecologically-valid production, with freely-moving expert musicians, as they interact in a jazz performance with a live audience. Here, the presence of a live audience is important, as our cohort of musicians are accustomed to them, to the point that they become a relevant part of their performance. An inter-brain synchronization analysis was implemented, by estimating the bispectrum of EEG signals between musician pairs during collaborative improvisations as they performed for a live audience. Following the concept of ecological validity, the exquisite corpse method was adopted to obtain realistic collaborative improvised art pieces.^40,62 The exquisite corpse is a game played by surrealists, in which different artists integrate their contributions into a unique piece, taking turns to add their input in an iterative manner until completing a final piece with the contributions of all members.⁴⁰ Under this paradigm, the complete performance is formed by a multi-participant improvisational, free jazz piece formed by the creativity from each player.

Human participants

The experimental methods were approved by the Institutional Review Board of the University of Houston, and are in accordance with the Declaration of Helsinki. All participants provided written informed consent, including agreement for publication in online open-access publication of information obtained during the experiments such as data, images, audio, and video. Three male healthy adults (P₁, P₂ and P₃) volunteered for this study. Musicians P₂ and P₃ received (formal) musical instruction for 12 and 6 years, respectively, and P₁ (informal) for 6 years. To the date of the experiments, P₁, P₂ and P₃ had 31, 38, and 26 years of experience performing music, respectively. P₂ and P₃ were music educators at the University of Houston at the time of the experiment. The musicians performed jazz improvisation in a public event at the Student Center of the University of Houston while wearing the MoBI technology. Musicians P₁ and P₂ have a jazz musical background, whereas P₃ had a’classical music’ education. Musician P₁ played the drums, musician P₂ played the saxophone, and P₃ played using a soprano saxophone. P₁ and P₂ had performed jazz regularly together for 6 years, P₂ and P₃ had performed a concert together once before, and P₁ and P₃ had not performed together previously. The heterogeneity between participants' experience and familiarity was not voluntary targeted; however, it led to interesting results and interpretations, presented in Results and Discussion sections.

Equipment

High-density active electrode scalp EEG and electrooculography (EOG) recordings were obtained simultaneously for the three musicians during their musical performances. EEG data was wirelessly acquired using the 64 channel actiCAP (BP gel) electrodes along with the Brain Amp DC amplifer (actiCap system, Brain Products GmbH, Germany) at a sampling frequency of 1000 Hz. Electrode distribution follows the 10-20 international system. EEG data was online referenced to channel FCz on the superior region of the scalp. Four channels were used to record EOG data. Channels TP9 and TP10 were placed on the right and left temples, respectively, to record horizontal eye movement, whereas channels PO9 and PO10 were placed above and below the right eye, respectively, to record vertical eye movement. Electrode impedances were recorded prior to the experiment, and maintained below 60 kΩ following the manufacturer's guidelines in the Brain Vision Recorder software and User manual on impedance measurement (where impedance values above 60 kΩ are considered ‘bad’, and below 25 kΩ ‘good’).⁶⁴

Performances were recorded by three video cameras coupled with a Zoom H6 (https://zoomcorp.com/) audio recorder from a frontal, superior and lateral perspective. Audio was recorded in a single stereo file at 44100 Hz. Three Sterling ST31 FET condenser microphones (https://sterlingaudio.net/) were used to amplify the sound from each musician’s instrument during the live performance.

Experimental design

Musicians performed three 15 minutes improvisations (trials). Each trial was divided in three 5 minutes pieces (segments). In Segment 1, one musician was performing while the other two were listening. In Segment 2, a different musician joined the first, while the remaining musician listened to the performance of the other two. In Segment 3, the third musician joined and all participants performed together until the end of the trial. In a given segment, the musicians who are performing are referred to as the “active” musicians, whereas the musicians who are not performing are the “passive” musicians. At the beginning and at the end of the experiment, three blocks comprising of an EEG impedance check, a one-minute eyes open (EO) and a one-minute eyes closed (EC) were recorded.

In each trial, the order of the musicians joining at each segment was pseudo-randomized so that each musician entered one trial as the first, second or third player. Each musician was given a visual cue to signal their start time in the piece. Between one trial and the next, there were short pauses of 3-5 minutes, in which the audience clapped and the musicians prepared for the next trial.

Figure 1(a) shows the protocol for baseline measurements, and the order of musicians joining at each segment and trial. Figure 1(b) shows the locations of EEG electrodes with impedance higher than 25 kΩ at the start and at the end of the recordings, for all musicians. Figure 1(c) shows the setup of the instruments and microphones during the experiments, and the three musicians wearing the EEG caps. Figure 1(d) depicts, as an example, from top to bottom, the recorded raw EOG, EEG and audio signals obtained for the first five seconds of Segment 1 of Trial 1, when only P₃ is performing.

Figure 1. a) Impedance check, baseline (eyes open and eyes closed) measurements and performance times for each participant across the three improvisation trials. b) Impedance values larger than 25 kΩ across electroencephalographic (EEG) electrodes at the start and end of all experiments for the three participants. c) Experimental setup of musicians on stage wearing EEG caps and performing. From left to right: P₁ (drums), P₂ (saxophone) and P₃ (soprano saxophone). d) Representative electrooculography (EOG), EEG and sound recordings during musical performance of the first five seconds of Trial 1.

Three independent raters with training in music composition annotated the data. The annotators were not familiar with the researchers nor with the musicians involved in the performances; and were tasked to write annotations of the performance independently from each other, by watching a recording of the live performance. The annotators were asked to write a short description (e.g. “players are performing in sync”, “mirroring each other”, “performing in discord”), and the time each event happened. Table 1 shows sample descriptions from the annotators during one trial of musical improvisation.

At the beginning of each trial, video, audio and physiological signals were synchronized using manual event markers (i.e. pressing a button). Recordings from the three trials were obtained simultaneously using this procedure. Unfortunately, data transmission was interrupted from 4:25-5:25 of Trial 3 due to a loss in connection, which resulted in missing data. The events that happened in this period were therefore not included in the analyses.

Signal preprocessing

EEG signals were acquired at 1000 Hz and resampled to 250 Hz to reduce computational cost in subsequent calculations. Signals were bandpass filtered from 0.1 to 100 Hz using a 4^th order Butterworth filter to remove unwanted noise. The PREP pipeline from the EEGLAB package (https://sccn.ucsd.edu/eeglab/download.php) was used as the initial step to clean the data.⁶⁵ This procedure ensures the removal of power line noise, as well as a “true” average reference of the signals. EOG artifacts were removed from the EEG signals using an adaptive noise cancelling (ANC) framework, known as $H^{\infty }$ filter.⁶⁶ Raw EOG signals were used as input in the $H^{\infty }$ filter with parameters γ = 1.15 and q = 1e⁻¹⁰ for removal of eye blinks, eye motions, amplitude drifts and recording biases simultaneously. The obtained signals were further processed using the artifact subspace reconstruction algorithm (ASR) included in the EEGLAB package.⁶⁷ The ASR algorithm calculates the standard deviation of a “clean” portion of the signals in the principal component analysis (PCA) subspace, and reconstructs artifacts with standard deviations as κ times higher than in the clean portion. Here, a value of κ = 15 was chosen to remove remnants of eye movement and muscle artifacts. According to,⁶⁸ κ values between 10-20 are recommended to reduce artifacts and at the same time preserve the content of EEG signals. As a final step, independent component analysis (ICA) was performed on the data and suspicious components (eye, muscle, electrode popping) were removed before projecting back the signals. A graphical representation of the pre-processing steps is presented in Figure 2, as well as feature extraction and subsequent signals analysis.

Figure 2. Signal processing methodology flowchart divided into four main steps: (1) electroencephalographic (EEG) data acquisition, (2) pre-processing and denoising, (3) brain-to-brain feature extraction, and (4) statistical analysis.

Each step is described in detail in the Methods section.

Brain-to-brain feature extraction

The improvisational nature of the performance allowed for the examination of the musical communication between the musicians as the piece progressed. At times, they built from the theme established, returned to the main theme, or proposed new ideas. With the annotations from three independent raters, we clustered sections of the performance where the participants performed synchronously or not as synchronized performance (SP), and desynchronized performance (DP). SP included moments of in-time synchronized execution, as well as improvisation and interactions under the same underlying pulse; while DP reflected moments where musicians traded material, though not aligned to the same underlying pulse, and with no coordination (as well as deviation from the current theme). A temporal (across-time) analysis was performed to observe neural synchronization in those moments of SP and DP; and these times were evaluated across three participant conditions: passive-passive, when no musicians in a dyad were performing; passive-active, when one musician in a dyad was performing; or active-active, when the two musicians in a dyad were performing. For the sake of the ecological validity approach, rather than manipulating experimental conditions (e.g. rest vs improvisation), we observed brain synchrony across SP and DP, and the participant conditions posed by the exquisite corpse approach.

The quantitative measurement of brain-to-brain communication was achieved by calculating the bispectrum between musician dyads of EEG data obtained during improvised musical performance at different stages of interactive performance. As referred to in previous works, higher bispectrum magnitudes at given pairs of frequencies reflect non-random interactions, phase coupling,¹⁹ and non-linear multi-frequency interactions,²⁶ which have been observed as traces of inter-brain synchrony during teamwork interactions.^19,24

The denoised EEG signals were used to estimate the bispectrum between all possible channel combinations, for all participant pairs, trials and segments. Bispectrum was estimated across the EEG recordings using four-second windows with 75% (one-second) overlap. The bispectrum at each time window was estimated using Equation 1:

Bispectrum was estimated at 60² EEG channel combinations between pairs of participants for all segments, and trials. A bispectral representation of a segment was obtained averaging all four-second windows in each segment, for each frequency bin (1-50 Hz). Bispectral representations were normalized to the bispectral representations of the same channel combinations during pre-trial EO task using Equation 2. Pre-trial EO was treated as rest condition, where participants did not communicate with each other.

Bispectral values for five frequency bands were obtained as the average of the normalized bispectral representation in the following frequency ranges: delta (1-4 Hz), theta (4-7 Hz), alpha (8-12 Hz), beta (13-29 Hz) and gamma (30-50 Hz).

A temporal bispectrum series was also estimated, using sliding overlapping windows of four-seconds (75%). At each window, the temporal bispectrum values were obtained as the average normalized bispectral representation (Equation 2) at each frequency band, thus obtaining a temporal representation of the EEG signals’ synchronization between musicians. These temporal bispectrum values were estimated for all windows using the channel combinations which were found to be significant in the implemented statistical analysis. The analysis is described in detail in the statistical analysis subsection.

Statistical analysis

Right-tailed Wilcoxon signed rank tests were used to evaluate statistically significant differences between the average bispectrum at different frequency bands for all channel combinations. Average bispectral representations during rest and specific segments were compared.

This procedure ensures the discovery of only those channel combinations with significantly higher bispectrum at a specific segment and for a given frequency band as compared to rest. At each Segment, 60² tests were performed (p < 0.05, corrected for multiple comparisons via Bonferroni correction). Statistical tests were performed for all trials (3), segments (3), participant pairs (3) and frequency bands (5), for a total of 486, 000 tests. A different amount of samples was used for each frequency band, due to bandwidth difference; 16 for delta and theta, 20 for alpha, 68 for beta and 82 for gamma.

Through this procedure, the identified channel combinations were used as representative traces of brain-to-brain communication during musical improvisation. To further explore the behaviour of such traces, temporal and spatial analyses were implemented.

Temporal analysis

The temporal analysis of bispectrum was implemented in representative bispectrum traces to observe its dynamics under different conditions during the performance. The bispectrum traces used in this analysis were those of the most significant channel combination (in the gamma band as described in the Results section) at the third segment of each trial, for each participant pair.

The bispectrum analysis was divided into two groups of events of naturally occurring experimental conditions: SP and DP, as labelled by the annotators in the audience. For each event, a two-minutes representative bispectrum trace in a time period (-60 to 60 s) was obtained per dyad. For each specific event, the time of the annotation was considered as the 0 s mark. To observe relative differences between SP and DP, the bispectrum traces were baseline corrected at each event for both groups. To obtain baseline corrected traces, average bispectrum in the (-60 to 0 s) period was obtained and substracted from each two-minute bispectrum trace.

Baseline corrected bispectrum traces were obtained for all events, trials and participant pairs, and were grouped and compared between the two groups. Wilcoxon signed rank tests were used to find statistically significant differences (p < 0.05) at every (-60 to 60 s) time point, between SP and DP at each performance condition. This analysis was implemented independently for events in the passive-passive, passive-active and active-active performance.

Spatial analysis

Spatial analysis was implemented to identify regions of interest (ROIs) involved in musical performance. The selected ROIs group spatially close electrodes in 13 regions: anterior frontal (AF), left fronto-central (LFC), midline fronto-central (MFC), right fronto-central (RFC), left centro-parietal (LCP), midline centro-parietal (MCP), right centro-parietal (RCP), left parieto-occipital (LPO), middle parieto-occipital (MPO), right parieto-occipital (RPO), left temporal (LT), right temporal (RT) and occipital (O).⁷⁰ Figure 7 shows the location for the 13 ROIs within the scalp map. The significant channel combinations identified through the statistical analysis at every segment and trial were grouped for the different performance conditions: passive-passive, passive-active and active-active.

Statistical analysis

The procedure of the statistical tests presented in the Statistical analysis subsection was implemented for all Segments, Trials, frequency bands and participant pairs. No significant channel combinations were found for the delta and theta band for any segment. Most statistically significant channel combinations were found for the beta and gamma bands, and a few in the alpha band.

Figure 4(a) shows the total significant channel combinations (between all dyads) when different musicians were performing together, and Figure 4(b) shows a topographical representation of the statistical analyses.

In Figure 4(a), the dyads P₁₂ and P₂₃ show consistently more significant inter-brain synchronized channels that for the P₁₃ dyad. The three dyads showed few synchronized channels when musician P₃ performed alone, and when P₁ performed together with P₃.

The most significant channel combinations (visualizing the top 5 channel pairs) for the alpha, beta and gamma bands are shown for each participant pair. Bar graphs show the amount of significant channel combinations at each frequency band, and dyad. At each specific segment, passive and active musicians are shown as white and gray heads, respectively. Topographical representations are presented for all trials and segments, therefore the first row of Figure 4 corresponds to the data shown in Figure 3.

In Trial 1 (top row of Figure 4(b)) and Segment 1, P₃ starts performing, and only a few significant channel combinations were found for dyad P₁₂ in the gamma band. In Segment 2, P₁ joins and more channel combinations are shown in the beta and gamma bands for dyad P₁₂ and P₁₃, who are performing. In Segment 3, when all musicians are performing, an increase in the amount of significant channel combinations is observed for all participant pairs. In this last Segment, a few channel combinations were observed in the alpha band.

In Trial 2 (middle row of Figure 4(b)), in Segment 1, P₁ starts performing. A few channel combinations were found to be significant for all participants in the beta and gamma bands. In Segment 2, P₂ joins and an increase in the amount of significant channel combinations is observed for dyads P₁₂ and P₂₃). In the Segment 3, P₃ joins and a decrease in the amount of significant channel combinations is observed across all participants).

In Trial 3 (bottom row of Figure 4(b)), P₂ starts performing, and significant channel combinations for P₁₂ and P₂₃ are observed for beta and gamma, and only one for P₁₃. In Segment 2, P₃ joins and a similar connection pattern is observed between participants at Segment 1. In Segment 3, P₁ joins and a considerable increase in significant channel combinations is observed for both P₁₂ and P₂₃, while those for P₂₃ remain similar.

Some general patterns were observed through this analysis. It was observed that the amount of significant channel combinations increased as more musicians joined the performance, which can be observed in Figure 4(a). Dyad P₁₃ showed less amount of significant channel combinations throughout the experiment, at different segments and trials 4 (a). Also, in all segments, the amount of significant channel combinations was higher for the gamma band than for beta or alpha bands. Finally, the most common interconnected regions across segments and trials are those involving the temporal, occipital and parietal regions.

Temporal analysis

Figure 5 shows the normalized bispectrum trace for the 15 minutes of Trial 1, using the significant channel combinations described in the temporal analysis subsection. The vertical dashed lines mark the division between segments, and bars are used to visualize the moments during the performance when the experts identified either an SP or DP event. The volume of the recorded audio file from the performance is shown below the bispectrum traces. The individual events shown in Figure 5 are described in Table 1. The corresponding Figures and Tables for Trials 2 and 3 are presented in the Extended data, in Figures S1 and S2, and Tables S1 and S2, respectively. Representative SP and DP events from Trials 1-3 are presented in Videoclips S1-S3 in the Extended data.⁹⁸

Figure 5. Bispectrum temporal dynamics at the most significant channel combination per participant pair in the gamma band, and normalized volume intensity (unitless) of the audio recorded during the performance of Trial 1.

Vertical dashed lines represent the times when a new musician joined the performance. Vertical bars represent time of synchronized performance (SP) or desynchronized performance (DP) events, as labelled by experts. A representation of the selected channels for each dyad is shown in the bottom right corner. The annotations from the events are shown in Table 1.

Figure 6(a) and (b) show the average bispectrum change across participant pairs for the passive-active and active-active conditions, respectively, for both SP and DP. The 0 seconds vertical dotted line in Figure 6 indicates the start of the event: either SP or DP. The amount of averaged traces for each condition were, 24 and 31 for SP; and 14 and 26 for DP; for the conditions passive-active and active-active, respectively. Figures c) and d) show every individual trace analyzed in a) and b), respectively.

Figure 6. Average bispectrum (BS) change (%) across all segments and participant pairs at their most significant channel combination in gamma band, for both synchronized performance (SP) and desynchronized performance (DP) events.

BS change (%) was obtained applying baseline correction on each trace by using the 60s previous to each event. Average BS changes (%), and histograms (distribution of all traces) are shown for passive-active (a) and active-active (b) conditions. The amount of averaged traces at a) and b), respectively, are: 24 and 31 during SP (n_SP); 14 and 26 during DP (n_DP). Individual BS (%) changes for all n_SP and n_DP events are presented in c) and d) for the passive-active and active-active conditions, respectively.

No statistical significance was observed between SP and DP in the passive-active condition. Bispectrum change was significantly higher during SP than DP in the active-active condition, slightly before the onset of annotated events (− 3 s), as well as 40 s after the onset.

Spatial analysis

A topographical visualization of the most significant scalp ROIs for participant pair conditions (summarizing the findings of Figure 4), is shown in Figure 7. Visualization maps were plotted to represent the degree of synchronization between and within the evaluated ROIs for all conditions (passive-passive, passive-active and active-active), dyads (3) and trials (3). Figure 7 show this representation in the gamma and beta bands. It can be observed that the most synchronized ROIs are in the active-active condition, while less are observed in the Passive-Active condition and the lesser during passive-passive condition.

Figure 7. Topographical representations of between-participants inter-synchronization of 13 regions of interest (ROIs) (anterior frontal, left fronto-central, midline fronto-central, right fronto-central, left centro-parietal, midline centro-parietal, right centro-parietal, left parieto-occipital, middle parieto-occipital, right parieto-occipital, left temporal, right temporal and occipital) across all dyads (3) and trials (3), in passive-passive (left), passive-active (middle) and active-active (right) conditions, in the gamma (top) and beta (bottom) bands.

Shading represents the degree of inter-synchronization within the same (circles) and different (lines) ROIs.

Summary of main findings

Brain synchrony (as assessed via the bispectrum) increases when more musicians are performing. This is most notorious in the higher frequency bands (beta and gamma).

The number of inter-connected channels and ROIs increase when there are more musicians performing. This happened mainly across temporal, occipital and parietal regions, related to multi-sensory processing.

Changes in brain synchrony (bispectrum) allowed to identify different performing states across musicians during the improvisation: Bispectrum was higher during Synchronized Performance and lower during Desynchronized Performance.

Underlying data

OSF: MOBILE EEG RECORDINGS OF MUSICAL (JAZZ) IMPROVISATION.

https://doi.org/10.17605/OSF.IO/YUEQK⁹⁸

This project contains the following underlying data:

Extended data

OSF: MOBILE EEG RECORDINGS OF MUSICAL (JAZZ) IMPROVISATION.

https://doi.org/10.17605/OSF.IO/YUEQK⁹⁸

This project contains the following extended data:

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).