Investigating human interaction: Study combines AI with simultaneous dual-brain neuroimaging for first time

Bringing research from the lab to the home, from a controlled environment to real life, can be a way to understand human interaction. As technology evolves, its potential grows, driving both scientific exploration and real-world applications. In this sense, the authors of a new study have taken a step forward to understanding what happens at the brain level when two people come into contact and interact with each other, such as during a conversation, when giving each other a gift, or in other situations of cooperation.

The methodology and results are described in a paper titled "Emotional content and semantic structure of dialogues are associated with Interpersonal Neural Synchrony in the Prefrontal Cortex," appearing in the journal NeuroImage.

The paper was authored by Alessandro Carollo and Gianluca Esposito (corresponding authors), Massimo Stella, and Andrea Bizzego of the University of Trento (Department of Psychology and Cognitive Science) as part of an international collaboration with Mengyu Lim of the Nanyang Technological University of Singapore.

Their work has shed new light on the association between the way in which people communicate, in terms of emotions and language, and their brain activity.

"For the first time, we have combined AI techniques with neuroimaging measurements obtained on two people at the same time. We worked in a laboratory setting, but we tried to create less controlled situations than usual, so that each participating couple was free to invent a dialogue as well as to imagine giving each other a gift and being surprised to receive it," says Carollo, first author of the study.

The research, which was conducted in the laboratories of the Department of Psychology and Cognitive Science of the University of Trento in Rovereto, involved 42 pairs of participants (84 individuals) between 18 and 35 years old.

"We combined artificial intelligence techniques with the most advanced brain imaging technology to study how emotions and the structure of language influence brain activity in interactions. This study reveals that when two people interact, their brain activity is synchronized, especially in the prefrontal cortex. Emotional content and the structure of language are connected to this neural synchrony," explains Esposito.

The dialogues were transcribed by hand, then artificial intelligence techniques were used to encode the transcriptions and obtain emotional and syntactic/semantic indexes of the conversations.

For neuroimaging measurements, functional near-infrared spectroscopy (fNIRS) was used.

This technique is similar to an electroencephalogram, but is less invasive than magnetic resonance imaging and other methods, and is capable of recording the dynamics of hemoglobin, the molecule that carries oxygen in the blood, in different brain areas. With a light source, which emits beams of photons, and a photodetector placed on a helmet, the amount of light absorbed by hemoglobin is measured and brain activity is thus evaluated.

Carollo explains, "It is an easy-to-carry and lightweight technique; it only takes a small box with a pair of caps and their cables. Then you plug it into a laptop computer and that is all you need to study human interactions. The goal is to bring research from the lab to the home, from the controlled environment to real life, where people are free to talk to each other and interact."

The contribution of the research team is promising.

Esposito states, "The best approach seems to be the transdisciplinary one, which integrates emotional content and semantic/syntactic information. The results obtained on neuronal synchronization have a number of interesting implications.

"The study shows that emotions and language structure influence our conversations and the neural processes that then guide how we interact with each other. This opens up new avenues for research into human interactions. We think of interactions between parent and child, between partners, friends, or simply two strangers who find themselves interacting by chance."