Listen to music with the sense of touch

Summary: An innovative new prototype allows people with hearing loss to ‘listen’ to music through the sense of touch.

Source: University of Málaga

People with hearing loss will be able to listen to music through the sense of touch thanks to a pioneering prototype designed by researchers from the Department of Electronics of the University of Malaga, members of the R&D group “Electronics for Instrumentation and Systems” .

It consists of an audio-tactile algorithm which, using “tactile illusions”, transforms monophonic music into tangible stimuli based on vibration. “It’s like ‘hacking’ the nervous system to receive a different response to the real stimulus being sent,” they say.

“What we want to achieve in the long term is that people who cannot hear can ‘listen’ to music”, assures the researcher of the doctoral program in mechatronic engineering Paul Remache, lead author of this article, who emphasizes the power of music to influence mood, as well as its possibilities as a therapy for mental disorders and the treatment of pain.

The result would be a portable terminal that could be taken to a concert, since this prototype, according to the researchers, will be easily transferable to technological devices such as mobile phones.

Map music

The algorithm developed by this young researcher, together with UMA professors Andrés Trujillo and Fernando Vidal, is able to convert characteristics and musical structures extracted from MIDI files –Musical instrument digital interface– at “vibrotactile stimuli.

“It’s something akin to musical mapping,” says Remache, who adds that it’s possible because these types of files can not only be played and generate sound, but also provide “symbolic representations.”

Vibration control

Current models do not guarantee the correspondence between the emotional response to music and its vibrotactile version. With this in mind, these UMA engineers propose an arrangement of “tactile illusions” to enhance and expand the spectrum of musical characteristics, adding dynamics to vibration in the form of movement, change in direction and location. .

Early experiments have shown that the “tactile illusions” perceived by the vibrations elicit in people who do not hear a different – ​​mostly positive – emotional response from that of the original music. Credit: University of Malaga

“This is a difficult process because the perceptible frequency range of the skin is lower than that of the auditory system, which can lead to the loss of certain musical characteristics,” they explain.

Different emotional response

The first experiments carried out, in which more than fifty volunteers took part, suggest that the layout of the “tactile illusions” evokes more positive emotions than negative ones. They are also perceived to be more enjoyable and stimulating than audio, eliciting a different emotional response than the original music.

Smart instrumentation and application in healthcare

This first prototype was presented at the 11th International Workshop on Haptic & Audio Interaction Design (UK) – the largest international event specializing in these fields of study – after being published in the scientific journal LNCS. Currently, UMA researchers are working on a second model and are continuing experiments.

The research stems from Paul Remache’s doctoral thesis and is part of the National Plan project “Intelligent instrumentation and application in health”.

About this neurotechnology research news

Author: Maria Guerrero
Source: University of Málaga
Contact: María Guerrero – University of Malaga
Image: The image is credited to the University of Malaga

See also

This shows the dopaminergic neurons

Original research: Access closed.
“Mapping Monophonic MIDI Tracks to Vibrotactile Stimuli Using Tactile Illusions” by Paul Remache et al. LNCS


Summary

Mapping monophonic MIDI tracks to vibrotactile stimuli using tactile illusions

In this project, we propose an algorithm to convert musical characteristics and structures extracted from monophonic MIDI files into tactile illusions.

Mapping music to vibrotactile stimuli is a difficult process because the perceptible frequency range of the skin is lower than that of the auditory system, which can lead to the loss of certain musical characteristics. Moreover, the models currently proposed do not guarantee the correspondence between the emotional response to music and its vibrotactile version.

We propose to use tactile illusions as an additional resource to convey more meaningful vibrotactile stimuli. Tactile illusions allow us to add dynamics to vibrotactile stimuli in the form of movement, change of direction and location.

The suggested algorithm converts monophonic MIDI files into arrangements of two tactile illusions: “ghost motion” and “funnel”. The validation of the rendered material consisted of presenting the participants with the audio rendered from the MIDI files and then adding the vibrotactile component to it. The layout of the tactile illusions was also evaluated on its own.

The results suggest that the arrangement of tactile illusions evokes more positive than negative emotions. This arrangement was also perceived as more enjoyable and stimulating than the original audio.

Although musical characteristics such as rhythm, tempo, and melody were primarily recognized in the tactile illusion arrangement, it elicited a different emotional response than the original audio.

Comments are closed.