If you were to lose your hearing today, you probably think you’d have zero chance of enjoying music ever again, you’d be wrong.

Research stemming out of the emoti-chair (a device which recently celebrated its tenth year anniversary) has not only made it possible for the hearing impaired to experience music, it’s also spawned new technologies that have the power to alter the way you feel–and has given birth to a whole new medium: vibrotactile composition.

When the emoti-chair was first introduced by Toronto researchers Frank Russo, Maria Karam and Deborah I. Fels, its full potential was unknown. The chair, which converts audio input into a variety of tactile sensations through 16 separate channels, made it possible for the hearing impaired to ‘hear’ intricate music through their skin–a significant upgrade from placing a hand on a speaker to hear the thump. Music is fed in, the chair spits this information out through vibrations, and the person sitting in the chair can feel everything from jazz and classical to R&B and punk.

But when Stephane Vera first sat down in the chair, he wondered: What would it be like to actually compose music for this thing? Initially, the electronic music producer considered the obvious: write songs and have the chair interpret them. “It was only when I had the idea of artificially deafening myself that things took a new path,” he says. “Don’t approach this as if this were for people with hearing disabilities. Really try to put yourself in the mindset of someone who is hearing impaired.”

Vera would artificially deafen himself using earbuds blasting white noise inside the heavy duty noise cancelling headphones that air traffic controllers use. “I had complete block of sound,” he explains. “I would hit the keys really loud… someone would scream STEPHANE! Like, nope, can’t hear anything. Perfect.” He would sit in one of the prototypes with a miniature keyboard on his lap, recording notes into a sequencer. “But in this instance,” he says, “I didn’t have any tempo going–because I wasn’t going to be hearing it. I just started composing for vibration, so I didn’t hear what I was doing. I could only feel it.”

Composing music for the deaf is a lot harder than it would appear. “Keep in mind,” says Vera, “for many, [vibrotactile composition] is a virgin experience (some even started to cry when they first felt this music)… so no pressure or anything! And two, the style of composition is so different. You’ve composed something, essentially, that does away with key. Because you’re composing for vibration, it may jump in a way that isn’t necessarily ‘musical’, or it won’t follow proper theory. You have to adapt your composition to the medium, as opposed to adapt the medium to the composition.”

When the research team led a workshop to investigate how the hearing impaired composed their own music using the chair, they got similar results. “Composers [with hearing] worry about things like consonance and tonality. They’ve tried to recreate those things with vibration. We’ve seen no attempts to do that with deaf composers,” says Russo. And the most successful ‘vibe tracks’ are those that do away with the rules: “It doesn’t necessarily have a tight metric framework,” says Russo. “There was always a pulse…  Doing things that span the full gamut of vibration–those are always very stimulating.”

Researchers mapped out the parts of the body most sensitive to certain frequencies. These areas are where the glabrous (or hairless) skin lies; fingertips, palms, feet, genitalia and lips.  “For the really high frequencies–that couldn’t necessarily be translated because there’s a limited spectrum of sensation–we would use air jets, on the face, typically,” says Vera. “For the low sensations; there’s obviously the low vibration–a ‘buttkicker’ motor would translate the lower frequencies into actual movement.”

What does it feel like? “A musical massage on steroids,” he says. “The most incredible thing is that the vibration — for people who are deaf or hard of hearing — is it’s exactly like hearing sound,” adds Maria Karam. “It’s a cross-modal thing — you can hear it, and feel it — when it goes through your body it goes to the same part of your brain. There’s such a link between your skin and your auditory cortex.”

Frank Russo is a cognitive neuroscientist who has researched vibration in music for years. “I think for hearing musicians, one of the challenges for composing for the chair is that it does make some sound. We’re used to thinking about music as an exclusively auditory phenomenon. And it can actually–I think it can be a hindrance to hear things. The vibrations are also very foreign in a way. Musical instruments create vibrations that we feel in some cases. We can feel the vibration with woodwinds for example. But it can be disorienting to have these vibrations appearing on different parts of the body where you’re not accustomed to them… some people are frazzled by this foreign experience, and some people welcome it.”

And just like a horror movie soundtrack can cue a fear response, vibrotactile composition has the same power to elicit an emotional reaction.“We do find that the biomarkers that are linked to autonomic arousal also seem to be moved by the same vibrotactile music,” he says. “So those biomarkers would be things like heart rate, and galvanic skin response; so your heart beats faster, and you sweat more when you’re aroused. So we have a piece of music that conveys let’s say joy, which is high in autonomic arousal–compared to calm. When we play that music as audio, we get higher levels of arousal. When we play that music as vibration–when we completely mask the audio, or when we play it for deaf listeners–we also get biomarkers that suggest to us increases in autonomic arousal.”

Further experiments concluded that in combining the auditory with the vibrotactile can heighten your emotional response to music; so that sad movie soundtrack–coupled with vibration–is going to trigger all the feels. “A lot of hearing impaired people can hear, its exceedingly rare to find someone who doesn’t hear anything,” says Russo. “Often they will go to movies, or concerts, or raves– and enjoy music the same way hearing people do, but maybe the fullness of the experience isn’t there. The synergistic effects of audio and vibe are probably even greater for people with hearing impairment.”

His team also unearthed some surprises about which aspects of music can be felt without sound. “The most obvious one is frequency, we can feel when things get higher or lower in frequency,” he says. “The other is loudness; you can feel that something is getting stronger in magnitude or weaker by feeling it or by using our ears. So the less obvious thing that we’ve discovered that can be conveyed thru vibration is the spectral balance.” Spectral balance is when you have two instruments that can play the exact same note at the same intensity, but one may sound bassier. For example, a bass and a violin play middle C. The bass has more low frequency energy than the violin  because of its acoustic properties. “We can feel that through vibration,” says Russo. “We are certain now that people can feel the difference in spectral balance through touch.This means a whole bunch of things. If you take two female speakers saying the exact same thing — ahhhh — at the same pitch, same duration. If we use our ears, we can tell, oh that’s Susan, that’s Rachel. Those are the spectral properties of their voices that allow us to do this. You can do the same thing through vibration. You can play Susan or Rachel’s ‘ahhhh’ through the chair, or with vibration on the skin, and people can tell the difference. The way we discovered this is by working with composers that were trying to do things to create new compositions.”

Since the emoti-chair, this technology is picking up steam in the commercial market. Haptic controllers are becoming commonplace in VR gaming. Subpac’s wearable technology enables you to ‘feel physical audio in music, gaming, VR and cinema’. Maria Karam’s Tactile Audio Displays Inc. (TADs) are currently adapting a custom TAD for the Jaguar Landrover–an entertainment and information system for the body. They’re also working on a motorcycle jacket for ARC motorcycles that allow riders to feel safety notifications on the road (helpful if a car is approaching from behind, for instance).

Karam even bought an old theatre, The Redwood, where you can see these systems for yourself (similar tech is available to try at Inter/Access). The Redwood contains a VR lab with a variety TAD systems that can be used by everyone that comes in. Her latest development is the somatocherry, which basically replicates what the emoti-chair does in a single node (as opposed to 16). “We’ve basically developed little components you can place that wherever you want — clothing, gloves, hats –and you have this potential for a full body experience.”

“I am particularly excited about the ability to transmit texture — and other kinds of physical feeling. The music and the sound is a great application, but what I’m very interested now with the somatocherry is you can basically translate texture over the wire, so we basically have a telephone for feeling. That was a serendipitous discovery of translating sound, that you could also translate texture.”

Karam, Russo, and Jean-Paul Boudreau (a developmental psychologist) even developed a ‘baby-vibe’, a mini-version of the emoti-chair. “Interestingly, children are incredibly sensitive to vibration… they have an innate ability to understand what the vibration is. We know that humans respond to, and need development in, their touch,” says Karam. “Besides physical touch from another person, having touch from a remote object like sound — might have some very interesting and potentially positive effects on the development of children,” she says.

“The way we look at it is, we would like this to become a ubiquitous system so we do not have to isolate deaf and hard of hearing people,” says Maria. “Our goal is to have everybody using it — to embed them in fixtures, in movie theatres, other theatres. We don’t want to isolate deaf people as subjects. We want it to be a physical device that just happens to benefit people who don’t have hearing. Then you don’t have to identify yourself as a person with a disability — you just can be a person using it and using your body.”

Want to try your hand at this technology? Register here for a Touch, Sound, Pulse: DIY Vibrotactile Systems – Workshop, where you’ll be taken through the process of building a simple vibrotactile system of your own. Saturday, June 22 | 1-4PM & Wednesday, June 26 | 7-9PM at Inter/Access (950 Dupont Street, Unit 1)

You can also meet Maria Karam and check out The Redwood (1300 Gerrard St E) later this month at Doors Open Toronto. See guide here.