Making audio accessible for XR events
I’ve worked on a lot of XR, VR, AR and 360 film projects. Very few of these are available via the Oculus Store, etc for at-home use. Instead, many of them are as part of a live event, whether utilising live actors, or taking place in some form of venue, whether for a specific event or for one of the film festivals like TriBeCa or Sundance. Often, budgets are limited, time is short and new software builds are being released rapidly. Often the priority is to get the basic functionality working. In these situations, accessibility is often compromised.
Whilst some people may be born Deaf, or with some degree of hearing loss; virtually every single person experiences age-related hearing loss as they grow older - your ear follicles start to die off from the age of 18. This can be exacerbated by illness, injury or exposure to loud noise. Whilst there are many proud members of the Deaf community, there are also people, particularly those with age-related hearing loss, who are ashamed or embarrassed about their hearing loss. Depending on the severity, their hearing loss may not have any significant impact on their day to day life except in very specific circumstances (like when they come to put on a VR headset). I remember that the first time I found out my Dad had started wearing a hearing aid, was when I tried to get him to listen to something on some on-the-ear headphones, and he couldn’t hear anything. Hearing loss is often an invisible disability, and as creators we are vastly underestimating the amount of people with hearing loss who are visiting or experiencing our VR and XR events. This article is a quick guide to how you can make your XR experience more accessible to people with hearing loss. And even better, all of what you need is relatively cheap and easy to setup.
It’s important to understand that there is no one-size-fits-all solution. Instead, we need a range of solutions to deal with a wide variety of hearing impairments and hearing aids. Some quick facts:
Hearing loss is often asymmetric, i.e. each ear may be affected differently. Unilateral hearing loss (UHL)) refers to loss in both ears (though it may be by different amounts). Hearing loss in one ear is referred to as single-sided deafness (SSD)
Hearing impairment can mean a loss of hearing, but it can also mean a sensitivity to loud noise, or certain frequencies.
People with a hearing impairment may have some form of hearing aid, but there is a wide variety of these. Hearing aids typically involve some form of microphone to pickup sound and relay to the wearer, some which are inside the ear, behind the ear, and some on the back of the head (common for cochlear implants). Older hearing aids may have a T setting which optionally allows them to pickup sound from an induction loop. More modern hearing aids have Bluetooth functionality, or the ability to plug an audio cable in directly.
Do I need extra equipment?
Some VR/AR headsets are already quite accessible for users with a hearing aid.
The Oculus Quest, HTC Vive Focus, Rift-S and Magic Leap 1 have built-in speakers in the headset. This means sound can often be picked up by the user’s hearing aid without any further effort. However, the volume on these devices does not go super loud, so if your soundtrack has a large dynamic range (a mix of loud and quiet sections) you may wish to consider the below equipment. From a sound design point-of-view, the built-in speakers on these devices aren’t great, so if you are building an audio-rich experience you don’t want to be using these built-in speakers anyway.
The Oculus Quest is often used with an additional set of clip-on on-ear headphones (like these). These improve the sound quality of the Quest considerably. However these are typically not compatible with most hearing aids.
The Valve Index headset has built-in speakers on the headset. These are better than those found on the Quest, etc, and you may find these work well with a user’s hearing aid.
The HTC Vive (original), Vive Cosmos use on-ear headphones. These are typically not compatible with most hearing aids (except in-ear hearing aids), as there’s no way for the sound coming out of the headphone to reach the microphone of the hearing aid, which is typically behind the ear, or on the back of the head.
Kit to have available:
The majority of VR headsets have a 3.5mm headphone socket on them, so it’s actually pretty easy to provide a small stock of equipment that visitors can use to get the best experience.
This allows the VR headset output to drive two different components, if the visitor has different requirements for each ear. Some VR headsets, like the Quest, have dual 3.5mm headphone outputs, one on each side of the headset. [Link]
This is a volume control that allows the left and right ear volumes to be independent controlled. It’s not an amplifier, it can’t make it louder, but it can reduce the sound in either or both ears for people with sensitive hearing. [Link]
This can be plugged in and the loop creates a small induction field, which can be picked up with a hearing aid with a T-setting. This is quite an old fashioned technology, and the sound quality isn’t very high. Increasingly hearing aids don’t feature it, but it’s good to have for backwards compatibility! [Link]
Many modern hearing aids allow for a Bluetooth connection. Whilst some VR headsets can also connect to Bluetooth headphones (and therefore a Bluetooth hearing aid), it is fiddly and time consuming to set-up, and requires someone who can configure the headset to pair with the hearing aid, and then to unpair it afterwards. This can be difficult to achieve whilst maintaining short onboarding times and fast turnarounds between visitors. Instead a separate Bluetooth transmitter can be used, which can be paired with a couple of button presses with the hearing aid, and can be done in the lobby area whilst the headsets are in use. These Bluetooth transmitters typically feature a small onboard battery. with a long battery life, that can be charged up with a USB cable, and have a 3.5mm plug that can be plugged into the headset.. If you choose an alternative model, look for models that utilise AptX low latency transmission and search the reviews for audio delays/latency.
These are standard headphones, but the cup that goes around your ear is quite large, which means it will encompass a behind-the-ear (BTE) hearing aid. Typically headphones have an inner pad size of 50mm or less which is not enough room to encase a heading aid behind the ear without muffling it. Beyer DT770 32ohm headphones are good for this.
Direct Input
Some hearing aids come with a special cable that can be plugged into the hearing aid and provides a 3.5mm plug on the end, so the owner can plug their hearing aid into an iPod, etc. These cables are bespoke to the model of hearing aid so visitors should be encouraged to bring their own if they have one.
Spatial sound
Another factor to bear in mind is that human beings use two ears (and often some head bobbing) in order to locate the source of spatialised sound. If you can only hear in one ear it’s much harder to work out the direction of a sound from the sound alone. A simple trick to this is to ensure that sounds, particularly voices don’t just appear in one ear only. This can be achieved by increasing the spread of a sound, or by using room acoustics so there is a reflected sound coming into both ears. Both these techniques make the sound more realistic and believable too, so double bonus!
Where possible it’s good to provide a mono version of the soundtrack to the visitor. Some headsets have an accessibility setting to allow this. Or you can create an option in your software build.
Obviously if you are using spatialised sound to draw the visual attention of the user, falling back to mono sound can be tricky, but captioning/subtitles can be useful here, for example “Dog barks, by the lamp-post”. There are also devices like this wearable haptic belt designed for gamers.
Captioning
Captioning (i.e. subtitles) are often important for people with hearing loss which cause difficulties in rendering speech intelligible, or for severe hearing impairment. For many deaf people, captioning is the preferred route, but this can be more time consuming to implement in software, so is best factored in early on in the development process.
Inclusion
As hearing loss is often an invisible disability, many deaf people feel self-conscious if they are given lots of different equipment that makes them stand out from non-disabled users.
The twitter thread below shows how some visitors reacted to being given prototype AR captioning glasses. It’s a useful reminder that the best intentions, the best plans on paper don’t serve our audience if they’re not implemented well.
Accessibility guidelines
There are a range of other ways you can make your XR experience more accessible, such as audio description. This requires some additional programming so needs to be factored in early on in development.
There are loads of suggestions for general accessibility (aimed at game design) here: http://gameaccessibilityguidelines.com
Providing the information
It’s important when creating accessible XR to provide information about these options to potential visitors. Some people are embarrassed by having hearing loss, and will avoid going to events if they feel they are unlikely to be catered for, or they’ll experience anxiety about booking something without knowing what will happen when they get there. Equally they may feel stigmatised if they feel they’re going to be treated differently because they have a hearing impairment. The more information you can provide in advance, on a website, in a brochure, and in the entrance/foyer/check-in/onboarding, the better.
Below is an excerpt of text we’ve included on booking websites so that people can plan their trip, know they’ll be catered too and to reduce any potential anxiety about attending. We also brief any onboarding staff about how to use the additional equipment, and how to discuss with visitors whether it is needed, and what all the options are.
Hearing aid wearers, particularly younger people, will often be quite tech-savvy about how they can plug it into devices, or how well they can use different types of headphones. Give them all the options, and let them decide what is the best solution for them.
Feel free to copy and adapt this for your own projects. if you need to vary it, please research how you use language to describe disability. Here’s a quick guide.
Hearing options for X
We use a VR headset with on-ear stereo headphones. All sounds are audible in both ears to a certain degree.
We have 3 options to help route sound from the VR headset to a hearing aid.
We have a set of slightly larger over-ear headphones, that can be placed on your head to provide sound to a behind-the-ear hearing aid. If you like, this can also be used with an independent left- right volume control, so that the volume on one ear can be turned down independently of the other.
We have personal induction loops, for hearing aids with a T setting.
We have a portable bluetooth transmitter, which can be paired to a hearing aid with bluetooth functionality. This takes 5-10 minutes to set up so come a bit early if you can.
If you have a hearing aid with a Direct Input cable, i.e. a cable with a 3.5mm plug on the end (as you might plug into a laptop or phone), you can bring that and plug it into the VR headset.
These are the headphones used on the VR headset: [link]
These are the over-sized headphones: [link]
This is the independent volume control: [link]
This is the personal induction loop: [link]
This is the bluetooth transmitter: [link]
If you have any comments or suggestions about this page, please do let me know by emailing me at garethfry@hotmail.com.