Freelance consultant for digital heritage

Immersive 3D sound as a tool for curators

Are you hearing my voice in your left ear? Then let’s get started.

Visitors to Versailles exhibition binaural audio experience, The Met, New York City, 2018.

Sound is a profoundly powerful sense. It can trigger memories, create experiences, and take people to new – and old – places. The philosophy of the Curatorial Research Centre is that 50% of a curators role is to communicate their knowledge. Sound can provide a rewarding tool to help you do that. Think of a film sound track – it is often more than just music, but carefully designed sound effects that enhance the mood and storyline; interpretation that doesn’t rely upon the written word.

Many of us will have visited museum or art gallery exhibitions that use sound effectively. This is often through background music, soundscapes, narration and oral history recordings. I enjoy listening to the sounds on offer, especially when they reinforce a story, or help to interpret an object or concept.

Echoes

On our recent visit to Estonia, and the Estonian National Museum, we visited their permanent exhibition, Echo of the Urals. It uses large visuals and dioramas reinforced with dramatic sound and light and is a deeply immersive and special experience, without being intrusive. Approaching the exhibition the sound of birdsong puts the visitor in a different place and a different time, one far away from today. Moving through the fishing gallery large graphics above the cases depict lakes and water, reinforced by the sound of insects and gentle waves lapping in the background. The soundtrack of Echo of the Urals, we were told, is the most extensive soundtrack used in a permanent exhibition. The sound, which includes the variety of languages and dialects of people, works together with complex gallery lighting that mimics the change of light in the days and seasons–both provide a sensation of time and environment for the artefacts and exhibits on display. Simple but incredibly thought-provoking and authentic use of in-gallery audio.

But you can go a step further.

In 2018, The Met in New York produced an exhibition called Visitors to Versailles 1682 – 1789:

Bringing together works from The Met, the Château de Versailles, and over fifty lenders, this exhibition highlights the experiences of travelers from 1682, when Louis XIV moved his court to Versailles, to 1789, when the royal family was forced to leave the palace and return to Paris. Through paintings, portraits, furniture, tapestries, carpets, costumes, porcelain, sculpture, arms and armor, and guidebooks, the exhibition illustrates what visitors encountered at court, what kind of welcome and access to the palace they received, and, most importantly, what impressions, gifts, and souvenirs they took home with them.

https://www.metmuseum.org/exhibitions/listings/2018/visitors-to-versailles

An immersive audio experience was created as a companion to the exhibition:

This immersive audio experience brings to life the impressions of those who visited the palace and court in the seventeenth and eighteenth centuries. It presents adaptations of their written accounts dramatized within atmospheric 3-D soundscapes.

https://www.metmuseum.org/exhibitions/listings/2018/visitors-to-versailles

Binaural or 3D audio

The ‘experience’ uses binaural audio to mimic the way we hear the world around us naturally. We have two ears, and yet can tell when a sound is in front of us or behind. Some call this “immersive 3D audio” because the direction of sound sources can be heard. No expensive and cumbersome equipment is needed to listen, such as multi-speaker arrays backed up by a cupboard full of amplifiers. Just a smartphone and a pair of headphones.

Produced with binaural audio recording methods, including sound effects and music, the project features theatrical performances by a dozen noted international actors. The resulting soundscapes mimic the 3-D experience of how we hear sound with our own ears, and sometimes give the uncanny sense that the listener shares physical space with those speaking. To provide authentic acoustics, production took place in historic locations, including Oldway Mansion in Devon, England, an estate with rooms modeled after those in the Palace of Versailles.

https://www.metmuseum.org/exhibitions/listings/2018/visitors-to-versailles

To get a sense of this, pop on some headphones and have a listen to the playlist below, beginning with the introduction.

Whilst this production used ‘sound-alike’ locations to record the audio, using a microphone that uses slightly unsettling silicone ‘ears’, it is possible to record the atmosphere of a real location and simulate it in a studio environment. Think of the effect of clapping once loudly in a big room. You will hear the sound of your clap die away – this is the reverb, or reverberation, the sound changing and falling off due to the characteristics of the room or space. This is called the ‘impulse response’ of a space.

The 3Dio Free Space binaural microphone – https://3diosound.com/products/free-space-binaural-microphone

It is not always possible or even practical to bring microphones and voice actors into a real space, and so recreating the sound of a space in a studio could have some great advantages in building immersive 3D audio experiences such as Visitors to Versailles.

The Met’s exhibition aimed to give visitors a sense of what it was like to visit the palace of Versailles in the 17th and 18th centuries. As well as the objects from their own collections combined with loans from around the world, they provided an optional extra for visitors to the exhibition. When looking at the wonderful costumes, the visitor could listen to someone being dressed, hearing hair being snipped from behind them, conversation, people pacing about, all sounding very natural and believable. What did it sound like to be in a horse-drawn coach rumbling along a driveway in the 18th century? The Met used UK-based Aurelia Soundworks to make these very convincing recordings.

Not everyone wants that extra level of theatre, or needs a convincing soundscape, but for those of us who like that extra level, it’s a wonderful experience. The binaural 3D audio, delivered just through headphones, sounds so natural and provides a very effective way of providing that extra special touch available to curators responsible for interpretation. Perhaps more importantly than a ‘nice to have’, audio-based interpretation can be much more meaningful and accessible for people who struggle to or don’t find reading about art and history a rewarding experience. To be able to communicate stories, ideas and knowledge to the significant section of society who are blind or visually impaired in a way that doesn’t seek to replicate the written word, e.g. braille, makes 3D audio a very compelling alternative to traditional interpretation.