Microphones and audio capture

Audio is a critical element in immersive media; selecting suitable microphones and recorders is an important part of the production process.

Audio is an important part of the storytelling process in immersive media. It drives viewer attention, communicates emotional arcs, and is a large part of what creates convincing immersion into virtual environments. While we can only look in one direction at a time, we can perceive sound in every direction.

Well-crafted immersive experiences include spatial audio that simulates how we hear the world around us. Unlike stereo audio source, which is based on left and right channels and is “headlocked” (the audio is locked to the headset orientation), spatial audio is “worldlocked” and is anchored within the virtual environment. When watching an immersive video with spatial audio in VR, you may hear a character talking over your left shoulder. When you turn to face that character, the character’s voice will be coming from where the character is standing. Spatial audio is also used to create immersion, and may also be leveraged for audio cues that help viewers to understand when and where they need to look, for narrative purposes.

Immersive audio capture can be done using specialized spatial audio microphones, but traditional mono mics that are clean and free of extraneous noise can also be used, with audio tracks ‘spatialized’ to Ambisonics (a popular full-sphere surround sound format) in post production by an audio mixer. Spatial “room tone” for environments can be captured using spatial audio microphones, and spoken audio can be recorded using lavalier mics or other methods, and then mixed into the experience.

In traditional film production, a shotgun microphone and boom pole are commonly used for audio recording. This method is not commonly used in 360 video production as the boom operator, boom pole, microphone, and all of their shadows will be captured (requiring complicated removal in post production).

Specialized spatial-audio mics can be used to capture ambient sound in Ambisonics. In immersive productions, these are usually mounted on or very near the camera and are useful for capturing environmental sounds from the point of view of the camera (and VR viewer). Lavalier microphones are often clipped to the talent and used to capture dialog. These record isolated mono sounds with minimal background noise, which is easy to spatialize during audio mixing. These, along with other audio elements like foley, are spatialized and mixed to create a feeling of natural presence in a space.

An important point is to capture sounds with as little background noise as possible. Some conditions can make this challenging, for example, filming a scene outside with traffic or aircraft noise. A sound engineer with spatial audio knowledge is an important part of immersive video productions.

Spatial audio microphones and recorders record the audio in all directions. They are good choices when a faithful reproduction of the sound in a space is required (e.g., recording an orchestra in a hall). They are not often not suitable for dialog as environmental sounds can overwhelm spoken dialog.

The Zoom H3-VR is a useful and popular Ambisonic mic. It is affordable and easy to use, and includes a built-in recorder that can capture audio files which are ready to be dropped straight into a video editor. The Zoom H3-VR can record straight to 1st-order AmbiX (a sub-standard in Ambisonics), which is accepted by most immersive video players and platforms.

Sennheiser’s Ambeo VR mic is another popular Ambisonic mic. The Ambeo VR needs to be paired with an external recorder that can record four channels ganged together in equal gain (usually in the form of an explicit Ambisonics mode in the recorder). Multi-channel recorders by Zoom and Sound Devices are commonly used to record Ambisonics alongside other audio channels, and can easily be configured to simultaneously record both.

Lavalier mics are often used to capture character dialog. These can be wired to a compact recorder hidden on the talent or connected to a small wireless transmitter for recording audio on a separate recorder. A lav mic needs to be positioned close to the subject’s mouth, and care needs to be used to avoid rustling sounds from clothing. See Lavalier microphone tips for full details and advice.

Although shotgun mics can be used in immersive video production, they need to be used carefully to ensure that mics are not visible in captured footage. For 360 work shotgun mics can be impractical, but they can be used in 180 productions.

Shotgun mics are also useful for on-location ‘wild sound’ capture independent of video capture. These sounds can then be spatialized and mixed during post production. Examples of sounds that are appropriate for capturing using a shotgun mic include footsteps, efforts (breathing, grunts and so on), and specific environmental sounds that would be hard to recreate with sound library clips.

Many VR cameras do not have genlock or timecode solutions for syncing video and audio in post production. If the camera supports internal audio recording of any kind, consider enabling this as a scratch track to simplify syncing in post production, or use an external timecode audio injector such as the Tentacle Sync E. It is also a good idea to slate each shot with a clapper board or hand clap that can be seen by one of the camera lenses and heard by all the microphones.