RESEARCH

Which tools are required to process audio in 3D?

Luckily, nowadays with the use of a single plugin per channel in a DAW session, it is possible to convert any sort of audio file to 3D. The choice among the tools to use is quite vast. During my research and investigation, I came across several ones:

• dearVR Pro by Dear Reality

• 360pan suite by Audio Ease

• Ambeo Orbit by Sennheiser. 

I have opted for dearVR Pro because it is part of a brand that is totally focused on 3D audio for immersive content (Dear Reality, 2020), which has developed dearVR Spatial Connect  (Dear Reality, 2020) (a platform that enables mixing using a VR Headset) and dear VR Unity  (Dear Reality, 2020) (a plugin to spatialize sounds inside game engine Unity), which I intend to acquire, study and use in future. Also, the plugin seemed fairly straightforward in terms of interface, features and general use.

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

How does the plugin and processing work?

To place a recording in tridimensional space, it is required to put the plugin in the track's inserts. It works both for mono and stereo tracks, however, it is advised to use mono sources to avoid phase issues and loss of quality (Lane, 2019). Studying the plugin, the ideal way to use stereo sources is to duplicate the track and set the plugin on both tracks with the input channel respectively on L and on R - using the dedicated control on the right master section. (Dear Reality, 2019) This allowed me to solve part of the main issues with stereo sources, although they were still not totally perfect, preferring mono sources for future similar productions.

​

Some tests provided by Dear Reality in regards of stereo and mono sources, comparing the outcomes of their spatialisation and how the split of stereo in two channels (assigning L & R in the input channel section of each dearVR Pro instance) or their sum in a single mono may differ. 

​

​

​

​

​

​

​

​

​

​

 

The sound can be visually dragged around a graph (that can switch from cartesian to polar), with the possibility to "position the object all around the listener - behind, in front of, to the right or to the left of and even below or above". (Dear Reality, 2019)

The content will be characterised by the virtual acoustics that the plugin applies, adding reverb to it. This can be tweaked, changed (choosing among a list of different ambiences and locations provided by the developers) or turned off. As the concept is to represent the sound around the listener, depending on its position, the perception may change, with the use of algorithms that change the frequency response and the features of the sound simulating how human ears would behave; this process allows to perceive the illusion of space as it would happen in the real world. The "Occlusion" control next to the graph enables cutting the high frequencies to create a custom presence of the sound. The main settings of the plugins are focused on shaping the space in which the sound is placed, as it is its acoustic character that will define the final result and perception at a given position. Size of the room, reflections, and dry/wet controls of the reverberations are the keys of this simulation.

Other than position and acoustic elements, there is a master section that allows you to:

• bypass

• boost the bass of the source 

• increase the performance of the process for a static object (this option is also ideal when using very large rooms, the longer the reverb the higher the performance and resources required by the process)

• use head tracking with dearVR Spatial Connect

• choose the final output depending on the desired format

• LFE

• master gain

​

 Several resources that helped me to understand how to use the plugin and complete a Binaural 3D Audio post-production.

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

​

Techniques

The main point of the mixing stage for Binaural 3D Audio floats around the fact that, compared to stereo, the point of view of the listener has changed. Hence it is important to consider where to place the sound elements, visualising them as objects in space. 

As Robinson suggests in Australian Film Television and Radio School (2018), less is more and trying to avoid layering sound to mask or cover elements, will result in a clearer and more enjoyable result. Also, it will help to keep the Binaural effect strong, as if the mix is too busy, the effect tends to get weaker. 

Distance and height may be great variables to create priorities among the sound sources, sending secondary elements further than main ones. Depending on the genre, decisions may change, shifting what to keep in the primary close area.

Balancing sources around the imaginery 360 sphere is the key to the 3D mix, although reflections, type of reverbs, dry/wet mixing will be required to obtain an acceptable result.

​

​

Conclusion

While working on the projects, I have understood that various music genres may respond differently to this format; it is all focused on the objective of the production and how the music is desired to be consumed. Indeed, the sense of "distance" of the sound around the listener creates an initial weird perception of it, especially in genres where the core elements perform perfectly in the "standard" way (Drum Beat in Hip Hop and Bass in Reggae for example). But it is a feeling that may change with more consumption; it is a new experience and the ear must get used to it before being able to completely appreciate it. 

Genres like Jazz may benefit from this, as space and sense of ambience is already a core feature. Ambient/Electronic music as well, where the pan-automation of synthesizers and patterns can create engaging and interesting journeys for the listeners. Still, it is important to highlight that it is not a substitute of standard post-production, at least in the refining (equalisation, dynamics) phase of each individual instrument; it is more of an additional step that may take the experience to a different route. (Sullivan, 2020)

From a technical point of view, live recorded music and events may benefit from this format, as it is then possible to recreate a sense of space and environment that is key of the experience of live concerts (ibid). The stage position of the artists on the X, the crowd on the Z and mixed elements on the Y would give a fully immersive perception, that is not possible to convey through the 2D of the stereo.