INTERVIEW / AustinTheWise: Metaverse might have a big impact on streaming deals

Audio plays a crucial role in immersing users in the metaverse experience, and with the increasing popularity of VR and AR, audio engineering in the metaverse has become a sought after skill. From a background in traditional studio engineering, our interviewee dove into the VR world and quickly recognized the opportunity for traditional media professionals to enter the VR and AR space. With a deep understanding of 3D spatial audio within a virtual space, Austin Egge, also known as Austin The Wise, brings immersion to users through sound in ways that can’t be achieved with stereo or surround sound.

Can you tell us about your background and how you became an audio engineer for the metaverse?
My background is in traditional studio engineering, mixing and mastering music was my main focus for more than a decade before diving into VR and exploring the many different metaverses in December of 2021. After trying out multiple sandbox style metaverses, including VRChat, AltspaceVR, and Horizon Worlds, I quickly recognized the opportunity for traditional media professionals to enter the VR and AR space. That includes videographers, photographers, graphic designers, writers, voice actors, or audio engineers to name a few professions.

Bringing my audio engineering skills into VR took a few adjustments and additions. Specifically, engineering audio in VR and AR requires an understanding of 3D spatial audio within a virtual space as it relates to and differs from stereo audio or surround sound within a real physical space. After tons of trial and error I am happy to say I am now well versed at bringing immersion to users through 3D spatial audio in ways that simply can’t be done with stereo or surround sound, that has been my favorite part of this journey thus far.

How does audio play a role in immersing users in the metaverse experience?
Audio is often overlooked, but in many ways it’s the most important part of immersion when it comes to VR or AR experiences. Ensuring that the user or player convincingly hears the sounds within a VR or AR experience emanating from their virtual (in the case of VR) or physical (in the case of AR) counterparts is crucial to immersing the user or player in the experience. As studies have shown, with increased immersion or sense of presence in VR, there is a greater chance that participants will behave in VR in a similar manner to their behavior in everyday reality. Similar studies have shown that the more immersive a VR environment is, the higher the chances are that the VR environment wins over physical reality in determining user responses.

Can you discuss any unique challenges you face when working with audio in the metaverse compared to traditional audio engineering?
Going back to handling audio in a 3D space versus handling it in a stereo or surround environment, there are many quirks and small challenges that you must overcome to get the right feel and immersion with your audio and sound in 3D space. Knowing about the different ways in which 3D spatial audio can project itself within a 3D space is crucial in understanding how to amplify and mix each sound within the 3D space. Mixing BGM music to be universally heard by everyone in the 3D space at the same loudness or having audio emanating from an object within the 3D space where amplification is determined by the users position in relation to the object require two very different approaches to mixing in 3D space.

Can you talk about any particularly interesting or innovative projects you’ve worked on in the metaverse?
I would have to say the most innovative and immersive projects I’ve had the pleasure of working on have come in the form of studio hang out spaces and PvP games. These projects give me an opportunity as both an audio engineer and sound designer to paint a picture with audio using the entire 3D space as my canvas. Engineering in stereo or surround sound offers a very limited canvas compared to 3D spatial. Instead of panning to just 2 channels in stereo or up to 7 in surround, you are mixing, panning, and adjusting the audio within a 3D space that can range from just a few to several hundred “virtual” meters resulting in nearly unlimited degrees of separation for the listener.

Can you tell us about any upcoming developments or advancements in audio technology for the metaverse that you are currently working on or are excited about? 
I’m excited about VR apps like TribeXR and PatchWorlds and where those are headed with each subsequent update, as I feel they showcase best what is possible with music and audio production/engineering in VR and AR. I feel that Apple adding support for 3D spatial audio with Dolby Atmos for Apple Music was a nod to the growing popularity of spatial audio, and that move gives us a glimpse into the potential future of 3D spatial audio. In the near future I think that 3D spatial audio will be more of a standard and less of a premium feature in headphones, as I believe they will have the same gyroscopes and accelerometers built into them that you find inside VR headsets that enable accurate playback of 3D spatial audio within a 3D space.

How do you think the widespread adoption of metaverse technology will affect our understanding and definition of privacy and personal identity?
I don’t think the existence of any specific metaverse will change our definition of privacy, however when it comes to personal identity I think avatar technology, specifically Metas Avatars SDK, gives people the opportunity to express themselves more freely while in a virtual social environment than they would in a physical social environment. I’ve witnessed first hand self admitted introverts acting and socializing in VR in ways they wouldn’t even begin to consider in real life and how happy they are that they can do so, which highlights the therapeutic effects that positive social interaction has on the brain and how such interactions have a net positive effect on one’s mental health.

How do you see the metaverse impacting the music industry?
The biggest impact will be in the way the big record labels will have to structure and restructure their streaming deals with metaverse platforms that offer streaming or playback of copyrighted music within user generated VR worlds and environments, also how and if that user generated content can be monetized while hosting such copyrighted content. Many platforms offer simple YouTube or Spotify integration, eliminating the need to renegotiate streaming deals that are currently in place, but the platforms that allow VR developers and builders to upload and playback copyrighted audio within user generated 3D environments will need to position themselves to not break any copyright laws as they pertain to music.

How do you see audio technology evolving in the metaverse in the next 5 years?
I think that over the next 5 years we will see technology evolve to better accommodate handling audio within 3D space as opposed to stereo or surround sound. I think most of this evolution will happen on the headset side opposed to the computer production side. In order to accurately engineer audio within a 3D space, ideally you use the hardware and software provided with a VR headset instead of relying on expensive plugins that only “emulate” 3D spaces in a digital audio workstation, and since many VR and AR applications treat audio differently there is no one size fits all approach to 3D spatial audio.

If you could be trapped in any virtual reality game or metaverse, which one would you choose and why?
I would have to say Meta Horizon Worlds. I’ve gotten quite used to the locomotion in that metaverse so I don’t think I’d get too nauseated being trapped in there for an extended period of time