ORTF-3D is an ideal microphone setup for VR and 360 video, as it is capable of recording a spacious ambience. It can be rendered quite easily with the help of the suitable tools.
That's how we did it:
- We recorded 8 channels with the ORTF-3D microphone setup.
- The video was taken using the Samsung Gear 360 camera. The microphone stand was placed in the blind sport of the camera in order to make it invisible.
- In Unity (a game development platform), the video was rendered on a sphere around the point of view ("camera").
- No processing was supplied to the audio tracks except of some limitation to make it louder on silent headphones. The limiter was inserted on the 8-channel bus and not on the separate tracks to maintain stereophony.
- In Unity, the 8 wav-files were assigned to 8 static, diegetic virtual sources arranged in a symmetric cube around the listener.
- To binauralise the sound from the virtual sources, we used the Unity spatialiser plug-in "dearVR 3D audio reality engine" from Dear Reality. We chose it for its natural sound, but free alternatives are also available, for example the Oculus Spatializer.
For our app, the First Order Ambisonics recording was made as a comparison with a widely used FOA microphone. For this, an A-to-B-Format converter was used. The B-Format was then decoded to eight channels in the shape of a symmetric cube with the "WigWare" Plug-ins by Bruce Wiggins. Also "dearVR 3D audio reality engine" was used to binauralise the 8 channels from Ambisonics.
In the app, you will be able to watch and listen to the SCHOEPS Jazz band "Wrong and Strong", consisting of Hannes (product management), Hans (development) and Kacper (intern). The band is located in the ancient basement vault below our company building.
Using the app:
Using the app is easy: Plug your headphones into your phone, adjust the volume and open up the app. You will be asked whether you want to use the app with or without Cardboard. If you own a Cardboard compatible headset, choose this option and put the phone into it. Otherwise, just hold it in front of you horizontally like a so called "magic window". Then wait, listen and look around. The audio will be switched between the ORTF-3D setup and the First Order Ambisonics microphone every 12 seconds. The active microphone setup will be displayed by a flying text. If you want to change the setup manually, push the button on your Cardboard or tap the screen in the magic window mode.
For installation, just load the APK file on the phone, give rights and install.
Android phone (Android version 6.0 or higher), optional: Cardboard compatible VR headset.
On Samsung Galaxy S8 there might be a bug causing the app to crash. It is caused by the most recent version of Google VR Services. The fix is to go to the app manager, select Google VR Services and then select 'disable' or 'uninstall updates'. Any further information on this bug can be found in this forum.