In the emerging Metaverse of Wondar Studios, interactivity and the potential of virtual reality are gaining ground in directing as well.
Having the ability to manage a real 3D film set, from virtual cameras to lights, from optics to framing, through virtual reality is the goal of WondAR Studios, an innovative Italian start-up in the field of motion capture animation, and Stefano Girardi, a director and artistic director of short films shot entirely in virtual production.
The project aims to implement a new approach to directing in 3D video productions, by inserting tools capable of making production management as immersive as possible.
This approach allows the director to manage everything directly within the 3D set and to change its appearance, for example by moving the elements present in the scene or visualizing the framing through a virtual video village. This anticipates what the direction within the Metaverse, the new virtual world being targeted by Meta, Microsoft, Roblox, Epic Games, Tencent, Alibaba, and ByteDance, could be.
The technology underlying direction in the Metaverse
All of this is made possible thanks to the combination of three of the most developed and experimental tools of recent years: Unreal Engine, Xsens, and Meta Quest. These three elements simultaneously provide everything this project needs to be developed: a 3D software with unique potential that provides realism to the elements, a motion capture animation system that allows real-time visualization of the actor's movements within the scene, testing their movements in the reference space, and finally a headset that allows entry into the set.
WondAR Studios and Stefano Girardi have started the project by first experimenting with the pipeline of the various elements and then with the way to combine them.
The creativity of the cinematographic set in the Metaverse
The creative aspect of this experimentation is in some ways one of the most interesting sides. Just think about being able to physically cross and experience the scene or environment in which it was set, teleporting to the virtual set thanks to the controller, getting up close and personal with what characterizes it and drawing inspiration from it; this would already be a significant part of the creative process and the construction of the final output idea.
Not only will it be possible to visualize the static elements, but all the animated components will also be visible: in this case, we refer to the motion capture animation technique that allows real-time visualization. The combination of these two parts, the environment and the moving subject, provides the direction with a complete picture and the closest possible to what will be the final result.
Metaverse, the potential of motion capture
One of the inspirations for this project and its potential was the new video clip by Salmo "Aldo Ritmo" produced by Reef Studios, entirely made in virtual production and motion capture. For this reason, WondAR's test features the avatar of the famous Italian singer, reconstructed with Character Creator and animated with the Xsens motion capture suit, inside Unreal Engine.
The intense work and the output of the elements of the Aldo Ritmo video clip have been a source of inspiration for the WondAR staff, which has implemented a new management and visualization methodology for this type of production.