There’s a new Unity 3D-powered creation suite in the works utilizing Live2D performance capture via Google’s ARCore augmented reality toolkit. The software allows for real-time facial tracking via web camera or other capture device, with rendered output taking place as the performance or movements are being visualized.

Denchi Soft rolled out a near ten minute video to showcase how the software works and demonstrate what you can expect from kit when it eventually releases to the public at some point in the first quarter of 2020.

You can check out the video demonstration below.

So as the video outlines, the facial data is read and stored by ARCore and then fed into the Tensor Flow neural learning pipeline that’s hooked into Unity 3D’s Barracuda. From there, the data is streamed through a UDP protocol to the main VTube Studio frontend while utilizing the Live2D animations and user interface controls.

After covering the basics of how the software works, we’re then given a demonstration of how color tinting and physically based rendered lighting can also be applied to the Live2D models, creating the perception of real-time dynamic lighting.

You can also customize the background, import different scenes, and even setup hotkeys so you can change face-expressions, load in custom macros, and create specific reactions to liven up the livestream.

There are already some 3D solutions that behave similar to what VTube Studio can do, but if you were more-so interested in utilizing hand-drawn characters, as depicted in the video above, then VTube Studio is definitely the way to go.

If YouTube wasn’t imploding then I would probably say there were exciting times ahead for virtual-tubers, but at the moment it’s not looking too good for enthusiasts looking to get into content creation, unless of course you were just doing it for the heck of it and not to actually generate any significant revenue out of the endeavor.

You can learn more about VTube Studio from Denchi Soft by checking out the GitHub page.

(Thanks for the news tip Wondy Bergers)