There is an increasing overlap between computer graphics, the metaverse and AI and that overlap is on display this week at the SIGGRAPH 2022 conference, where Nvidia is revealing its latest set of software innovations for computer graphics.
Nvidia announced a series of technology innovations that bring metaverse and AI closer together than ever before. Among the announcements is the Nvidia Omniverse Avatar Cloud Engine, a set of tools and services designed to create AI-powered virtual assistants.
The company also announced multiple technology efforts to advance its computer graphics generation capabilities for metaverse applications. One of the efforts is the new NeuralVBD library, the next generation of the OpenVBD open-source library for sparse volume data. Additionally, Nvidia is working on enhancing the open-source Universal Scene Description (USD) format to help further enable metaverse applications.
“3D content is especially critical for the Metaverse as we need to put stuff in the virtual world,” Sanja Fidler, VP of AI research at Nvidia, said in a press briefing. “We believe AI is existential for 3D content creation, especially for the metaverse.”
Neural graphics AI-powered innovations are key to the future of the metaverse
Computer graphics are no longer simply rendered images, they can be much more with the concept of neural graphics.
Fidler explained that neural graphics aim to insert AI capabilities into various parts of the graphics pipeline. The addition of AI can accelerate graphics in various applications, including gaming, digital twins and the metaverse.
At SIGGRAPH 2022, Nvidia announced a pair of new software development kits (SDKs) with Kaolin WISP and NeuralVDB that apply the power of neural graphics to the creation and presentation of animation and 3D objects. Kaolin WISP is an extension to an existing PyTorch machine learning library designed to enable fast 3D deep learning. Fidler explained that Kaolin WISP is all about neural fields, a subset of neural graphics focusing on 3D image representation and content creation using neural techniques.
While Kaolin WISP is about speed, NeuralVDB is a project designed to help compact 3D images.
“Using machine learning, NeuralVDB introduces compact neural representations that dramatically reduce the memory footprint, which means that we can now represent the much higher resolution of 3D data,” Fidler said.
Defining the emerging metaverse
According to Rev Lebaredian, VP of Omniverse and simulation technology at Nvidia — one of the most important but probably the least understood aspects of creating the metaverse is the core technology needed to represent everything inside the metaverse.
For Nvidia, that technology is the open-source Universal Scene Description (USD) technology, originally developed by the animation studio Pixar. Nvidia’s Omniverse platform is built on top of USD.
“We’ve been hard at work advancing Universal Scene Description, extending it and making it viable as the core pillar and foundation of the metaverse, so that it will be analogous to the metaverse just like HTML is to the web,” Lebaredian said.
At SIGGRAPH 2022, Nvidia is announcing its plans for extending USD, including new compatibility suites with graphics tools and tools to help users learn how to use USD.
Get ready for lifelike virtual assistants with Nvidia Avatar Cloud Engine
Chatbots and virtual avatars are not a new phenomenon, but to date, they haven’t been particularly lifelike, but that could soon change thanks to the new Nvidia Avatar Engine.
Lebaredian explained that the Avatar Cloud Engine is a framework with the core technologies necessary to create avatars. The avatars are robots driven by artificial intelligence that enables them to converse, perceive and behave inside virtual worlds and the metaverse.
“The metaverse without human-like representations or artificial intelligence inside it will be a dull and sad place,” Lebaredian said. “We provide the toolkit of technologies necessary to construct avatars of different forms so that others can take these technologies and build their specific ideas around what avatars should look, feel and behave like in those worlds.”
