Be part of our day by day and weekly newsletters for the most recent updates and unique content material on industry-leading AI protection. Study Extra
Right here’s a secret. The commercial metaverse isn’t lifeless. Moderately, it’s taken the type of tasks for OpenUSD, the common 3D knowledge interchange framework for issues like digital twins.
Nvidia right this moment introduced main developments to Common Scene Description, or OpenUSD, that may increase adoption of the common 3D knowledge interchange framework to robotics, industrial design and engineering, and speed up builders’ skills to construct extremely correct digital worlds for the subsequent evolution of AI.
“This Siggraph I’m excited to announce Nvidia is bringing generative AI to OpenUSD. We built the world’s first generative AI models that can understand OpenUSD-based language, geometry, materials, physics and spaces,” mentioned Rev Lebaredian, vp of Omniverse and simulation know-how at Nvidia, in a press briefing.
By way of new OpenUSD-based generative AI and Nvidia-accelerated growth frameworks constructed on the Nvidia Omniverse platform, extra industries can now develop functions for visualizing industrial design and engineering tasks, and for simulating environments to construct the subsequent wave of bodily AI and robots.
The corporate made the bulletins throughout Nvidia CEO Jensen Huang’s speak on the Siggraph laptop graphics convention in Denver, Colorado.
The brand new choices embody Nvidia NIM microservices for AI fashions that may generate OpenUSD language to reply consumer queries, generate OpenUSD Python code, apply supplies to 3D objects, and perceive 3D area and physics to assist speed up digital twin growth. As well as, new USD connectors to robotics and industrial simulation knowledge codecs and developer instruments let customers stream huge, absolutely Nvidia RTX ray-traced datasets to Apple Imaginative and prescient Professional.
“The generative AI boom for heavy industries is here,” mentioned Lebaredian. “Until recently, digital worlds have been primarily used by creative industries; now, with the enhancements and accessibility Nvidia NIM microservices are bringing to OpenUSD, industries of all kinds can build physically based virtual worlds and digital twins to drive innovation while preparing for the next wave of AI: robotics.”
Generative AI involves USD With Nvidia NIM
The world’s first generative AI fashions for OpenUSD growth, developed by Nvidia, will probably be out there as Nvidia NIM microservices. The fashions allow builders to include generative AI copilots and brokers into USD workflows, broadening the probabilities in 3D worlds and serving to pace the adoption of USD throughout a brand new vary of commercial sectors, like manufacturing, automotive and robotics.
“Omniverse is a platform for developing and deploying physically based industrial digitalization applications. It’s the feedback loop for AI to test and validate their skills before being deployed to the real world,” Lebaredian mentioned in a press briefing.
Omniverse is getting used at over 10,000 firms now and 30 firms are contributing to OpenUSD.
“On the 3D side, we have a pretty good sense of where we want to be in terms of constructing 3D virtual worlds, making it easy for people to construct them using these AI. But we’re still not there yet. Creating 3D worlds is really, really hard,” Lebaredian mentioned. “So we pay very close attention to what’s happening in the research side of this.”
The microservices out there in preview are:
- USD Code NIM microservice — solutions normal data OpenUSD questions and mechanically generates OpenUSD-Python code primarily based on textual content prompts that may then be inputted into an OpenUSD viewing app, similar to usdview from Pixar, or an Nvidia Omniverse Package-based utility to visualise the corresponding 3D knowledge.
- USD Search NIM microservice — allows builders to go looking via huge libraries of OpenUSD, 3D and picture knowledge utilizing pure language or picture inputs.
- USD Validate NIM microservice — checks the compatibility of uploaded recordsdata in opposition to OpenUSD launch variations and generates a completely RTX-rendered, path-traced picture, powered by Nvidia Omniverse Cloud APIs, or utility programming interfaces.
Newly introduced microservices that will probably be out there quickly are:
- USD Structure NIM microservice — allows customers to assemble OpenUSD-based scenes from a collection of textual content prompts primarily based on spatial intelligence.
- USD SmartMaterial NIM microservice — predicts and applies a practical materials to a computer-aided design object.
- fVDB Mesh Era NIM microservice — generates an OpenUSD-based mesh, rendered by Omniverse Cloud APIs, from point-cloud knowledge.
- fVDB Physics Tremendous-Res NIM microservice — performs AI tremendous decision on a body or sequence of frames to generate an OpenUSD-based, high-resolution physics simulation.
- fVDB NeRF-XL NIM microservice — generates large-scale neural radiance fields in OpenUSD utilizing Omniverse Cloud APIs.
Foxconn, a world manufacturing chief with greater than 170 factories worldwide, is already benefiting from Nvidia’s computing platform, utilizing NIM microservices and Omniverse to create a digital twin of a manufacturing facility underneath growth.
“Digital twins will help us accelerate the next wave of industrial manufacturing and autonomous machines,” mentioned Zhe Shi, chief digital officer at Foxconn, in a press release. “Nvidia Omniverse and the new NIM microservices will democratize the ability to develop digital twins and help our teams build physically
based virtual factories faster than ever.”
WPP, a world chief in advertising and marketing and communications providers firm, is an early adopter of USD Search and USD Code NIM microservices, implementing them in its generative AI-enabled content material creation pipeline, constructed on Nvidia Omniverse, for purchasers similar to The Coca-Cola Firm.
“The beauty of the innovation is how compatible it is with the way we work, and that it leverages open standards — accelerating not only future work, but allowing us to continue to build on and extend the usefulness of all our previous investments in standards like OpenUSD,” mentioned Stephan Pretourius, chief know-how officer at WPP. “Using Nvidia NIM microservices with Nvidia Omniverse has made it possible for us to launch innovative new production tools with companies like The Coca-Cola Company at unprecedented speed.”
“A designer or artist builds a ground truth 3D scene that conditions the generative AI. They assemble their scene in Omniverse, which lets them aggregate brand approved assets like a Coke bottle and various models for props and the environment into one scene,” Lebaredian mentioned. “They place their scene exactly how they want and render the view they want. Then they take this 2D render and feed it into edify image or any other image generator along with the text prompt. The result is an upraised version of the basic stage when they want to move their bottle or plate of wings to a specific place. They do it in Omniverse and re-render the view, then reproject the AI.”
He added, “This results in a controllable and collaborative content creation process. WPP, the world’s largest market agent marketing agency, is the first to adopt this workflow. Using it with the Coca Cola company to scale its global advertising campaigns.”
USD connectors convey generative AI to extra industries
A collection of recent USD connectors for robotics knowledge codecs and streaming to Apple Imaginative and prescient Professional opens the portals of OpenUSD interoperability and superior authoring to extra industries.
Nvidia and Siemens, a world chief in industrial automation and software program, are extending their collaboration to facilitate extra industrial workloads utilizing OpenUSD. Siemens will combine OpenUSD pipelines with its Simcenter portfolio of simulation applied sciences to assist evidence-based decision-making and collaboration amongst key stakeholders.
This integration allows high-fidelity, real-time, photorealistic visualization of advanced simulation knowledge, offering deeper insights right into a product’s efficiency inside its real-world working surroundings. The work will construct on Siemens’ efforts to include Omniverse into its Teamcenter Product Lifecycle Administration portfolio.
Nvidia additionally launched a connector from Unified Robotics Description Format to OpenUSD, letting roboticists seamlessly convey their robotic knowledge throughout functions, together with for design, simulation and reinforcement studying.
To additional advance OpenUSD ecosystem enlargement, Nvidia introduced the OpenUSD Trade software program growth package, enabling builders to construct their very own sturdy OpenUSD knowledge connectors.
New developer instruments and APIs to stream large-scale OpenUSD scenes from an utility constructed on the Omniverse platform to Apple Imaginative and prescient Professional by way of the Nvidia Graphics Supply Community at the moment are out there in early entry.
“OpenUSD is revolutionizing the way we create and interact with 3D content,” mentioned Steve Might, chief know-how officer of Pixar and chairman of the Alliance for OpenUSD (AOUSD), in a press release. “Now, with these new services and APIs for OpenUSD built by Nvidia, we expect to see accelerated growth and adoption of USD, paving the way for new users and industries to more easily engage with our ecosystem.”
OpenUSD ecosystem momentum
Final 12 months, Nvidia cofounded the AOUSD together with Pixar, Adobe, Apple and Autodesk. By way of AOUSD, Nvidia and different collaborators have introduced a brand new OpenUSD launch, progress on an OpenUSD core specification and new members.
Availability
The USD Search, USD Code and USD Validate NIM microservices can be found in preview on the Nvidia API catalog. The OpenUSD to URDF connector is now out there with Nvidia Isaac Sim.
Builders can get began integrating generative AI into OpenUSD workflows with new Omniverse developer instruments and a reference workflow for constructing a generative AI-enabled artificial knowledge pipeline with OpenUSD.
Study extra concerning the newest in generative AI and accelerated computing by tuning in to Nvidia founder and CEO Jensen Huang’s hearth chats at Siggraph, the premier laptop graphics convention, operating via Aug. 1 in Denver.