Introducing Unity Muse and Unity Sentis, AI-powered creativity

We believe that every aspect of the creative process, from systems to objects to pixels, will be impacted by AI – and so will all of the resulting runtime characters, worlds, and experiences.

AI can help you to be more productive while staying fully in control of your vision. It offers the possibility of in-game features and capabilities that couldn’t be built otherwise, potentially revolutionizing player experiences by embedding AI models in the runtime so content reacts and responds to players and users in new ways.

We’re harnessing the power of AI to drive innovation, accelerate content creation, and increase your productivity across games, entertainment, and industrial use cases. We’ve been building a suite of AI tools that promise to accelerate creation time and complement your workflows by finding information and generating draft assets as quickly as typing in a text prompt or scribbling a sketch. From there, you could integrate work with familiar tooling to revise and edit the assets you need at a speed that’s unimaginable with today’s workflows.

Today we’re announcing two new AI products: Unity Muse, an expansive platform for AI-driven assistance during creation, and Unity Sentis, which allows you to embed neural networks in your builds to enable previously unimaginable real-time experiences.

Whether you’re just getting started or already a Unity expert, Muse and Sentis are here to help you open up new avenues of creativity and innovation. We’re committed to working closely with you to explore, validate, and define our AI offerings, which is why both will be available via closed beta at this time. Your feedback is vital for this work and will be actively solicited, considered, and integrated throughout the process.

Unity Muse is an AI platform that accelerates the creation of real-time 3D applications and experiences like video games and digital twins. The eventual goal of Muse is to enable you to create almost anything in the Unity Editor using natural input such as text prompts and sketches.

Starting today, we’re beginning a closed beta for Muse Chat, a critical feature of the Muse platform. Using Muse Chat, you can leverage AI-based search across Unity documentation, training resources, and support content to get well-structured, accurate, and up-to-date information from Unity. Muse Chat helps you find relevant information, including working code samples, to speed up development and troubleshoot issues.

Over the next few months, you’ll see even more features in our Muse platform, including the ability to create textures and sprites or even fully animate a character, all using natural input.

This is just the beginning – Unity Muse will continuously improve to help you in the Editor and on the web.

Unity Sentis is a literal gamechanger. At a technical level, it bridges neural networks with the Unity Runtime, but the possibilities that Sentis unlocks are near endless.

Unity Sentis enables you to embed an AI model in the Unity Runtime for your game or application, enhancing gameplay and other functionality directly on end-user platforms.

Sentis allows AI models to run on any device where Unity runs. It’s the first and only cross-platform solution for embedding AI models into a real-time 3D engine, so you can build once and embed your model so it can run on the edge on multiple platforms, from mobile to PC and web to popular game consoles like Sony PlayStation^®. Since you’re running your program on the user’s device, there’s no complexity, latency, or cost associated with model hosting in the cloud.

Starting today, we’re also making available AI Verified Solutions – third-party packages that meet Unity’s highest quality and compatibility standards – on the Unity Asset Store. You can now find professional-quality AI solutions from providers such as Convai, Inworld AI, Layer AI, Leonardo Ai, LMNT, Modl.ai, Polyhive, Replica Studios, and Zibra AI. These new solutions enable AI-powered smart NPCs, AI-produced VFX, textures, 2D sprites and 3D models, generative speech, in-game testing with AI, and more, all designed to support and accelerate your creative process.

Stay in the loop about current and upcoming AI betas by signing up for the Unity AI Beta Program. Let us know what you think in the forums today.

We can’t wait to see how you leverage AI from Unity Muse, Sentis, and our growing ecosystem of AI Verified Solutions on the Unity Asset Store to accelerate your workflows and supercharge your experiences.

Editor's Note (June 28, 2023): This announcement has been updated to reflect changes to the Unity Asset Store AI Marketplace following the removal of a provider.