Today, the journal Nature is publishing our latest research, which introduces the first World and Human Action Model (WHAM). The WHAM, which we’ve named “Muse,” is a generative AI model of a video game that can generate game visuals, controller actions, or both.
In our research, we focus on exploring the capabilities that models like Muse need to effectively support human creatives by demonstrating how to develop research insights to support creative uses of generative AI models.
In our research, we focus on exploring the capabilities that models like Muse need to effectively support human creatives by demonstrating how to develop research insights to support creative uses of generative AI models.
Categoría
🎮️
VideojuegosTranscripción
00:00Hey, it's great to be here.
00:02You might have seen like some exciting news came out about
00:05work that our teams are doing on the world human
00:08action model.
00:09So Katia and Dom Katia from our Microsoft Research team.
00:12Dom from Ninja Theory. We're here to talk about this
00:16interesting work and how it impacts our kind of future
00:19in research or future in game development.
00:22It's also interesting note that this model was just published
00:26on Azure Foundry.
00:27The weights for the model were just published on Hugging
00:30Face.
00:31Because we want to make sure that the industry of
00:34creators and researchers are able to engage with this work
00:37and we can all learn together.
00:39We definitely look at this work as a unique opportunity
00:42to kind of leverage the intersection of art and science,
00:46which is what video games are to grow with the
00:48creator community.
00:49That's what these tools are about.
00:52We think there's a really unique opportunity to use technology
00:55to enable creators to do things that they've never done
00:58before.
00:59We have a great great opportunity with two teams that
01:02happen to be frankly right next to each other in
01:04Cambridge in the UK in our game studios and Microsoft
01:07Research,
01:08and it's been awesome to see the collaboration with between
01:11those two teams.
01:12So what I'm going to ask is I want to
01:14have Katia talk about how they see this technology in
01:17the work that they've done.
01:19Absolutely thank you so much.
01:21Phil and Dom fantastic to be here and reflect on
01:24the work that is coming out today.
01:26A key to this work from the very beginning was
01:29to try and understand how we can really ground the
01:33development of generative AI models in an actual understanding of
01:38what users need.
01:39In this case, what are the kinds of model capabilities
01:43that would be needed in order to really empower creatives
01:47to allow to let them explore ideas more quickly,
01:51more deeply than they might be able to do with
01:55traditional tools.
01:57And so in this work we grounded our generative AI model
02:01development in a user study with 27 game creatives and
02:06we were able to tease out that aspects like iterative
02:10tweaking and divergent thinking,
02:13which have already been well established in the computer supported
02:19creativity research. Those continue to be really key when creatives
02:24want to engage with these types of models,
02:27but often they're an afterthought in model development.
02:31So we turned that on its head.
02:33We started from those capabilities and then try to really
02:38systematically push on the model capabilities that might be needed
02:43in order to support tweaking,
02:45and rapid interaction. And so that is how we developed
02:49WAM and focus specifically on those model capabilities of consistency,
02:54diversity and persistency. And you can read a lot more in
02:57the research publication and the demo videos associated with that.
03:02I'll tell you one of the things that I get
03:04excited about.
03:05You know, one of the things we care a lot
03:08about at Xbox is game preservation,
03:10and I think about an opportunity to have models learn
03:13old games, games that were maybe tied to unique pieces
03:17of hardware,
03:18where that engine on that hardware is kind of time
03:20will erode the amount of hardware that's out there that
03:24can actually play a game.
03:25But you could imagine a world where from gameplay data
03:28and video that a model could learn old games and
03:31really make them portable to any platform where these models
03:34could run.
03:35I think that's really exciting.
03:37We talked about game preservation as an activity for us
03:40and this these models and their ability to learn old
03:43games and their ability to learn completely how a
03:46game plays without the necessity of the original engine running on
03:49the original hardware,
03:50I think opens up a ton of opportunity,
03:52but the fact that this is research that was done
03:55on top of bleeding edge for Ninja Theory.
03:57Dom, why don't you talk about how bleeding with the
04:00role that bleeding edge plays and how you as a
04:03creator think about this technology?
04:05Yeah, thanks Phil. Yeah,
04:06so as you mentioned, catchier and catches team and our
04:09team here in Ninja Theory.
04:11We happen to be in the same city in Cambridge
04:13in the UK.
04:14We're just down the road from each other,
04:16so when we joined Xbox Game Studios we were introduced to
04:19catcher and we started to think about how could we collaborate
04:23and we've been working together now for a few years
04:26and catcher asked if they could use bleeding edge as
04:29a multiplayer catalog title as a test bed for their
04:32research and for this technology.
04:34We were happy for them to do that and what we
04:37are looking at today is really the fruits of the
04:40work that catcher and the team have done and I
04:43think for us as a as a game creator,
04:45the thing that I'm always really interested in is how
04:49can we help our teams do more?
04:51How can we help them to push their creative ambitions
04:54and their creativity even further?
04:56So. Technology like this for me and I think for
05:00our students,
05:01I think it's a great opportunity to be able to
05:04use technology like this for me and I think for
05:07our studio here at Ninja Theory is not about using
05:11AI to generate content,
05:13but it's actually about creating workflows and approaches that allow
05:18our team here of 100 creative experts to do more
05:21to go further to iterate quicker to ideate quicker to
05:25bring their what's the ideas in their heads to life
05:29in a tangible form for other people to use.
05:32So, you know, although this is technology that we haven't
05:36used in the creation of our games and we don't
05:40intend to use this technology for the creation of content,
05:43I think the interesting aspect for us that is exciting
05:47is how can we use technology like this to make
05:51the process of making games quicker and easier for our
05:54talented team so that they can really focus on the
05:58things that they're passionate about?
06:00So they can really focus on the thing that I
06:03think is really special about games,
06:06which is that human creativity.
06:08Games to me are really creators.
06:13Using the medium of games to talk to their audience
06:17to communicate through the medium of interactivity,
06:20and that is core to game creation for me.
06:22So what's important with this is that we have the
06:26human creativity,
06:27but through this technology we can empower them to
06:30do more and push their own dreams further than
06:33they have been before.
06:35Yeah, like we're always thinking about how do we
06:37put tools in the hands of creators so that they
06:40can create things that they haven't been able to
06:43create before.
06:44I know you and the team will continue to make
06:47progress.
06:47We obviously have game developers conference coming up.
06:50It's always a unique opportunity for us to talk
06:52about more of the innovation that's happening between our
06:56creative teams,
06:57our platform teams and our research teams,
06:59which we will get to show next.
07:01But congrats again on all of the progress.
07:04Really, really exciting.
07:05Some thanks so much and thanks for the fantastic collaboration.