With “Neo NPC,” Ubisoft provides a glimpse into future video games powered by generative AI.

“ We can enter the house by using the ladder to pass through this window. But maybe you have another idea ? » Controller in hand and headset microphone on ear, it’s the player’s turn to propose an action plan to succeed in the mission. But instead of clicking and selecting from a list of ideas displayed on the screen, he must speak them literally. “ It’s too dangerous, I give up »We dare. “ Don’t worry, we will do everything to protect you »Responds instantly and verbally, our virtual companion is displayed on the screen.

This improvisation session between humans and AI is one of three experiences powered by the generative artificial intelligence of Neo NPC (“Neo Non-Playable Characters”), a prototype designed by a team at Ubisoft Paris, bringing together engineers and screenwriters. is This is the first concrete example of the potential use of this technology in video games, which Figaro The test was announced at the Game Developers Conference event in San Francisco.

Also readLove, coaching, influence, customer service… Tomorrow, will we all be talking to AIs?

“We have been working on AI for years. But when we saw, during 2022, the acceleration of generative AI, we chose to study how best to integrate it into our games to improve our creative processes and, above all, to bring something new to players. »explains Figaro Guillemette Picard, Director of Production Technology at Ubisoft.

strategic topic

Internal experiments have been started since 2022 on various topics. The Paris team, made up of about 20 employees, has been working for almost two years on creating a more natural exchange between the player and the fictional characters that inhabit his adventure. Neo NPC is one of his prototypes. “ Establishing a team with this level of ambition is quite a strategic topic for us. »She continues.

While Ubisoft has its own research lab, La Forge, it has joined forces with third parties for this project. “ Generative AI is evolving so quickly that it seemed obvious to us to find external technologies to blend it with ours. », Guillemette Picard underlines. So the group turned to Nvidia and its AI Audio2Face, which automatically manages a character’s facial animation from an audio file. Another partner, American InWorld AI, whose language model (LLM) is the backbone of Neo NPC. “ We interacted with them daily to refine the model. »Narrative director Virginie Moser explains. “ This closeness to InWorld made it possible to design characters with greater depth » than what online services like Character.AI offer, adds Guillemette Picard.

“A lot of work” for the narrative team.

To achieve this, “ There was a lot of work upstream and iterations » From the narrative team, Virginie Moser explains. “ Never before have we delved so deeply into the profiles of characters who behave like improvisational actors. We must ensure that their answers are consistent by feeding the model with data about their personality, their history, their opinions, the way they speak, and natural outcomes. So it is necessary to work with dialogue writers and narration experts. » The team has also put safety measures in place so that characters don’t derail if the player tries to provoke them.

Neo NPC was tested throughout its creation with nearly a hundred players, but also Ubisoft employees. “ We do not underestimate the sensitivity of using generative AI, which is a complex topic. The approach we’ve taken is to examine what interests the player and our creative », Guillemette Picard underlines. During these tests, Virginie Moser was able to observe “ That half of the players developed sympathy for the sadistic character, and feared harming him by interrogating him to discover his secret. The other half just wanted to complete the mission. »

It will take time for these innovations to make their way into tomorrow’s games. “ But given the very rapid evolution of language models this learning is necessary », Technical Director continues. The group is specifically monitoring the miniaturization of LLMs, which will operate from gaming machines and not from data centers, thus significantly reducing the cost of this technology. “ I’m sure the only question is when will all this happen. »