Video credit: Jitao Zhou/Rikkyo University
CNN
—
When you read “artificial intelligence,” you might think of new innovations like Chat GPT, but AI has been used in video games since the 1950s.
From Pacman’s trademark ghosts to autonomous decision-making in “The Sims,” AI is essential to things like creating customizable characters and storylines.
now, the rapid development of generative AI opens up a new frontier for video games: infinite open worlds, unique content, autonomous characters and the potential for faster game development.
Generative AI – an artificial intelligence that creates text, images and audio in response to a prompt – is set to shake up one of the signature components of video games: non-playable characters, known as NPCs. These characters usually have an established pattern of behavior and their mannerisms and speech are often inflated and unnatural.
“When we think about these NPCs, they seem a little strange,” says Alexis Roland, development director of La Forge China, the Chinese branch of the research and development department of video game publisher Ubisoft. “You can tell there’s something wrong with what you see or hear.”
Enter generative AI. Earlier this year La Forge launched Ghostwriter, an AI text generation tool designed to help writers create a wider variety of original dialogue for NPCs, and in 2022 tested new technology to help generate more realistic and natural gestures in an NPC that matches the tone and mood of their speech.
“It takes speech as input and generates body gestures as output, so we can imagine these NPCs expressing themselves with unscripted dialogue, having almost natural body animation synthesized from speech,” Rolland says.
Combining generative AI elements like dialogue and animation can create “a fully fleshed out, AI-powered NPC that can have a little more natural and unpredictable behavior,” Roland says.
Jitao Zhou, a student at Rikkyo University in Tokyo, is doing just that to generate more realistic NPCs that are smarter and less predictable.
“This NPC using deep learning does not have a fixed pattern, so it can have a greater variety of movements,” Zhou says, adding that more intelligent NPCs will make games more fun and challenging.
Some publishers already use generative AI in NPCs to make conversations more realistic. Chinese game company NetEase used ChatGPT to generate dialogue with NPCs in its recently released mobile game “Justice”, while Replica Studios recently introduced “smart NPCs powered by artificial intelligence” for game engine giant Unreal Engine, which allows developers of games to use AI to read dialogue to NPCs instead of hiring a human voice actor.
However, one risk of use AI-generated NPCs mean game designers can lose control of the game’s narrative, says Julian Togelius, an associate professor at New York University, where he conducts AI research. and games. “(NPCs) can say things that break the game or are rude or break the immersion,” he explains.
Creating “good” AI-generated NPCs that help the player is also much more difficult than creating enemies that fight against you, Togelius adds. “We haven’t seen as much progress in artificial intelligence that powers other characters in games, or that tries to model the player, or tries to generate the world — so we’re going to see a lot of progress in those directions.”
Open-world games like “Grand Theft Auto,” “Skyrim” and “Elden Ring” approach gameplay with non-linear quests and stories. This offers yet another opportunity for generative AI to change the gaming experience.
Togelius envisions a “huge open game” with endless possibilities, new cities, landscapes and people, each with its own backstory and interactive elements. By using player data gathered from previous gameplay, generative AI can create unique storylines and tailor-made quests for players placed at just the right level — “like a custom dungeon master,” he adds.
Some research is already being done in this area. Takehiko Hoshino, also a student at Rokyo University, has created an AI tool that he teaches to generate his own mazes and dungeons one field at a time, based on previous ones he’s encountered.
It’s still early in development, but Hoshino says the next step is to decorate the maze with features including “treasure chests, enemy characters, and other game-like features like traps and other tiles.”
Near-infinite open worlds are already possible to some degree: No Man’s Sky (2016) is “virtually an infinite game,” says Togelius, which uses a technique called procedurally generated content to create custom fauna, flora, geology, and atmospheric conditions for its planets – of which there are 18 quintillion unique variations.
To the average person, procedural content generation looks a lot like generative AI, Togelius says. But it uses algorithms that generate content based on predefined rules, based on data entered by the game developer.
While developers retain control over procedurally generated content, generative AI has the potential to develop unplayable levels or deviate in unintended ways from the game’s narrative. “Games have functional limitations, so levels must be completed and NPCs must not lie about the game world,” adds Togelius.
But generative AI can affect the game world in other ways.
Players now tailor gameplay to their own preferences and needs with user-generated content (UGC), a key component of many games, including “Fortnite,” “Minecraft” and “The Sims.” Generative AI can make the production of UGC easier and more accessible to players, as well as increase the quality of content.
“Generative AI has the potential to enable a much broader and more emergent set of personalized and reactive player experiences,” a spokesperson for Maxis, the developer behind “The Sims,” told CNN in an email.
“Today, player customization is limited by the complexity of the tools and (user experience) we can expose players to, but some of the new models can make it easier for the game to interpret and respond to what the player wants to do,” says Maxis.
While gamers are excited about the potential for gameplay, generative AI will likely influence development before it changes the user experience.
Maxis is developing generative AI tools, currently in various stages of maturity, that can support game designers, eliminating repetitive tasks and allowing developers to work on more interesting problems, according to a spokesperson.
At La Forge, generative AI tools like Ghostwriter or ZooBuilder, a 3D animator that animates quadrupeds based on videos, can help designers “speed up the most tedious part of their process so they can really focus on the more creative and the interesting part,” says Rolland.
Creators in all kinds of industries have expressed concerns about generative AI taking over their jobs, but Rolland is quick to add that this new technology won’t replace human game developers. Animators face a similar existential threat with the advent of motion capture, which Roland says hasn’t really affected jobs, but has become a tool for creating better graphics.
“We’ve never had as many animators as we have today, and we still need more. Motion capture really became part of their workflow as an additional tool,” says Rolland. “I think with generative AI it’s essentially the same – or at least we approach it with exactly the same mindset here at Ubisoft.” However, there are still many unanswered “legal and ethical aspects” of using generative AI, including artists’ copyrights, he adds.
La Forge is eager to explore opportunities, such as the potential to increase iteration speed, and independent game designers will also benefit, he adds. “This technology, which is becoming more available, will enable many of the smaller studios to produce games and scale their output and perhaps reach a higher quality than they would have without generative AI.”
Generative AI Ghostwriter and speech-to-gesture animation are “past the simple prototype stage,” Roland says, and La Forge is now exploring how these technologies will work in the development process.
“Video games are on quite a journey over the next decade or two,” says Togelius. “It requires us to change the way we think when it comes to game design, but I think when that happens, games are just going to get a lot better.”