The AI game dev push is patently absurd | This Week in Games
This patent for Game Development Via Generative AI raises a lot more questions than it answers
So here's what we're doing this week. First, we're going to have some fun looking at a silly generative AI game development patent application filed by a company I've never heard of that I expect nobody in the industry will take seriously.
Then we're going to look at a more serious generative AI patent applications filed by Electronic Arts, a company I assume you've heard of that everybody in the industry takes seriously.
Let's start with the fun stuff.
A Hog in Armor
Last week, the USPTO published a patent application for "Game Development Via Generative AI" that had been initially filed for in February.
It’s a pretty fun application (as far as these things go), so here’s a copy for anyone with an urge to read through it:
On the off-chance you aren't broken in the exact way I am that makes reading through that seem like A Good Time, I'll go ahead and summarize.
It's a patent application covering a game development platform that has a user enter natural text prompts the platform then feeds into generative AI tools like ChatGPT, DALL-E, GPT-4, Midjourney, Imagen, Bard, Stable Diffusion, and whatever else to create the look, personality, and dialogue of characters, as well as cutscene scenarios where they interact.
So there's a "character generation" process where you tell it what you want a character to be and it creates an image of it.
Adorable, right? But our first draft of Jek Porkins there isn't going to cut it. Game development is all about the hard work of iteration. So let's roll up our sleeves and offload that hard work to a computer.
That's one character done. What's that? A 3D model? Animation? Gameplay? Don't worry about it. The generative AI will do something.
What's important is that this little dude looks ripped straight out of, uh... "Starwars." Particularly the messed up Darth Vader nose with the vertical vents it keeps giving him.
Now let's give him a personality that really shows how generative AI makes narrative teams obsolete.
My kingdom for an "Inside the Actors Studio"-style show about games where a James Lipton-type asks the developer using this "tool" about his creative process.
Anyway, let's see how the generative AI handled this one.
Does the This Little Piggy nursery rhyme exist in your Starwars-inspired universe? It does now! And if you don't want it to, just go back and try again and again until you get something that... Well, it probably won't fit the project the way something bespoke and thoughtfully created for that purpose would, and it might take you just as long as rubbing a few brain cells together to come up with your own snarky pig-themed battle cry, but look on the bright side: you'll have wasted a lot of water and electricity in the process, and you might end up with something that could pass for "good enough" if nobody's thinking too hard about it.
The applicants for this patent understand the importance of not throwing away all that "work," so they also suggest using those pre-final draft images to represent the final character in different states, for example, if he has had his battle axe knocked away.
Ok, we now have a couple pictures of a pigman warrior man, some sub-Schwarzenegger banter, and an awkward line of dialogue papering over discrepancies in the game that players may notice. That's everything on the video game checklist, so we should be ready to ship, right?
Not quite. There's a whole lot more to making a game than that. In fact, we're only about halfway done, and the other half of game development is where the real magic happens: Cutscenes.
Thankfully, this patent application has us covered with a Cutscene Generation feature, which it shows off with a character created by the prompt "Steampunk professor with owl pet owl."
Next this guy needs to say something. But about what? The patent application imagines a user inputting the theme "Trying to unlock the knowledge of the universe written in several secret books." Sure, whatever. Let's see what it came up with.
Oh, the generative AI was given the slightest idea of a character's motivation and now it's just going to write the plot of the game for the user? There's an evil force the prompt said nothing about, and a greatest apprentice who I guess is the protagonist. And it's specifically six books now?
Well, the whole point seems to be that a computer makes the game for you, so let's just roll with it. The patent application then gives the professor here someone to chat with, creating a character out of the prompt "Brainy child speaks precociously."
I don't know. Besides the obvious weirdness with the professor and his "my my owl" (presumably the aforementioned owl pet owl), that kid doesn't seem particularly brainy or precocious to me; he just seems like a jerk. But I guess it wouldn't be the first time tech bros though being "smart" or "advanced beyond your years" was the same thing as being an asshole.
And finally, the patent turns to the lead image of this article, an illustration of an online game that uses speech recognition to hear what players want to say to NPCs, then uses a generative chat tool to respond.
This particular implementation is given all of three sentences in the patent application, and does not begin to get into the questions of how the fuzzy, unpredictable, and imprecise nature of the NPC's chatbot responses will be informed by or interpreted as in-game actions that would need to be specific and based on things that are actually relevant and impactful to the player experience at hand. (There's a huge possibility space for actually experimental games where chatbots might exist untethered to things like combat or quest goals, but the games this applicant envisions are decidedly more traditional and less interesting.)
Now obviously, I'm being a bit hard on this proposed "game generation platform" here. It's a patent application and not a finished product, after all. And it's unreasonable to hold the authors to the same standards of writing you would expect to see from people who create engaging fictional worlds and characters for a living. It's also not necessary to have every little question and edge case addressed before applying for a patent on a new technique.
But the tools described in this application are transparently ill-suited to actual game development. In fact, they appear to be nothing more than a user-interface to access AI image generation and chatbot tools, obvious applications of existing technology that don't even begin to hint at the myriad logistical concerns of actually implementing them into the game development process.
To me, this patent application reads like it came from some people who hadn't thought any of this through and had no answers for how this would ever work in practice, but sure would like to have a patent that big companies would need to pay them for if they want to use generative AI to make their games. If generative AI game development ends up being "a thing," these people want to own it, regardless of whether they really figured it out in the first place.
The patent applicant's name even gives away the game, in that sense: ChatbotTBD, LLC. By evoking perhaps the best-known generativeAI tool in ChatGPT, it identifies itself as a shameless attempt to ride a cresting wave of hype. By including Chatbot in the company name, it implies a severely limited and well-trodden set of actual use cases.
And finally, it punts on the question of what makes its product worth pursuing with a perfect little "TBD."
They're not so different, them and them
OK, OK, I've had my fun. The patent is bad. But that's just a small operation nobody's heard of over-reaching with a patent application, right? Surely an entrenched titan like Electronic Arts is taking a different approach.
Well, yes. Where ChatbotTBD went with a single very broad application of generative AI in game development, Electronic Arts has been filing numerous applications looking to claim far more specific applications of AI.
On the same day as the ChatbotTBD filing was made public, the patent office published a raft of 20 EA patent applications, including six that detail applications I would expect to be grouped under the AI as a marketing catch-all term (like machine learning, or use of models trained on past authored content to generate novel content).
Let's look at one of these, a "System for Automated Generation of Facial Shapes for Virtual Character Models."
Ok, so what is the problem this invention is looking to solve?
"Populating virtual worlds with many realistic-looking characters is far from trivial yet in high demand. The efficient generation of random realistic human heads is motivated by the need for a substantial number of background and non-player characters without hand-authoring them."
While EA can't compete with ChatTBD when it comes to entertaining visuals, it did at least give us this gem:
Sadly, that top row of faces in FIG. 5A isn't the output of EA's new-fangled tech. It's actually an example of the "grotesque or cartoonish heads" that can be produced by existing face creation systems when you crank the sliders all the way to one end. But because the patent says limiting the extremes of those sliders could lead to "a repetitive synthetic appearance breaking the fiction of the virtual world," it offers a new process using machine learning, which will instead produce repetitive synthetic-appearing faces like those in FIG. 5B. Progress!
Frankly, both rows remind me of the Monster Factory YouTube series, even if the heads produced by EA's new tech have admittedly had some of the rough edges literally sanded off.
Interestingly enough, Monster Factory started with a nice illustration of how models designed to generate "normal-looking" outputs can collapse when their own not-quite-normal products are fed back into them, something which is a very real problem companies using generative AI tools will need to deal with.
Back on topic, you don't actually need to solve a problem better than what's been done before in order to get a patent; you just need to solve it differently than what's been done before.
I'm skeptical as to whether or not this solution is any more efficient or produces better results than the go-to move of Mr. Potato Heading the whole thing, but that's not the point, and the patent even admits as such.
"It should be understood that the original applicant herein determines which technologies to use and/or productize based on their usefulness and relevance in a constantly evolving field, and what is best for it and its players and users. Accordingly, it may be the case that the systems and methods described herein have not yet been and/or will not later be used and/or productized by the original applicant."
In other words, this thing might not actually be useful, or relevant. And it's not just the face generation tech patent that has this. It's a bit of boilerplate you can find on a bunch of EA's patents these days.
We're really just taking EA at its word that its new technique for generating faces works at all. But if this way of generating character faces winds up being a thing, EA wants to own it, regardless of whether they really figured it out in the first place.
And that's fine, I guess. That's somewhat essential to the idea of a patent, and I have no doubt that EA's applications – and other such generative AI and machine learning patents being applied for by industry giants – are more substantial than those of ChatbotTBD.
But a lot of what I see with AI gaming patents these days suggests that companies are still in a speculative mode when it comes to incorporating AI into game development, still struggling to figure out exactly what it makes sense to use it for.
They might tell you they're throwing spaghetti at the wall to see if it sticks, but it looks to me like they're going for another, far more crude version of the same saying.
If you liked this piece, please consider signing up for the Unlosing Writer newsletter. It's free, but if you enjoy the work, I would greatly appreciate a monthly subscription or a one-time tip.
