Outsmarting Humans Just One Step for AI Video Game Players

February 27, 2023

Speed around a French village in the video game Gran Turismo and you might spot a Corvette behind you trying to catch your slipstream.

The technique of using the draft of an opponent’s racecar to speed up and overtake them is one favored by skilled players of PlayStation’s realistic racing game.

But this Corvette driver is not being controlled by a human — it’s GT Sophy, a powerful artificial intelligence agent built by PlayStation-maker Sony.

Gran Turismo players have been competing against computer-generated racecars since the franchise launched in the 1990s, but the new AI driver that was unleashed last week on Gran Turismo 7 is smarter and faster because it’s been trained using the latest AI methods.

“Gran Turismo had a built-in AI existing from the beginning of the game, but it has a very narrow band of performance and it isn’t very good,” said Michael Spranger, chief operating officer of Sony AI. “It’s very predictable. Once you get past a certain level, it doesn’t really entice you anymore.”

But now, he said, “this AI is going to put up a fight.”

Visit an artificial intelligence laboratory at universities and companies like Sony, Google, Meta, Microsoft and ChatGPT-maker OpenAI and it’s not unusual to find AI agents like Sophy racing cars, slinging angry birds at pigs, fighting epic interstellar battles or helping human gamers build new Minecraft worlds — all part of the job description for computer systems trying to learn how to get smarter in games.

But in some instances, they are also trying to learn how to get smarter in the real world. In a January paper, a University of Cambridge researcher who built an AI agent to control Pokémon characters argued it could “inspire all sorts of applications that require team management under conditions of extreme uncertainty, including managing a team of doctors, robots or employees in an ever-changing environment, like a pandemic-stricken region or a war zone.”

And while that might sound like a kid making a case for playing three more hours of Pokémon Violet, the study of games has been used to advance AI research — and train computers to solve complex problems — since the mid-20th century.

Initially, AI was used on games like checkers and chess to test at winning strategy games. Now a new branch of research is more focused on performing open-ended tasks in complex worlds and interacting with humans, not just for the purpose of beating them.

“Reality is like a super-complicated game,” said Nicholas Sarantinos, who authored the Pokémon paper and recently turned down a doctoral offer at Oxford University to start an AI company aiming to help corporate workplaces set up more collaborative teams.

In the web-based Pokémon Showdown battle simulator, Sarantinos developed an algorithm to analyze a team of six Pokémon — predicting how they would perform based on all the possible battle scenarios ahead of them and their comparative strengths and weaknesses.

Microsoft, which owns the popular Minecraft game franchise as well as the Xbox game system, has tasked AI agents with a variety of activities — from steering clear of lava to chopping trees and making furnaces. Researchers hope some of their learnings could eventually play a role in real-world technology, such as how to get a home robot to take on certain chores without having to program it to do so.

While it ”goes without stating” that real humans behave quite differently from fictional video game creatures, “the core ideas can still be used,” Sarantinos said. “If you use psychology tests, you can take this information to conclude how well they can work together.”

Amy Hoover, an assistant professor of informatics at the New Jersey Institute of Technology who’s built algorithms for the digital card game Hearthstone, said “there really is a reason for studying games” but it is not always easy to explain.

“People aren’t always understanding that the point is about the optimization method rather than the game,” she said.

Games also offer a useful testbed for AI — including for some real-world applications in robotics or health care — that’s safer to try in a virtual world, said Vanessa Volz, a researcher and co-founder of the Danish startup Modl.ai, which builds AI systems for game development.

But, she adds, “it can get overhyped.”

“It’s probably not going to be one big breakthrough and that everything is going to be shifted to the real world,” Volz said.

Japanese electronics giant Sony launched its own AI research division in 2020 with entertainment in mind, but it’s nonetheless attracted broader academic attention. Its research paper introducing Sophy last year made it on the cover of the prestigious science journal Nature, which said it could potentially have effects on other applications such as drones and self-driving vehicles.

The technology behind Sophy is based on an algorithmic method known as reinforcement learning, which trains the system by rewarding it when it gets something right as it runs virtual races thousands of times.

“The reward is going to tell you that, ‘You’re making progress. This is good,’ or, ‘You’re off the track. Well, that’s not good,'” Spranger said.

PlayStation players will only get to try racing against Sophy until Wednesday, on a limited number of circuits, so it can get some feedback and go back into testing. Peter Wurman, director of Sony AI America and project lead on GT Sophy, said it takes about two weeks for AI agents to train on 20 PlayStations.

“To get it spread throughout the whole game, it takes some more breakthroughs and some more time before we’re ready for that,” he said.

And to get it onto real streets or Formula One tracks? That could take a lot longer.

Self-driving car companies adopt similar machine-learning techniques, but “they don’t hand over complete control of the car the way we are able to,” Wurman said. “In a simulated world, there’s nobody’s life at risk. You know exactly the kinds of things you’re going to see in the environment. There’s no people crossing the road or anything like that.”


CAPE CANAVERAL, Fla. (AP) — New research indicates the sun’s magnetic field originates much closer to the surface than previously thought, a finding that could help predict periods of extreme solar storms like the ones that slammed Earth earlier this month.

Top Stories


A pregnant woman was driving in the HOV lane near Dallas.

General News

NEW YORK – Meropi Kyriacou, the new Principal of The Cathedral School in Manhattan, was honored as The National Herald’s Educator of the Year.


Tornado Kills Multiple People in Iowa as Powerful Storms Again Tear Through Midwest

GREENFIELD, Iowa (AP) — Multiple people were killed when a tornado tore through a small town in Iowa and left a wide swath of obliterated homes and crumpled cars, while the howling winds also twisted and toppled wind turbines.

WASHINGTON  — House Republicans have summoned the leaders of Northwestern University and Rutgers University to testify about concessions they gave to pro-Palestinian protesters to end demonstrations on their campus.

JERUSALEM  — A group representing the families of hostages held in Gaza has released new video footage showing Hamas’ capture of five female Israeli soldiers near the Gaza border on Oct.

HALKIDIKI – Professionals and amateur golfers from Europe, the ideal weather, the impressive golf course of Porto Carras, and the excellent facilities of Porto Carras Grand Resort, created a unique setting for the international golf tournament Porto Carras Pro-Am Aegean Mini Tour, held from May 14-18, on the peninsula of Sithonia in Halkidiki.

Enter your email address to subscribe

Provide your email address to subscribe. For e.g. [email protected]

You may unsubscribe at any time using the link in our newsletter.