Youtuber Harrison Kinsley has used an AI tool named GameGAN to recreate a highway stretch from GTA 5. A video showing off the results – named GTA5: GAN Theft Auto – is worth a watch.
Using GameGAN, Kinsley collaborated with developer Daniel Kukieła on this two-month project, which followed a similar idea last year that recreated Pac-Man by watching another AI play through the game.
Playing a Neural Network’s version of GTA V: GAN Theft Auto Watch on YouTube
Neither of the two collaborators came in with much knowledge of GANs, Kinsley told Eurogamer. “It was a lot of trial and error and just small tweaks and seeing if and how they improved. We didn’t have much knowledge, so we had a lot of learning to do, and still do.”
GameGAN is a generative adversarial network created by Nvidia that learns to visibly imitate a desired game by ingesting screenplay and keyboard actions during training.
Every GAN consists of two competing networks – a generator and a discriminator. The generator is trained on a sample dataset (the highway) and then told to produce content based on what it saw. Meanwhile, the discriminator will compare the output of the generator with the original dataset, and in the process train its counterpart to produce content close to the source material. In this case the source material being GTA 5.
In total, Kinsley and Kukieła had 30 upsampler models trained along with 15 GameGAN models before settling on a final one.
“The very first attempt worked way better than we expected though,” Kinsley continued, “so it was overall pretty exciting. We were just trying to do a lot of experimentation.”
The playable demo consists of driving down a short highway stretch in GTA 5. There are some impressive details visible in the demo, such as the shadow underneath the car, accurate sunlight reflections in the rear window that change as the car moves around, and the mountain in the distance, which gets closer as the car nears.