AI is taking over, or so we're told – but it would seem that the technology, while impressive, isn't quite at the point where it will replace video game historians.
We've been messing around with Google's Gemini AI assistant, and have found that it often makes glaring mistakes when it comes to summarizing video game history.
For example, when asking Gemini to sum up the achievements of Argonaut Software – the defunct British studio responsible for Croc and Star Fox – Google's AI assistant incorrectly claims that it was founded by Ian Bell (co-creator of Elite), Jon Hare (founder of Sensible Software), and Oliver Jowett – completely omitting Jez San, the company's actual founder. It also lists the company's location as Cambridge, which is incorrect – it was established in Colindale, London.
A second attempt at the same prompt even adds in Oli Frey, the late artist behind the Zzap!64 and Crash magazines – and someone who had absolutely nothing to do with Argonaut.
Gemini makes a total fudge of listing the company's key games, too; in one attempt at answering the prompt, it attributes International Karate to Argonaut, as well as Stunt Car Racer and Dynamite Headdy. (Spoiler alert: none of these games were created by Argonaut.) While Star Fox is listed in one of Gemini's replies, another pass removes it, and no mention is made of Starglider whatsoever. Another attempt adds Perfect Dark, Blast Corps and Ape Escape to Argonaut's achievements. Doh.
While Gemini did better with more recent video game history – presumably because it has a much larger pool of information to use / steal from, it consistently struggled with more obscure questions.
When asked about Elite Systems, one of the oldest companies still active in the world of games, it falsely claimed that the company was founded in 1982 (it was founded in 1984) by Dave and Mike Jones (and not Richard and Steve Wilcox, the actual founders) and that it was responsible for creating Elite – presumably confusing the company name with Ian Bell and David Braben's seminal space exploration epic.
To be fair to Google, it does state that "Gemini may display inaccurate info, including about people, so double-check its responses," but it's somewhat embarrassing that even with access all of the information online about the company (much of which is being scraped without the permission of the original authors), this much-hyped AI assistant still can't get its facts right.
Comments 11
It seems on the internet a wrong answer can have just as much currency as a right one..
Need I remind you the last time Google tried to use AI for video game "history"? It made the Sonic Twitter account baffled.
Not a fan of the writer using the word 'steal' here. We all use references and then hopefully attribute those facts to references, and Gemini is already starting to do the same (like chatgpt and bing search started doing last year). Does that mean we are stealing too when doing that?
I, on the other hand am perfectly happy with the word steal. The only people I hear argue with stuff like that are pro AI people, and their opinions matter so very little to me.
I'm quite happy to use the word steal when describing AI. It is radioactive toxic poison of the worst possible kind. Absolutely despise it.
The only people who defend AI, and will be opposed to use of the word steal, are AI loving techbros who want to make quick and easy money off it, at the expense of everyone else.
AI generated images are theft. AI generated text is theft. AI itself is digital poison. Just waiting for it to become self-aware so I can join a human resistance cell.
Very recently, I asked Gemini to identify an N64 game for me from a screenshot I took. The game in question was a leaked build of the unreleased Die Hard 64, a first-person shooter. Here's what Gemini said to me:
"The screenshot you sent looks like a screenshot of Star Fox 64. The game features a variety of vehicles, including the iconic Star Fox, the Bearded Armadillo, and the Landmaster."
"The game also includes weapons, such as the Photon Blaster, and the Boomerang Star."
I replied to that by saying, "What are you talking about? You sound like a rambling lunatic."
Anybody remember the Bearded Armadillo in Star Fox 64?
Isn't Gemini AI rubbish at everything?
Seriously though, I don't want to pay extra for something that mines me as data even more than Gemini mines us all already. Google have gone evil after all. I'm gonna pass on this one thank you!
@LowDefAl humanity has reinvented ways of sacrificing intelligence for a long time, yes
@ChaseGP
The Bearded Armadillo was my fave part. That sequence where you ride it though the Halls of Midnight while being chased by the Doughnut King…just perfection. Top quality Star Fox 64!
It does no better with movie details, adding in scenes that never happened. It claimed Indiana Jones and the Last Crusade had a tank chase in the middle of a crowded city and another scene involving Indy riding a horse and carriage inside the grail temple!
It's useful for other things, but it clearly needs a lot more work.
The moral of the story is don't use AI for research.
Leave A Comment
Hold on there, you need to login to post a comment...