The incident is embarrassing for Google, given the extremely high profile of the Super Bowl and the extra scrutiny ads for it receive.
Blogger Nate Hawke - who spotted the error - said it was an example of "AI slop".
But it's not the first time the tech giant has found itself on the back foot over its AI products.
A year ago, Gemini was "paused" following criticism of the "woke" images it generated, such as an image of the US Founding Fathers which inaccurately included a black man.
It's even had previous issues with cheese.
In May last year, its AI overviews search feature was criticised after it told some users to use "non-toxic glue" when they searched for 'how to make cheese stick to pizza better'.
The search engine's AI-generated responses also said geologists recommend humans eat one rock per day.
Problems with AI tools are not limited to Google - in January, Apple was forced to suspend its news alert summariser after it invented a slew of inaccurate headlines.
Adverts at the Super Bowl themselves are also no stranger to controversy.
Last year, Uber Eats made a last minute change to its advertisement after criticism that it inappropriately made light of food allergies.