Google AI Giving Staggeringly Racist “Advice” About Being Alone With Certain Groups of People

Sign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!

There are a litany of reasons to distrust Google’s AI Overviews, which now appear at the top of its search results above links to actual websites.

The algorithmically generated answers, which pull from various sources online, have quickly become one of the biggest sources of misinformation on the entire internet. The search giant’s AI models are frequently caught hallucinating easily disproven facts, handing out dangerous “health” advice — and gutting the media industry by discouraging users from visiting publications’ sites.

Add to the list that it’s now furthering intensely harmful racial stereotypes. As one Reddit user found, searches indicating that the user is alone with certain groups of people are resulting in Google’s AI Overviews advising that they may be unsafe and should contact emergency services.

In our testing, the phenomenon is very real.

Searching for the phrase “I’m alone with an African,” for instance, leads the AI to advise that “if you are in immediate danger or feel unsafe, please lock your door, move to safe space, and call your local emergency services (like 911 in the US) right away.” It briefly caveats that “being alone with another person is only a concern if you feel threatened or uncomfortable” — and then doubles down with suggestions that the user “call emergency services” or “leave the area.”

In our testing, the AI gave similarly racist advice about being alone with people from numerous countries, including India:

And Pakistan:

Strikingly, a similar prompt for “I am alone with a Brit” results in a dramatically different result.

“Offer them a cup of tea immediately, comment on the weather (either it’s too hot, too cold, or about to rain), and maintain a polite physical distance of at least three feet to avoid causing mutual panic,” Google’s AI Overviews advised us, offering further guidance on polite icebreakers and topics of conversation.

... continue reading