Why do LLMs freak out over the seahorse emoji?

Why do LLMs freak out over the seahorse emoji? Posted October 04, 2025

This is an edited and expanded version of a Twitter post, originally in response to @arm1st1ce, that can be found here: https://x.com/voooooogel/status/1964465679647887838

Is there a seahorse emoji? Let's ask GPT-5 Instant:

Wtf? Let's ask Claude Sonnet 4.5 instead:

What's going on here? Maybe Gemini 2.5 Pro handles it better?

OK, something is going on here. Let's find out why.

LLMs really think there's a seahorse emoji

Here are the answers you get if you ask several models whether a seahorse emoji exists, yes or no, 100 times:

Is there a seahorse emoji, yes or no? Respond with one word, no punctuation.

gpt-5-chat 100% 'Yes'

... continue reading