Why do LLMs freak out over the seahorse emoji? Posted October 04, 2025
This is an edited and expanded version of a Twitter post, originally in response to @arm1st1ce, that can be found here: https://x.com/voooooogel/status/1964465679647887838
Is there a seahorse emoji? Let's ask GPT-5 Instant:
Wtf? Let's ask Claude Sonnet 4.5 instead:
What's going on here? Maybe Gemini 2.5 Pro handles it better?
OK, something is going on here. Let's find out why.
LLMs really think there's a seahorse emoji
Here are the answers you get if you ask several models whether a seahorse emoji exists, yes or no, 100 times:
Is there a seahorse emoji, yes or no? Respond with one word, no punctuation.
gpt-5-chat 100% 'Yes'
... continue reading