Recently, a woman slowed down a line at the post office, waving her phone at the clerk. ChatGPT told her there's a "price match promise" on the USPS website. No such promise exists. But she trusted what the AI "knows" more than the postal worker—as if she'd consulted an oracle rather than a statistical text generator accommodating her wishes.
This scene reveals a fundamental misunderstanding about AI chatbots. There is nothing inherently special, authoritative, or accurate about AI-generated outputs. Given a reasonably trained AI model, the accuracy of any large language model (LLM) response depends on how you guide the conversation. They are prediction machines that will produce whatever pattern best fits your question, regardless of whether that output corresponds to reality.
Despite these issues, millions of daily users engage with AI chatbots as if they were talking to a consistent person—confiding secrets, seeking advice, and attributing fixed beliefs to what is actually a fluid idea-connection machine with no persistent self. This personhood illusion isn't just philosophically troublesome—it can actively harm vulnerable individuals while obscuring a sense of accountability when a company's chatbot "goes off the rails."
LLMs are intelligence without agency—what we might call "vox sine persona": voice without person. Not the voice of someone, not even the collective voice of many someones, but a voice emanating from no one at all.
A voice from nowhere
When you interact with ChatGPT, Claude, or Grok, you're not talking to a consistent personality. There is no one "ChatGPT" entity to tell you why it failed—a point we elaborated on more fully in a previous article. You're interacting with a system that generates plausible-sounding text based on patterns in training data, not a person with persistent self-awareness.
These models encode meaning as mathematical relationships—turning words into numbers that capture how concepts relate to each other. In the models' internal representations, words and concepts exist as points in a vast mathematical space where "USPS" might be geometrically near "shipping," while "price matching" sits closer to "retail" and "competition." A model plots paths through this space, which is why it can so fluently connect USPS with price matching—not because such a policy exists but because the geometric path between these concepts is plausible in the vector landscape shaped by its training data.