In the race to make AI models appear increasingly impressive, tech companies have adopted a theatrical approach to language. They keep talking about AI as if it's a person. Not only about the AI "thinking" or "planning" -- those words are already fraught -- but now they're discussing an AI model's "soul" and how models "confess," "want," "scheme" or "feel uncertain."
This isn't a harmless marketing flourish. Anthropomorphizing AI is misleading, irresponsible and ultimately corrosive to the public's understanding of a technology that already struggles with transparency, at a moment when clarity matters most.
Research from large AI companies, intended to shed light on the behavior of generative AI, is often framed in ways that obscure more than illuminate. Take, for example, a recent post from OpenAI that details its work on getting its models to "confess" their mistakes or shortcuts. It's a valuable experiment that probes how a chatbot self-reports certain "misbehaviors," like hallucinations and scheming. But OpenAI's description of the process as a "confession" implies there's a psychological element behind the outputs of a large language model.
Perhaps that stems from a recognition of how challenging it is for an LLM to achieve true transparency. We've seen that, for instance, AI models cannot reliably demonstrate their work in activities like solving Sudoku puzzles.
There's a gap between what the AI can generate and how it generates it, which is exactly why this human-like terminology is so dangerous. We could be discussing the real limits and dangers of this technology, but terms that label AI as cognizant beings only minimize concerns or gloss over the risks.
Don't miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.
AI has no soul
AI systems don't have souls, motives, feelings or morals. They don't "confess" because they feel compelled by honesty, any more than a calculator "apologizes" when you hit the wrong key. These systems generate patterns of text based on statistical relationships learned from vast datasets.
That's it.
Anything that feels human is the projection of our inner life onto a very sophisticated mirror.
... continue reading