Skip to content
Tech News
← Back to articles

The Education of a Doomer

read original get Superintelligence by Nick Bostrom → more articles
Why This Matters

A personal essay from a tech writer documenting a public shift from AI optimism to serious concern, walking through the specific arguments — economic, political — that changed their mind. It matters because such 'conversion' narratives from insiders shape elite opinion on AI policy, and it reframes the AGI debate away from job loss and meaning toward power: if humans become economically useless, states may have no incentive to sustain them.

Key Takeaways
Worth a Look

Superintelligence by Nick Bostrom — If this author's shift from AI optimist to worrier resonates, Bostrom's Superintelligence is the foundational text laying out the alignment and control problems that drive such changes of mind. It's a dense, rigorous read that maps the very scenarios the post gestures at, from economic displacement to truly general AI.

See Superintelligence by Nick Bostrom on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.

If you follow me on Twitter, or read this blog, you have noticed that I went from being generally optimistic and excited about AI to being extremely concerned. And I thought, I should explain why I changed my mind. Each section of this post is about some aspect of AI where my views shifted. I begin each section by explaining my previous beliefs, and why I held them, and then explain why those beliefs changed.

Contents

Economics

Automation has been good. We’ve automated 99% of the jobs people did in 1790, and the result is not mass unemployment, rather, we are wealthier, healthier, more educated, we have more leisure, etc. I had this vague, inductive idea that, while I can’t predict what jobs will exist after AGI, there will be demand for me to do something. If nothing else, the much higher economic growth of the post-AGI world means that the human niche, while small in absolute terms, might be much larger than today’s economy.

And this may yet be true. Or it may show a lack of imagination on my part. If AI develops such that we have enduring complementarity between humans and AIs, then we might still have jobs in the post-AGI future. But if AI becomes truly general, the G in AGI, and on top of that it is vastly smarter, faster, and cheaper than humans, then there might be nothing for us to do, except live off UBI.

When people talk about UBI, they typically worry about the problem of meaning in a world without work. I’ve never had this worry. When I was funemployed last year, I spent my time reading books and writing code and hanging out with friends. If the future is an infinite UBI-funded vacation, I know what I’ll do. “Before the Singularity, read books and throw house parties; after the singularity, etc.”

But then I started thinking about the political consequences of AGI, and started writing about it:

No-One Escapes the Permanent Underclass: if humans are economically useless, the state does not need them. Why pay out UBI to people who have neither economic nor political power?

When The Future Doesn’t Need Us: factory workers can sabotage the machines, truck drivers can shut down logistics. But if humans are economically useless, there is no way for people to veto the political order by withdrawing their contribution to it. And if the AIs fight wars, then the state can be arbitrarily repressive.

Mathematics Without Mathematicians: most arguments about humans moving “one job up” fail because the AI can do those jobs too.

... continue reading