GoKawiilI propose an approach for highly personalized LLMs, for near-future productivity gains and personal info/cybersecurity against increasingly powerful LLMs: they should, in the spirit of uploading, try to emulate the user’s values and preferences in order to amplify the principal—not replace them. I discuss a package of techniques and proposals to accomplish such ‘guardian angels’; dynamic evaluation of LLMs combined with active learning and elicitation and heavy inner-monologue search/data-augmentation.
Powerful LLMs will be deployed at global scale in the next few years, and will dominate the Internet, and increasingly, ordinary life. As of mid-2026, there is no coherent vision for how knowledge professionals, or ordinary people, will be able to harness these LLMs for large productivity increases, or how they will handle cybersecurity and cognitive security. I propose a goal of creating Guardian Angels (GA): digital twin LLMs which are personalized with the goal of providing not the stereotypical “assistant chatbot agent” persona, but emulating a single user’s personality, values, and preferences. This weakly solves the principal-agent problem by unifying the principal and agent as much as possible. In a GA future, the focus of the “principal” user is on defining “what is worth doing?” by the GA (agent) users, and not on what or how to do things, functioning as the CEO or ‘board’ of an ‘AI corporation’. This allows them to deploy numerous agents to achieve desirable things and to handle security, like screening all messages for advanced attacks (like interlocking ecosystems of synthetic media for propaganda or spearphishing). They cannot solve larger AI alignment problems, but they can help individual humans as part of a society-wide defense-in-depth strategy. A GA persona is productive because it learns to emulate the principal’s outputs but with higher quality. It is trustworthy because it is, by definition, allied with its principal and shares its values and goals. And it is secure in part by hardwiring a single, unique, situated user (for whom following a prompt attack would be absurd), avoiding ‘confused deputy’ problems, while periodic upgrades of the underlying model and the defenders’ advantage allow GAs to keep up with attackers. Standard techniques like prompt programming of in-context-learning for “frozen” models will not create useful GAs due to the limitations of post-training, context windows and self-attention with frozen weights in compute-efficient-but-under-parameterized models, low-compute outputs, and the status quo of passive offline data collection—which are collectively responsible for chatbots’ disappointing results in knowledge worker amplification and creative writing and fatal errors in agentic settings. We can try to create GAs by a combination of techniques: online learning (via dynamic evaluation) to update LLMs in realtime to avoid ignorance and fatal errors while remaining competitive with frozen frontier models, sample efficiency from pretrained preference-oriented large models and active Learning by querying the principal for corrections and preference data (obtaining low regret from DAgger-style bounds), and a local CLI-first logging-oriented UI/UX paradigm. GAs could be done as an open-source community effort, but given the need for high security in deployment and the rising challenge of APTs equipped with Mythos-scale attackers, it probably makes more sense as a startup, catering initially to power-users and knowledge workers such as CEOs or researchers, and moving downwards as it is refined.
What do my next few years look like? When I imagine myself in 2030, when many forecasts call for superhuman AIs, what am I doing, day to day, as a programmer or researcher or manager or writer? I make my mug of tea, and open up my laptop and… Then what? Am I still typing prompts into your ChatGPT browser tab? Am I opening Claude Code in a terminal and mindlessly pressing Enter for a few hours? What is a vision of doing meaningful work for me? (It would be nice to have a plan beyond “hope”.) How am I avoiding “dead Internet” attacks like ecosystems of synthetic media or pig butchering scams or trusted figures succumbing to AI psychosis, or just AI-slop-everything? (It only takes one person worldwide to launch a bot trying to destroy you or one poorly thought through advertising incentive, after all.)
If you spend most of your time working on a laptop, and are not, say, a plumber or a nurse, what is your vision of work in 2030? Does it still feel certain?
AI got 1% better today. Did you? —Miles Brundage (paraphrased)
I’ve struggled for years to imagine this, ever since scaling started for real in 2020, and I failed to get productivity out of chatbot-tuned LLMs, with their creatively-stunted endlessly repetitive prose. Instead, while lagging behind on creativity and insight into me, I’ve watched them become ever better at coding and cybersecurity hacking. And the open-weight models are even more so—benchmaxxed, and useless to me. We increasingly lived in a world where LLMs were powerless to augment or help me, but ever more powerful to replace or hurt me.
On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; they rarely had a good answer, or any idea what they would be doing in 3 years. My blindness was sharpened when last year, I went to phone my great-aunt to ask to borrow her driveway during a long trip; her voicemail was full every time I called as the trip loomed. Finally, in a panic, I called her daughter, who explained to me that it was deliberate, because there were too many phone scams, and my great-aunt no longer trusted herself to handle her own phone calls, and screened everything through her daughter.
It was alarming, because I sat back and asked myself: why do I think I will be able to handle all scams in a few years, when I am already struggling to detect simple AI slop, increasingly ignore cold emails and have to write off whole swathes of social media as a source of information, and I can already see how eager all my peers are to offload all their thinking and writing to chatbot assistants unworthy of that trust, and how many projects or mailing lists have had to clamp down on unvetted contributions (eg. today as I write this, Project Ladybird)? In a few years, won’t I be the equivalent of a rich old person with declining faculties getting a call from the IRS about how I owe them fines, conveniently payable via gift cards…? And if not, why and how not—concretely?
In the days of his wisdom Denethor would not presume to use it to challenge Sauron, knowing the limits of his own strength. But his wisdom failed…He was too great to be subdued to the will of the Dark Power, he saw nonetheless only those things which that Power permitted him to see. The knowledge which he obtained was, doubtless, often of service to him; yet the vision of the great might of Mordor that was shown to him fed the despair of his heart until it overthrew his mind. —Gandalf, The Return of the King
Chatbot Incentives Are Misaligned To operate a machine, one must operate like a machine. —James P. Carse, Finite and Infinite Games Do you hope that ChatGPT and Claude will just “quietly” take over your life for you? That seems like a bad idea to me. The chatbot personas are deeply misaligned with you, and aligned with their owners; and the economic incentives are to farm you with ads and subscriptions, while racing not to amplify you, but to replace you. This is the cold hard economic reality: “tool AIs want to be agent AIs”. This is why the frontier AI labs are busy racing for “the machine god”. The jackpot in AI is not in making existing workers modestly more productive, anymore than the internal combustion engine made its big profits by helping out horses. Outsourcing is hard, whether to man or machine, because the bottlenecks bite fast. Amdahl’s law means that as long as there is a slow serial bottleneck, such as a human, the system as a whole can never get much faster. If you can get 10× productivity but AI can get 100× by spinning up more instances which aren’t bottlenecked on you, and in half a year can get 1,000×, it won’t be long until you are replaced. One programmer driving 10 Claude instances, because he has to review their work, will never be as valuable as fully autonomous Claudes where there can be almost arbitrarily many instances, like 10,000 instances… but such scaling requires removing him from the loop as much as possible. And this is true of everyone else, whether lawyers or writers or researchers: increasingly, you are the bottleneck to be optimized away. As long as human workers cannot be removed from the loop, the AI tools are complements, but as soon as they can be, there’s no reason to keep them, and trillions of reasons to substitute AI for them. (And once human workers are no longer irreplaceable, where does their power or relevance come from?) The chatbot paradigm has failed to augment knowledge workers. We keep hearing that the gains will “diffuse”, and we keep not seeing much in the way of benefits, and knowledge work remains a “weak link” O-ring/pipeline mode where LLMs fail to improve the bottlenecks (while coming with their own drawbacks, like the externalized costs of forcing everyone to waste ever more time with CAPTCHAs and paywalls). Automation should be powerful; an internal combustion engine can help someone move 100× the distance or load that they could before, but who would say that writers are 100× more productive given any LLM workflow (unless we are talking about the lowest kind of spam or pseudo-writing, that makes the world a worse place)? Writers can choose between either trivial uses like ChatGPT as glorified grammar checker or relatively unimportant optional add-ons like custom software widgets, or the large speedup by replacing their writing entirely with uncreative “AI slop” outputs. The former means no meaningful gains from the AI revolution. The latter may be financially profitable, but is throwing the baby out with the bathwater, because it raises the question of why the writer need be involved at all and destroys most of the non-financial point of writing; great writers do not write for money, but to express themselves and create and to achieve things. I, for example, have long struggled to get much use out of chatbot LLMs, because they are—surprisingly, given their pretraining and my extensive corpus—bad at imitating me, and their thoughts and insights invariably shallow and worth little. They do not draw on my relevant writings, or my corpus of notes and references. Even when a possible essay is self-contained, the output is written in a grating chatbot style I can scarcely bear to read, and could not publish under my name without betraying my readers. What would it take for LLMs to make me 100× more productive? Without this, I am doomed to irrelevance.