GoKawiilThe word “recursion” is the latest buzzword in AI circles. Two separate startups have taken on the name, and many more have started referencing Recursive self-improvement (RSI) in their roadmaps. Like AGI before it, RSI has become a three-letter byword for a cataclysmic AI takeoff – even if there’s still a little disagreement about exactly what it means.
In basic terms, RSI refers to an AI system that can continuously upgrade itself. Once AI systems can manage the upgrade cycle better than humans, the process can become a closed loop, limited only by the compute power they can access, and humans no longer necessary or even helpful.
Scary or not, that’s a vision that a lot of AI labs are eager to chase.
Earlier this month, well-known AI researcher, Richard Socher, launched the aptly named Recursive Superintelligence launched with RSI as an explicit goal. “Our main focus is to build truly recursive, self-improving superintelligence at scale,” Socher told TechCrunch at launch, “which means that the entire process of ideation, implementation, and validation of research ideas would be automatic.”
A number of other prominent researchers are already chasing that same goal, hoping for a breakthrough that will make recursive self-improvement possible.
One of the most prominent is Alex Karpathy, a legendary figure from Tesla and OpenAI, who is using agent swarms to train LLMs on simple tasks for a project he calls Auto-Research. Karpathy has been unusually open about the project, tweeting about milestones regularly and making the building blocks available through a public GitHub repo. So far, the work has mostly been confined to making minor improvements on a GPT-2 scale model – as Karpathy noted in March, “It’s not novel, ground-breaking ‘research’ (yet)” – but it’s been enough to convince lots of other researchers to follow the RSI dream. And with Karpathy now working on pre-training at Anthropic, he will have plenty of opportunity to apply the idea at a larger scale.
Adaption – founded by Cohere and Google alum Sara Hooker – recently launched a similar tool called AutoScientist in an effort to automate frontier training. Like Karpathy’s auto-researchers, the system trains agents to make incremental improvements – but for Adaption, the goal is to make it easier to train a full-scale frontier model. If those same researchers start to push the frontier forward, the system could quickly spiral into something very much like RSI.
Disarray founder Doris Xin drew more specific RSI interest when her self-trained machine learning agent took home 28 medals in a recent Kaggle competition, beating out many human-trained agents. As she sees it, the major challenge is reliability.
“I would argue, given infinite compute and infinite time horizon, we are already there,” Xin told me. “I want to make an argument that this is not a creative endeavor, really. It’s just a lot of meat-and-potatoes engineering.”
Not there yet
... continue reading