GoKawiilAt a time when there's keen public interest in AI that can make itself better, researchers here have tackled one of the central challenges affecting its development.
The research team, which includes collaborators from NVIDIA and Flower Labs, have come up with a new method for recursive self-improving AI agents to continue improving themselves (by repeatedly testing and enhancing their own code) without hitting the evaluation ceiling that they frequently encounter.
Their method also suggests a way of cutting the costs of the computational resource needed for the development of such AI agents.
While agents can already improve themselves by editing their own code, testing variants, and keeping changes that perform better, this process is usually limited by a fixed evaluator, benchmark, or test suite. Once the agent has learned everything that fixed signal can distinguish, improvement slows or stalls.
"A self-improving agent can only get as good as the test that scores it," explains team member Alex Iacob, a PhD student in the Machine Learning Systems Lab under the supervision of Prof Nic Lane. "The test does not merely measure progress, it defines it, so the efficacy of the test becomes a ceiling the agent cannot climb past."
Now the researchers have addressed this issue by having both the self-improving agent and the evaluator evolve together. "Instead of improving an agent against a fixed test, we let the evaluation evolve alongside the agent," Alex adds. "As the agent gets better, the evaluation also gets harder, and the bar keeps rising."
The Red Queen Gödel Machine
Alex is the first author on the pre-print paper the research team has just uploaded to arXiv. The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators shares the technical details of their work, along with some impressive results from using this framework across a number of tasks.
Figure: Agents and evaluators improving together. The Red Queen Gödel Machine searches through many possible versions of an AI agent, while also improving the evaluator that judges those agents. During each phase, the evaluator is kept fixed so progress can be measured reliably. At checkpoints, a stronger evaluator can replace the old one if it performs better on trusted ground-truth examples. Scores from the old evaluator are then removed, so the next phase is guided by the new, more demanding standard. This creates a self-improving loop in which agents and evaluators improve together, while the system remains anchored to reliable checks.
In both scientific paper writing and reviewing, and (Maths) Olympiad-level proof writing and grading, the Red Queen Gödel Machine improved performance over previous self-improving AI agents.
... continue reading