Systems optimization should be part of CI/CD

This post is part of our AI-Driven Research for Systems (ADRS) case study series, where we use AI to automatically discover better algorithms for real-world systems problems. Algorithmic discovery frameworks like OpenEvolve and GEPA have shown that AI-Driven Research for Systems (ADRS) can produce strong algorithms. But today's frameworks are too expensive for where ADRS should go next. The future is in continuous and bespoke optimization, not one-off benchmark results. The system should generate solutions tailored to the deployment's exact workload, hardware, and SLOs; adapting as these change. This is not possible when every optimization costs a fortune. LEVI is a framework built around lowering the cost of algorithmic discovery. Instead of using the strongest, most expensive models for every step, it invests in the search harness: smaller, cheaper models (e.g. QWEN 30B) do most mutations, while larger models are reserved for rarer paradigm shifts. This is made possible because LEVI maintains diversity across both code structure (e.g. number of loops) and actual behavior (e.g. performance on subset x), ensuring the search archive does not collapse into a single solution family. The result is a framework that gets stronger ADRS results at a fraction of the cost: roughly 3–7× cheaper than baselines in the main benchmark comparison. ✍️ Previous ADRS Blogs: https://ucbskyadrs.github.io/

https://ucbskyadrs.github.io/ 👩‍💻 Code: github.com/ttanv/levi

github.com/ttanv/levi 💬 Join us : join.slack.com/t/adrs-global and Discord

: join.slack.com/t/adrs-global and Discord 🌎 Follow us: x.com/ai4research_ucb

LLM algorithmic discovery frameworks have shown promise in delivering strong results for ADRS. However, a key bottleneck remains: cost. This blog argues why cost plays an important role in ADRS, and then introduces LEVI; an algorithmic discovery framework that outperforms others at a fraction of the cost.

Existing frameworks require using expensive and large closed-source LLMs. This is problematic for obvious reasons. For example, it raises the barrier of entry since most researchers can't afford such experiments. But the more important issue is broader: ADRS should not be viewed as something we run once to produce a single strong result.

Let the Barbarians Spread?

Decreasing the cost until it is orders of magnitude less should be the natural next step for ADRS. This is because the results from ADRS should not be treated as one-off research outcomes similar to how usual systems papers look like. Where for a given problem, researchers improve upon algorithms and heuristics to yield better results. Where industry then follows along by porting these algorithms and adapting them to their setup.

Instead we should move towards completely bespoke solutions. Where each solution is tailored to the exact setup and environment that each deployment has, squeezing out the most juice.

Figure 1: One expensive ADRS run for everyone (top) vs. cheap, bespoke optimization per deployment (bottom).

... continue reading