Show HN: Factorio Learning Environment – Agents Build Factories

Large Language Models (LLMs) are rapidly saturating existing benchmarks, necessitating new open-ended evaluations. We introduce the Factorio Learning Environment (FLE), based on the game of Factorio, that tests agents in long-term planning, program synthesis, and resource optimization. FLE provides open-ended and exponentially scaling challenges - from basic automation to complex factories processing millions of resource units per second. We provide two settings: Lab-play consisting of 24 structured tasks with fixed resources. Open-play with the unbounded task of building the largest factory from scratch on a procedurally generated map. We demonstrate across both settings that models still lack strong spatial reasoning. In lab-play, we find that LLMs exhibit promising short-horizon skills, yet are unable to operate effectively in constrained environments, reflecting limitations in error analysis. In open-play, while LLMs discover automation strategies that improve growth (e.g electr ... Read full article.