Inner Loop Agents

Inner Loop Agents Sat April 19, 2025 What if an LLM could use tools directly? As in, what if LLMs executed tool calls without going back to the client. That’s the idea behind inner loop agents. It’s a conceptual shift. Instead of thinking of agents as being a system involving client & server, you just have a single entity, the LLM. I hope it will help clarify how o3 and o4-mini work. (note: this post isn’t as long as it looks, there’s a lot of diagrams and examples) To illustrate, regular LLMs rely on the client to parse and execute tools, like this: graph TD subgraph inn["LLM (Inner Loop)"] Tokenizer-->nn[Neural Net]-->samp[Select Next Token]-->Tokenizer end text((Input))-->Tokenizer parse--->out((Output)) samp-->parse[Parse Tool Calls]-->exec[Run Tools]-->parse parse--"tool result"-->Tokenizer On the other hand, with inner loop agents, the LLM can parse and execute tools on it’s own, like this: graph TD subgraph inn["Inner Loop Agent"] direction TB Tokenizer nn[Neural Net] sa ... Read full article.