How I program with Agents
2025-06-08
This is the second part of my ongoing self-education in how to adapt my programming experience to a world with computers that talk. The first part, How I program with LLMs, covered ways LLMs can be adapted into our existing tools (basically, autocomplete) and how careful prompting can replace traditional web search. Now I want to talk about the harder, and more rewarding act of using agents to program.
Define Agent
It is worthwhile starting with a definition of the word “agent” in the context of LLMs. The “AI” hype cycle has been throwing this very generic word around longer than agents have actually been useful constructs. As a result there is a bit of smoke and mirrors marketing and general mysticism to dig through to find any value in the word. For someone with an engineering background there is now a straightforward definition: an agent is 9 lines of code. That is, an agent is a for loop which contains an LLM call. The LLM can execute commands and see their output without a human in the loop.
User LLM prompt bash, patch, etc tool call tool result Response & End of turn
That’s it. Like all simple things your instinct, quite reasonably, could be to say “so what?” It is a for loop. But the results are an astonishing improvement on the power of a raw large language model.
Whiteboard programming
Consider yourself in front of a whiteboard, writing a function in C with a marker designed to test if a UTF-8 string is valid. (This has actually happened to me, it is a standard interview technique. The stakes were high, this interview changed the course of my career!) How well you do at this task depends on your experience as a programmer, and your ability to fudge your inability to use external resources. You need to remember the encoding of UTF-8. You need to not confuse your C grammar with all the other C-like programming languages you have used over your career (is it name-then-type or type-then-name?). In daily life you get feedback from a compiler if you make a mistake, you can look up a specification of UTF-8, and best of all you can write your program and sprinkle some printfs in it to see what you got wrong.
Asking an agentless LLM to write code is equivalent to asking you to write code on a whiteboard. It is an exercise in dredging up half-forgotten memories, executing a parser on an extraordinarily inefficient substrate, attempting to avoid hallucinating an actually-useful programming interface. The fact LLMs can invent programs out of whole cloth is an astonishing technical achievement that I consider myself blessed to have lived to see, but it is also not a huge shock that attaching a GPU to a virtual whiteboard is not going to do a significant amount of useful programming.
... continue reading