GoKawiilPeople are trusting their agents to do more work without supervision, but how do you decide when to trust them?
Some think the answer depends on how good the model is – i.e. when models get better, you can trust them to do more. But trusting your agents just because the models got smarter is like skipping your seatbelt because you got a nicer car.
The real answer has nothing to do with the model, and everything to do with the task. You need to build a mental model for when to delegate and how much, which you can then adapt.
Here’s a simple guide to maximizing agent autonomy, so you can ship fast without breaking things.
The two factors that set the ceiling
You need to ask yourself two questions first:
1. Is it easy to check the agent’s work?
To run safely on their own, agents need immediate feedback when they’re wrong. This is possible for most code with deterministic checks, like unit and integration tests. But subjective tasks, like renaming a parameter for clarity, are harder without human taste and judgment.
2. Is it cheap to undo an agent’s mistake?
Just like in traditional software engineering, if you want to trust an agent run without supervision, you need a guaranteed Ctrl+Z for worst case scenarios. That’s why StampHog, our PR approval agent, routes anything that contains deny-list keywords to a human.
... continue reading