Nobody likes lag, especially not developers. When you type a character in a terminal or editor, you expect it to appear instantly. At Compyle, we spin up ephemeral cloud dev environments where an agent and a user share an IDE + terminal. So how can we make a remote sandbox feel local?
TL;DR: If you want low latency sandboxes, cut out the middlemen and put your servers next to your users.
The Naive Approach
Here is what our initial architecture looked like:
The flow would be something like:
User starts task We provision a new sandbox in our primary region We communicate with the agent through a socket server that handles authorization, routing requests to the correct sandbox, and persisting any messages the agent sends (we don’t want to give the sandbox any credentials).
There are three main concerns with running a coding agent sandbox.
Startup time
Latency
Security
... continue reading