GoKawiilAgents are moving to the edge.
Their memory should too.
I've been thinking about agent memory for quite some time. There are really two things that bother me. The first is how we represent memory in the agent space. The second is where that memory should live when the agent is running on a hardware constraints. Somehow my work converged both of the concerns into a single answer.
1. The memory needs to be deterministic when dealing with LLMs
What I've observed in my own agent use is that there are certain behaviors and patterns that LLMs can't follow and sometimes miss or forget. I would correct a fact and the agent would quote the old version a week later. I would change a preference and recall would return both versions, and the model had to guess which one I meant. I realized something. Right now we make the LLM sort this out by rereading old text, which costs tokens when it works and accuracy when it tries to rely on semantic retrieval. A schema with types and supersession rules answers the same question instantly.
When a model manages its own memory, it has to decide whether the recalled text is new or a correction, whether the instructions contradict something already stored. These are mostly schema decisions, and shouldn't be left to the model to figure out.
A typed memory store moves those decisions out of the model and into the schema. The model extracts a fact; the database determines what that fact means.
2. Serverless/cold-first database that must be accessible on a small hardware
The edge angle comes from something I keep returning to. During my time at Google Cloud Storage, I saw how much tooling and infrastructure was needed to support simple semantic storage use cases. The operational cost became obvious once we realized how the index and semantic storage would scale with the growing corpus. For example, hot vector index (along with availability guarantees via replication) in RAM or even SSDs is expensive, and isn't really needed for users with infrequent and cold queries. Every memory design I had seen assumed a search cluster or a managed API on the other side of a network link. Meanwhile embedding models were already running fine on CPU cycles and small models are getting better every few months.
I ended up building polign_db as a typed database on top of the hybrid vector + BM25 engine and living in an object store. The polign_db server holds nothing durable, so processes and machines can die and restart and the memory stays intact. Initially while working on polign_db, I wanted it to serve search from object storage with as little resident state as possible, and with query costs that do not grow with the corpus size. Those choices were about cloud bills, but they also happened to be what small devices need.
... continue reading