GoKawiilWhen Anthropic's Fable 5 was pulled from international use for nearly three weeks, some over-reliant businesses were left scrambling.But Liberty Mutual easily pivoted to other platforms. That’s because 18 months earlier, they built their "AI backbone" exactly for this kind of scenario.In this rapidly moving AI landscape, the 114-year-old property and casualty insurance company recognized independence as an operating advantage.“Things are changing so fast, you need a backbone that's flexible,” Brian Craig, Liberty Mutual’s senior director of architecture, said at a recent VB Impact event. “You can't lock in right now on one vendor or even one framework.”Enterprises need flexibility to hook into different models and vendors, depending not so much on the "flavor of the day," but “what can you feel confident about for the next six months,” he said. Runtime versus control plane The company’s "backbone" (or control plane) is its own, while everything underneath remains swappable. The architecture consists of roughly 50 components across security, identity, orchestration, tool restriction, and the policies that govern how agents behave. Each is designed to be independently and immediately replaceable to support interoperability. The agent runtime below this backbone is AWS's Amazon Bedrock AgentCore; this is not the strategic center, but explicitly “just for running the agents,” Craig said. He chose this offering because it (at least currently) supports multiple frameworks and Liberty Mutual’s model-agnostic philosophy. “We still have flexibility based on what we write,” Craig said. “But if something comes along and is better, we will move to it quite quickly.” The software factoryThis architecture delivers, as proven by Liberty’s “software factory,” an agentic pipeline that automates much of the software delivery process.They started with a business process with clear pain: Onboarding electronic content management documents for insurance products. This repetitive, manual task typically required engineers to code every change. Instead, the team built a factory of coordinated agents working “in tandem and in sequence”: An Epic agent consumes high-level requirements.A Story agent breaks work dictated by the Epic agent into narrow slices within specific application areas. This agent is “constraining the context, because the smaller the context, the better the output.” A planning agent defines the technical execution plan.A coding/testing agent handles coding, testing, and basic review.A triage (critic) agent sits across all other agents, reviewing quality and feeding back improvements.Finally, a librarian agent helps others find the “context of the knowledge for their job.” Craig and his team learned quickly that a single “do everything” agent was a mistake. “You were asking it to do too many things, which meant you had to give it too much information,” Craig said. Splitting into six agents let them dramatically shrink context windows and tighten scope.Once the factory hit production, the impact was immediate. In the initial deployment, they did “about three months of work” in roughly a week. They realized that “the current software engineering process has a massive amount of handoffs, which means there's a massive amount of wait time,” Craig said. Human-paced automationThe factory is not a fully autonomous pipeline; it runs at the speed of human overseers.Liberty’s first model was a “day shift/night shift” rhythm: Engineers set goals and rules and reviewed the previous night’s outputs during the day, then let the factory run overnight. But in practice, there was never enough work to keep the agents busy all night, and the cadence felt unnatural.They shifted to a more iterative loop. Users decide when to trigger the factory, how far it runs before pausing, and at which points they want to review outputs. “Then the factory kicked in, and it may only run for less than an hour, and then you would look at it again,” Craig said. “It was more controlled at the speed that our users felt comfortable with.”The orchestration layer lets teams choose whether to review after the Epic stage, after planning, or once coding and testing complete. “That is up to the users of the factory, but it has completely removed a lot of the wait time that we currently have within our processes,” Craig said. By contrast, early on, “every time the thing ran, they wanted to look at the output,” and that became the feedback loop that trained both humans and agents.Some of this is, as Craig put it, “just automating agile at speed” and giving iterative feedback; some from humans, some from the triage agent. “But we're seeing that start to bake in, and it starts to become the rules.” When the same feedback is coming from two different directions (agent and human), it makes sense to go into Liberty’s context repository. The agents will then rely on that context the next time they need to make that decision, "and the next time again, and you start to speed up."“It's like a flywheel once you start building these and you start to get them flowing,” Craig said. “You realize it listens to what you say.” Contracts that match the pace of changeLiberty paired that architectural posture with a contract posture, deliberately shifting from five-year enterprise deals to one-year agreements.
The logic is simple as Craig sees it: The AI market moves too fast to lock into one vendor or framework for half a decade or more. Shorter terms let them evaluate — and if necessary, swap — models and platforms at the speed the market actually changes.
Cost is part of the story. When a premium frontier model like Fable arrives, the sticker shock is real. “You see the price and go, ‘Goodness, it better be really good,’ Craig said. (It was; they got to use it enough to "fall in love with it.”)
The backbone’s interoperability lets his team compare models at different price points and route workloads based on price–performance rather than vendor inertia.
The same attitude governs how they plan to use agents from major SaaS platforms. Liberty is a customer of Salesforce and Splunk, who will both “bring their agents to the table.” His team has no interest in replicating engineering work, but they do insist on observability.
“We just want to harness it as part of our system,” rather than control it, Craig said. “But we want the observability to understand what their agent is doing with our data, with our users.” Closing the “control gap”Importantly, Liberty built observability into its backbone.
As Craig explained, it’s not just logging what an agent does, but what it accesses, which identity it uses, and which tools it’s allowed to invoke. Identity and access run in part on Microsoft Entra ID, and agents are given only the tools and permissions explicitly assigned to them.
Whenever an agent realizes, ‘I don't have the information,’ it asks for it, and it only gets what it needs, rather than giving authority to “use every tool in the box.”
“Because too much information given to an agent is worse than no information,” Craig said. “You just overload it, and it gets confused.”
For detection, Liberty runs evaluations with MLflow against “golden datasets.” Whenever prompts or models change, they regression-test and immediately see whether results improved or degraded.
... continue reading