How do you balance risk management and safety with innovation in agentic systems — and how do you grapple with core considerations around data and model selection? In this VB Transform session, Milind Naphade, SVP, technology, of AI Foundations at Capital One, offered best practices and lessons learned from real-world experiments and applications for deploying and scaling an agentic workflow.
Capital One, committed to staying at the forefront of emerging technologies, recently launched a production-grade, state-of-the-art multi-agent AI system to enhance the car-buying experience. In this system, multiple AI agents work together to not only provide information to the car buyer, but to take specific actions based on the customer’s preferences and needs. For example, one agent communicates with the customer. Another creates an action plan based on business rules and the tools it is allowed to use. A third agent evaluates the accuracy of the first two, and a fourth agent explains and validates the action plan with the user. With over 100 million customers using a wide range of other potential Capital One use case applications, the agentic system is built for scale and complexity.
“When we think of improving the customer experience, delighting the customer, we think of, what are the ways in which that can happen?” Naphade said. “Whether you’re opening an account or you want to know your balance or you’re trying to make a reservation to test a vehicle, there are a bunch of things that customers want to do. At the heart of this, very simply, how do you understand what the customer wants? How do you understand the fulfillment mechanisms at your disposal? How do you bring all the rigors of a regulated entity like Capital One, all the policies, all the business rules, all the constraints, regulatory and otherwise?”
Agentic AI was clearly the next step, he said, for internal as well as customer-facing use cases.
Designing an agentic workflow
Financial institutions have particularly stringent requirements when designing any workflow that supports customer journeys. And Capital One’s applications include a number of complex processes as customers raise issues and queries leveraging conversational tools. These two factors made the design process especially complex, requiring a holistic view of the entire journey — including how both customers and human agents respond, react, and reason at every step.
“When we looked at how humans do reasoning, we were struck by a few salient facts,” Naphade said. “We saw that if we designed it using multiple logical agents, we would be able to mimic human reasoning quite well. But then you ask yourself, what exactly do the different agents do? Why do you have four? Why not three? Why not 20?”
They studied customer experiences in the historic data: where those conversations go right, where they go wrong, how long they should take and other salient facts. They learned that it often takes multiple turns of conversation with an agent to understand what the customer wants, and any agentic workflow needs to plan for that, but also be completely grounded in an organization’s systems, available tools, APIs, and organizational policy guardrails.
“The main breakthrough for us was realizing that this had to be dynamic and iterative,” Naphade said. “If you look at how a lot of people are using LLMs, they’re slapping the LLMs as a front end to the same mechanism that used to exist. They’re just using LLMs for classification of intent. But we realized from the beginning that that was not scalable.”
Taking cues from existing workflows
... continue reading