Single copilots are yesterday’s news. Competitive differentiation is all about launching a network of specialized agents that collaborate, self-critique, and call the right model for every step. The latest installment of VentureBeat’s AI Impact Series, presented by SAP in San Francisco, tackled the issue of deploying and governing multi-agent AI systems.
Yaad Oren, managing director SAP Labs U.S. and global head of research & innovation at SAP, and Raj Jampa, SVP and CIO with Agilent, an analytical and clinical laboratory technology firm, discussed how to deploy these systems in real-world environments while staying inside cost, latency, and compliance guardrails. SAP’s goal is to ensure that customers can scale their AI agents, but safely, Oren said.
“You can be almost fully autonomous if you like, but we make sure there are a lot of checkpoints and monitoring to help to improve and fix,” he said. “This technology needs to be monitored at scale. It’s not perfect yet. This is the tip of the iceberg around what we’re doing to make sure that agents can scale, and also minimize any vulnerabilities.”
Deploying active AI pilots across the organization
Right now, Agilent is actively integrating AI across the organization, Jampa said. The results are promising, but they’re still in the process of tackling those vulnerability and scaling issues.
“We’re in a stage where we’re seeing results,” he explained. “We’re now having to deal with problems like, how do we enhance monitoring for AI? How do we do cost optimization for AI? We’re definitely in the second stage of it, where we’re not exploring anymore. We’re looking at new challenges and how we deal with these costs and monitoring tools.”
Within Agilent, AI is deployed in three strategic pillars, Jampa said. First, on the product side, they’re exploring how to accelerate innovation by embedding AI into the instruments they develop. Second, on the customer-facing side, they’re identifying which AI capabilities will deliver the greatest value to their clients. Third, they’re applying AI to internal operations, building solutions like self-healing networks to boost efficiency and capacity.
“As we implement these use cases, one thing that we’ve focused on a lot is the governance framework,” Jampa explained. That includes setting policy-based boundaries and ensuring the guardrails for each solution remove unnecessary restrictions while still maintaining compliance and security.
The importance of this was recently underscored when one of their agents did a config update, but they didn’t have a check in place to ensure its boundaries were solid. The upgrade immediately caused issues, Jampa said — but the network was quick to detect them, because the second piece of the pillar is auditing, or ensuring that every input and every output is logged and can be traced back.
Adding a human layer is the last piece.
“The small, lowercase use cases are pretty straightforward, but when you talk about natural language, big translations, those are scenarios where we have complex models involved,” he said. “For those bigger decisions, we add the element where the agent says, I need a human to intervene and approve my next step.”
And the question of speed versus accuracy comes into play early during the decision-making process, he added, because costs can add up fast. Complex models for low-latency tasks push those costs substantially higher. A governance layer helps monitor the speed, latency and accuracy of agent results, so that they can identify opportunities to build on their existing deployments and continue to expand their AI strategy.
Solving agent integration challenges
Integration between AI agents and existing enterprise solutions remains a major pain point. While legacy on-premise systems can connect through data APIs or event-driven architecture, the best practice is to first ensure all solutions operate within a cloud framework.
“As long as you have the cloud solution, it’s easier to have all the connections, all the delivery cycles,” Oren said. “Many enterprises have on-premise installations. We’re helping, using AI and agents, to migrate them into the cloud solution.”
With SAP’s integrated tool chain, complexities like customization of legacy software are easily maintained in the cloud as well. Once everything is within the cloud infrastructure, the data layer comes in, which is equally if not more important.
At SAP, the Business Data Cloud serves as a unified data platform that brings together information from both SAP and non-SAP sources. Much like Google indexes web content, the Business Data Cloud can index business data and add semantic context.
Added Oren: “The agents then have the ability to connect and create business processes end-to-end.”
Addressing gaps in enterprise agentic activations
While many elements factor into the equation, three are critical: the data layer, the orchestration layer, and the privacy and security layer. Clean, well-structured data is, of course, crucial, and successful agentic deployments depend on a unified data layer. The orchestration layer manages agent connections, enabling powerful agentic automation across the system.
“The way you orchestrate [agents] is a science, but an art as well,” Oren says. “Otherwise, you can have not only failures, but also auditing and other challenges.”
Finally, investing in security and privacy is non-negotiable — especially when a swarm of agents is operating across your databases and enterprise architecture, where authorization and identity management are paramount. For example, an HR team member may need access to salary or personally identifiable information, but no one else should be able to view it.
We’re headed toward a future in which human enterprise teams are joined by agent and robotic team members, and that’s when identity management becomes even more vital, Oren said.
“We’re starting to look at agents more and more like they’re humans, but they need extra monitoring,” he added. “This involves onboarding and authorization. It also needs change management. Agents are starting to take on a professional personality that you need to maintain, just like an employee, just with much more monitoring and improvement. It’s not autonomous in terms of life cycle management. You have checkpoints to see what you need to change and improve.”