Microsoft debuts its next big high-stakes AI feature in Windows - can you trust it?

Yuliya Taba/iStock/Getty Images Plus Follow ZDNET: Add us as a preferred source on Google. ZDNET's key takeaways Windows 11 is adding agents that can take actions on your behalf. They might create potential security and privacy risks. Expect testing and more security controls before the feature goes live. Every computer security decision ultimately comes down to a question of trust. Should you install this program you're about to download from an unfamiliar website? Are you certain that your email messages are going directly to their recipient without being intercepted? Is it safe to provide that merchant with your credit card details? Soon, owners of PCs running Windows 11 will have another question to add to that list: Should you trust this Copilot agent to poke around in your files and interact with apps on your behalf? Also: Microsoft Copilot AI can now pull information directly from Outlook, Gmail, and other apps Here's how Microsoft describes the Copilot Actions feature, which is now in testing by members of the Windows Insider Program: Copilot Actions is an AI agent that completes tasks for you by interacting with your apps and files, using vision and advanced reasoning to click, type, and scroll like a human would. This transforms agents from passive assistants into active digital collaborators that can carry out complex tasks for you to enhance efficiency and productivity -- like updating documents, organizing files, booking tickets, or sending emails. After you've granted the agent access, when integrated with Windows, the agent can take advantage of what you already have on your PC, like your apps and data, to complete tasks for you. These are pretty big trust decisions. Allowing an agent to interact with your personal files requires a leap of faith. So does the idea of letting an agent act on your behalf in apps -- where, presumably, you are signed in using some sort of secure credentials. Learning from the past The last time Microsoft rolled out a major AI feature with this level of access to your personal data, it ... didn't go well. The Windows Recall feature was slammed by security researchers, delayed for months, and finally relaunched with major privacy and security changes. Ultimately, it was nearly a year before the feature made it to public builds. This time around, Microsoft is taking no such chances. In a pair of on-the-record briefings ahead of the public debut of the Copilot Actions feature, executives at the company went to great pains to emphasize its commitment to privacy and security controls. Also: How to get free Windows 10 security updates through October 2026 For starters, the feature is rolling out as a preview, in "experimental mode," exclusively for customers who've opted into the Windows Insider Program for pre-release builds of Windows. The feature will be disabled by default and will only be enabled when the user flips the "Experimental agentic features" switch in Windows Settings > System > AI components > Agent tools. Agents that integrate with Windows must be digitally signed by a trusted source, much as executable apps are. That precaution should make it possible to revoke and block malicious agents. Agents will run under a separate standard account that is only provisioned when the user enables the feature. For now, at least, the agent account will have access to a limited set of so-called known folders in the logged-on user's profile -- including Documents, Downloads, Desktop, and Pictures. The user needs to explicitly grant permission to access files in other locations. All of those actions will happen in a contained environment called the Agent workspace, with its own desktop and only limited access to the user's desktop. In principle, this kind of runtime isolation and granular control over permissions is similar to existing features like the Windows Sandbox. In a blog post highlighting these security features, Dana Huang, corporate vice president, Windows Security, said, "[A]n agent will start with limited permissions and will only obtain access to resources you explicitly provide permission to, like your local files. There is a well-defined boundary for the agent's actions, and it has no ability to make changes to your device without your intervention. This access can be revoked at any time." Also: This new Copilot trick will save you tons of time in Windows 11 - here's how The security stakes for this kind of feature are high. As Huang noted, "[A]gentic AI applications introduce novel security risks, such as cross-prompt injection (XPIA), where malicious content embedded in UI elements or documents can override agent instructions, leading to unintended actions like data exfiltration or malware installation." And, of course, there's always the risk that an AI-powered agent will confidently perform the wrong action. In an interview, Microsoft's Peter Waxman confirmed that the company's security researchers are actively "red-teaming" the Copilot Actions feature, although he declined to discuss any specific scenarios that they've tested. Microsoft said the feature will be evolving continuously during the experimental preview period, with "more granular security and privacy controls" arriving before the features are released to the public. Will those caveats and disclaimers be enough to satisfy the notoriously skeptical community of security researchers? We're about to find out. Want to follow my work? Add ZDNET as a trusted source on Google.