A team of Apple researchers set out to understand what real users expect from AI agents, and how they’d rather interact with them. Here’s what they found.
Apple explores UX trends for the era of AI agents
In the study, titled Mapping the Design Space of User Experience for Computer Use Agents, a team of four Apple researchers says that while the market has been investing heavily in the development and evaluation of AI agents, some aspects of the user experience have been overlooked: how users might want to interact with them, and what these interfaces should look like.
To explore that, they divided the study into two phases: first, they identified the main UX patterns and design considerations that AI labs have been building into existing AI agents. Then, they tested and refined those ideas through hands-on user studies with an interesting method called Wizard of Oz.
By observing how those design patterns hold up in real-world user interactions, they were able to identify which current AI agent designs align with user expectations, and which fall short.
Phase 1: The taxonomy
The researchers looked into nine desktop, mobile, and web-based agents, including;
Claude Computer Use Tool
Adept
OpenAI Operator
... continue reading