is a reviewer covering laptops and the occasional gadget. He spent over 15 years in the photography industry before joining The Verge as a deals writer in 2021.
Microsoft is at it again with another round of ads showing people talking to Copilot AI on their computers. This time it’s holiday-themed, including a cameo from the big man in red. The 30-second TV spot asks if you’re “ready for the holidays” and features actors in various festive home settings asking Copilot for some ho-ho-ho-help with everything from holiday lighting and cooking to oversized outdoor decorations.
Just like the last time I tested Copilot Vision and Voice Mode, I made a list of all the prompts in Microsoft’s ad and I’m checking them twice. Or as many times as I can stomach before hitting the nog.
In the new ad, Copilot offers to help a homeowner make their smart home more festive when they prompt it, “Show me how to sync my holiday lights to my music.” The user clicks through their cloud-connected smart lighting controls on a website called Relecloud as Copilot says, “Let’s walk through it together.” The ad jump-cuts to the home lights pulsing to that classic Christmas song “A-Punk” by Vampire Weekend.
Relecloud is not a well-known smart home company like Philips Hue or Govee. In fact, it’s not a real business at all. It’s one of the fictional companies Microsoft uses in published case studies (see also: Contoso). I’m inclined to think its use here points to all these advertised Copilot actions being simulated, but a Microsoft rep insists that’s not the case. Nicci Trovinger, general manager of Windows marketing, tells The Verge, “All Copilot responses are actual responses Copilot gave to the scenarios shown and questions asked at a point in time. Responses were shortened for brevity to fit the length of the creative spot, in line with standard advertising practices.”
I tried this test with Copilot in two ways: one using a still image of the lighting interface from the ad, and another with the Philips Hue Sync app. Copilot made its best guess as to where I should click in the image from the ad, highlighting a “Sync Mode” drop-down menu with its own onscreen cursor, but it struggled to take me much further. It often told me it highlighted something when it didn’t, and it hallucinated a green “Apply” button that was actually just the color preset for green lighting.
Copilot’s answers were about as confusing when I presented a fully configured app for my Philips Hue lights. It identified the Hue Sync app, and at first told me, correctly, to click on the Music tab and the “Start light sync” button. But then it hallucinated buttons that weren’t there, pointed me to the Entertainment Zones I had already set up, and kept telling me it had highlighted things on my screen when it hadn’t. Copilot’s cursor highlight feature is useful, but it usually only does it when you ask — and it’s painfully slow to react. It often lingers on your screen well after Copilot’s moved on to other advice.
A couple other prompts go unanswered in the ad, including “Help me figure out these instructions” and “Convert this recipe on my screen so it feeds 12.”
“14!” another actor, presumably their partner, interjects.
We don’t see what’s on screen in these cases, so for the instructions, I took a shot at showing Copilot Ikea’s 2x4 Kallax shelf — a classic with assembly manuals available online. Copilot kept identifying dowels as screws or nails. And it would often detect page numbers as step numbers, making any attempts to follow along even more chaotic.
... continue reading