GoKawiilis a technology reporter who has been covering the industry since 2007. He co-hosts the Waveform podcast with MKBHD. A former engineer, he now specializes in topics like computational photography and social media protocols.
If you look at a photo taken by a smartphone today, it’s hard to find much that’s technically wrong. Faces are bright and visible. The skies are pristine and blue. Even at night, scenes are well lit and noise-free. If the physical limitations of a tiny camera are what you’re trying to solve for, you could probably label smartphone photography a mostly solved problem.
But in our current moment, the photos people are drawn to are not flawless — and that’s created a real problem for the people making smartphone cameras. “The gap between what two random people want from their camera is growing dramatically,” says Isaac Reynolds, who leads the Pixel camera team at Google. Some people want a perfectly optimized photo, Reynolds says. “But there’s a growing number of people who want something that they feel is more authentic or traditional, by their definition.”
That’s what Reynolds and the Pixel team are setting out to solve with Camera Looks, a total rethink of how the camera captures and styles an image on the Pixel 11 series. The phones include a new suite of processing tools aimed at giving people much more control over the camera’s output. At first glance, they look a lot like filters, with names like Black Tie, Classic, and most strikingly, Digi, in reference to the growing trend of buying cheap digital cameras from the early aughts. But unlike a filter, which generally applies editing tweaks after the processing happens, Camera Looks is changing how image data is processed at the sensor level.
Editorial Camera Look. Minimal Camera Look.
“The pipeline spans the entire system,” Reynolds says. “It’s inside HDR Plus. It’s inside the processing engine. It’s inside the camera app as well in how we make certain decisions about auto exposure and integration time and number of frames packaged and captured.”
In the last decade or so, smartphone cameras have been engineered to do battle with the limitations of physics using what’s known as “computational photography.” Too dark? Night mode uses multi-frame image stacking to brighten the photo without adding too much noise. Need to zoom in but don’t have a telephoto lens? An on-device model can artificially create data it expects to be in your photo. Too small of a sensor for natural blur? Phones can use subject segmentation to recognize objects, cut out the subject, and emulate depth of field.
The result has been crisp, clear photos — and also a lot of (mostly younger) consumers carrying around old iPhones and cheap point-and-shoots from the early aughts to get a simpler, less processed, and (to their eye) better look.
“If you are looking for a minimal look … we simply don’t do the processing at all.”
The Pixel 11’s new Camera Looks feature is Google’s response to this growing trend. And to solve it, the Pixel team is allowing users to dial back how much processing happens when they snap a shot. Some of the Pixel’s Camera Looks will merge fewer frames together, leaving the highlights a bit blown out instead of compressing them to a midtone. Some of them won’t apply as much sharpening and will instead leave the image softer on the edges. You could argue that these new photos are technically “worse” — with reduced detail and dynamic range — but the result should be a photo closer to what any given user actually wants. This is a completely different way of thinking about computational photography, using tools to optimize for style instead of for technical perfection.
... continue reading