GoKawiilWelcome everyone. My name is David Hayward and I am excited to talk about enhancements to Core Image, and its support for RAW image files.
I will talk about four main topics. First, I will review how RAW image files from cameras, are supported by Core Image on Apple platforms. And show you significant RAW quality improvements that are coming in iOS, iPadOS, macOS, and visionOS 27.
After that, I will discuss how to get optimal performance when rendering RAWs.
And lastly, I will describe features, that Apple added to the CIImageProcessor class for RAW.
To start, here is a quick summary of how Core Image supports RAW. RAW files come from a diverse set of camera makes and models, but unlike HEIFs and JPEGs, they need special handling before they can be displayed.
The first step, is to parse the file's metadata, and unpack the RAW sensor values. At this stage, each pixel location only has a red, green, or blue value arranged in a mosaic pattern. This image is cropped way in, so that this pattern is visible.
The next step, is to demosaic the sensor data, so that each pixel contains red, green, and blue values.
The following stage is to denoise the pixels, so that the image is free of photon noise, read noise and thermal noise. After this, convolutions are applied to sharpen edges and add local contrast.
The last major step is to adjust white balance, exposure, color and tone to make a pleasing final image.
The algorithms for all these steps are built in to iOS, iPadOS, macOS, and visionOS.
... continue reading