The Apple Photos app sometimes corrupts images when importing from my camera. I just wanted to make a blog post about it in case anyone else runs into the problem. I’ve seen other references to this online, but most of the people gave up trying to fix it, and none of them went as far as I did to debug the issue.
I’ll try to describe the problem, and the things I’ve tried to do to fix it. But also note that I’ve (sort of) given up on the Photos app too. Since I can’t trust it to import photos from my camera, I switched to a different workflow.
Here is a screenshot of a corrupted image in the Photos app:
How I used to import images
I’ve got an OM System OM-1 camera. I used to shoot in RAW + jpg, then when I would import to Photos app, I would check the “delete photos after import” checkbox in order to empty the SD card. Turns out “delete after import” was a huge mistake.
Getting corrupted images
I’m pretty sure I’d been getting corrupted images for a while, but it would only be 1 or 2 images out of thousands, so I thought nothing of it (it was probably my fault anyway, right?)
But the problem really got me upset when last year I went to a family member’s wedding and took tons of photos. Apple Photos combines RAW + jpg photos so you don’t have a bunch of duplicates, and when you view the images in the photos app, it just shows you the jpg version by default. After I imported all of the wedding photos I noticed some of them were corrupted. Upon closer inspection, I found that it sometimes had corrupted the jpg, sometimes corrupted the RAW file, and sometimes both. Since I had been checking the “delete after import” box, I didn’t know if the images on the SD card were corrupted before importing or not. After all, the files had been deleted so there was no way to check.
I estimate I completely lost about 30% of the images I took that day.
Losing so many photos really rattled me, but I wanted to figure out the problem so I didn’t lose images in the future.
Narrowing down the problem
I was worried this was somehow a hardware problem. Copying files seems so basic, I didn’t think there was any way a massively deployed app like Photos could fuck it up (especially since its main job is managing photo files). So, to narrow down the issue I changed out all of the hardware. Here are all the things I did:
Switched USB-C cables
Bought a new SD card direct from the manufacturer (to eliminate the possibility of buying a bootleg SD card)
Switched to only shooting in RAW (if importing messes up 30% of my images, but I cut the number of images I import by half, then that should be fewer corrupted images right? lol)
Bought a new laptop
Bought a new camera: the OM System OM-1 MKii
I did each of these steps over time, as to only change one variable at a time, and still the image corruption persisted. I didn’t really want to buy a new camera, the MKii is not really a big improvement over the OM-1, but we had a family trip coming up and the idea that pressing the shutter button on the camera might not actually record the image didn’t sit well with me.
Finally a smoking gun
Since I had replaced literally all of the hardware involved, I knew it must be a software problem. I stopped checking the “delete after import” button, and started reviewing all of the photos after import. After verifying none of them were corrupt, then I would format the SD card. I did this for months without finding any corrupt files. At this point I figured it was somehow a race condition or something when copying the photo files and deleting them at the same time.
However, after I got home from RailsConf and imported my photos, I found one corrupt image (the one above). I was able to verify that the image was not corrupt on the SD card, so the camera was working fine (meaning I probably didn’t need to buy a new camera body at all).
I tried deleting the corrupt file and re-importing the original to see if it was something about that particular image, but it re-imported just fine. In other words, it seems like the Photos app will corrupt files randomly.
I don’t know if this is a problem that is specific to OM System cameras, and I’m not particularly interested in investing in a new camera system just to find out.
If I compare the corrupted image with the non-corrupted image, the file sizes are exactly the same, but the bytes are different:
Checksums:
aaron@tc ~/Downloads> md5sum P7110136-from-camera.ORF Exports/P7110136.ORF 17ce895fd809a43bad1fe8832c811848 P7110136-from-camera.ORF 828a33005f6b71aea16d9c2f2991a997 Exports/P7110136.ORF
File sizes:
aaron@tc ~/Downloads> ls -al P7110136-from-camera.ORF Exports/P7110136.ORF -rw-------@ 1 aaron staff 18673943 Jul 12 04:38 Exports/P7110136.ORF -rwx------ 1 aaron staff 18673943 Jul 17 09:29 P7110136-from-camera.ORF*
The P7110136-from-camera.ORF is the non-corrupted file, and Exports/P7110136.ORF is the corrupted file from Photos app. Here’s a screenshot of the preview of the non-corrupted photo:
My new workflow
I’m not going to put any more effort into debugging this problem, but I wanted to blog about it in case anyone else is seeing the issue. I take a lot of photos, and to be frank, most of them are not very good. I don’t want to look through a bunch of bad photos every time I look at my library, so culling photos is important. Culling photos in the Photos app is way too cumbersome, so I’ve switched to using Darktable.
My current process is:
Import images to Darktable
Delete the ones I don’t like
Process ones I do like
Export both the jpg and the original raw file
Import those to the Photos app so they’re easy to view and share
Periodically format my SD card
I’ve not seen any file corruption when importing to Darktable, so I am convinced this is a problem with the Photos app. But now, since all of my images land in Darktable before making their way to the Photos app, I don’t really care anymore. The bad news is that I’ve spent a lot of time and money trying to debug this. I guess the good news is that now I have redundant hardware!