Apple’s new AI photo editing tools often work, for better and for worse

The most popular camera in the world just got its first AI photo editing tools, and I don’t think any of us are ready.

As far as AI graphics improvements go, iOS 27’s new features are pretty impressive compared to what you can do with, say, Google’s Pixel phones. But for the iPhone, they represent the entry points that the native photos app allows to do your photos. I mean it to remember. I mean, I don’t know anymore.

These new features are part of the iOS 27 beta right now, so be aware that Apple may continue to make tweaks to them before they are released to the general public. There are three, or maybe two and a half, new AI updates in this update. The new Clean tool counts as half, because it was already there but it was too bad that it didn’t count. It’s the tool that lets you get photobombers in the background of your photos, and it’s changed a lot this year. There’s also Extend, which allows you to extend the edges of your photo using AI to paint the surface. And there’s Spatial Reframing, which simulates moving the camera around a scene to allow you to reframe an existing image. He is the most demanding and probably the most difficult of the three.

But first things first: Clean up. It’s fine now. Instead of using hardware models to extract content and fill in information, it can use more powerful models in the cloud. This is something Google has been doing for years now, and that’s why the company’s Magic Editor tool was so much better than the version Apple introduced last year. The clean up on all the equipment was not good at painting the details instead of what it removed. It left a strange legacy and was often more complicated than it needed to be. Clean 2.0? It does the job.

Using AI to remove objects from images is the editing tool I’m not too worried about. I will use it to clear my child’s nose or to clear a stranger’s back. This new version in iOS does all this without a problem, and I think it will be popular with iPhone owners.

Stepping up to the next level of complexity and content-picture, there’s Expansion. Think of it like planting, but in a different way. It helps you expand the edges of your frame, which you might want to do if your text is too tight on your subject and you want to give it some breathing room, for example. Expansion allows you to do this, but only to a limited extent. It seems to avoid human editing, and sometimes it tells you that an image can be added to a certain part of it. It just adds a bit more padding, which limits the kind of shenanigans that can be used. I appreciate that. Like Cleansing, it does its job for sure. They seem to prefer looking for symmetry, which often works. It added the rally car part that was out of my original photo, and added the side mirror to match the one in the photo.

You don’t seem as eager to create things to put in your photos as, say, Samsung’s first attempt. But I caught it adding a potted plant to the side table; it seems convincing, but Me Note that it is not a real plant. I would have been awesome if I had posted this photo on Instagram.

Expand your work with your image in two places; Spatial Reframing adds a third dimension. It builds on existing features that make your photos look 3D-ish, allowing you to change the image as if you’ve moved the camera and changed your perspective. You can’t go too away from it – unless you moved your arm while taking the picture. But the idea is that you can fix your design if you didn’t nail it when you shot it.

See, this makes my type A happy. Sometimes I like everything about a photo I’ve taken, except wouldn’t it be nice if I could just move to the left to avoid creating something that messes with my head? You can’t always get it in real time. These are the small changes that Spatial Reframing was made for.

It seems reasonable, except that there is an opportunity to cause confusion in the present, even a small change. I tried to edit the image of a photo I took during a technical talk with Apple executives following the WWDC keynote. Since I was sitting on the sidelines, one of the actors was very hidden in the first shot. I changed the shape and the AI ​​model created a character sitting next to Craig Federighi.

Predicting what is to come in a two-dimensional environment seems to be an easier problem than a three-dimensional one. The results from Spatial Reframing compared to Extend show this. They are amazing. The further away you are from the subject, the less latitude you have to “rewrite,” and the more AI-generated content becomes real. But you end up with a picture that is subtly different from the actual picture you took, and then, what are we doing?

In pictures with the subject close to the camera, things get weird. The results of the “reproduction” are very surprising, and the AI ​​has to work hard to fill the gaps. You can change the shape in a selfie, but that means the AI ​​has a lot to fill in on your face, which quickly finds that weird little valley. Even within its minimal changes, it can make the face look crooked and “off”. It’s easy to create things that didn’t exist. Sure, that clear nice to be able to save the image from the music that was not correct. In practice, I don’t think I like it.

It’s a bit interesting that images edited with these AI tools get a Synth ID label to indicate that they’ve been edited by AI. Instagram found this when I uploaded a few photos, but it only shows the information if you click on “AI Info” on the photos. Writing isn’t the only thing blocking the air right now, and I think the biggest danger is the misguided notion that we can usually trust a picture someone takes and writes from their phone. Apple is not the smartest player here. But even raising a little doubt about the origin of the houseplant on the side table, or whether someone stood where it seems to have been photographed, can add up to a lot of trouble in the long run.

Photo by Allison Johnson/The Verge