Yesterday, a blog post from developer Jeff Johnson highlighted an “Enhanced Visual Search” toggle added to the Apple Photos app in iOS 18 that gives your phone permission to send photo data to Apple. In exchange, you can identify landmarks in your photos or search for them in your picture library. It’s also turned on by default, and until now, we weren’t aware of the setting, which has largely gone unmentioned by Apple or in coverage of the new iPhone OS.

Screenshot: iOS 18 Settings app

The feature is like Apple’s “Visual Look Up,” which uses machine learning to identify plants or animals in your photo library. To see it in action, swipe up on a picture in your library showing a building or landmark and tap “Look Up Landmark.” Photos will then ideally tell you what it is.

Here are some examples from my own library:

That’s definitely Austin’s Cathedral of Saint Mary, but the image on the right is not a Trappist monastery, but the Dubuque, Iowa city hall building. Screenshots: Apple Photos

A description under the Enhanced Visual Search toggle says the feature will “privately match places in your photos,” though it’s not clear from the description how that happens or that it means sending data from your photos to Apple. According to a company research blog, it involves your phone creating vector embeddings — of the part of a picture containing a landmark, and sending that and several “fake queries” to Apple for analysis. Your phone then chooses the final match from a batch of possibilities that Apple sends back.

…if something happens entirely on my computer, then it’s private, whereas if my computer sends data to the manufacturer of the computer, then it’s not private, or at least not entirely private … You don’t even have to hypothesize lies, conspiracies, or malicious intentions on the part of Apple to be suspicious of their privacy claims. A software bug would be sufficient to make users vulnerable, and Apple can’t guarantee that their software includes no bugs.

Apple did not respond to our request for comment on Johnson’s concerns.

You can find the Enhanced Visual Search option on iOS / iPadOS under Settings > Apps > Photos, or under the “Search” heading in the Photos > Settings on a Mac.

Correction, December 29th: An earlier version of this story misstated the location of the Enhanced Visual Search toggle. It is in iOS Settings under Apps > Phone and in Photos > Settings in the macOS Photos app. The headline was also made clearer.

Original Author: Wes Davis | Source: The Verge