This New AI Can Retouch Your Photos Before You Snap Them

What if your camera could professionally retouch your photos… before you even shoot them? That’s what researchers at Google and MIT are currently working on.

MIT News reports that this week at the Siggraph digital graphics conference, Google and MIT scientists will be showing off a new system that can automatically retouch photos in the style of a professional photographer in real-time.

Just as apps like Instagram can show you a real-time preview of what a scene looks like with a particular filter, the new AI system can show you a custom retouched version of any scene as you point your camera at it.

Here’s a demo of the retouching being done on a smartphone (it can be turned on and off):

“Performance is a critical challenge in mobile image processing,” writes MIT researcher Michael Gharbi. “Our algorithm processes high-resolution images on a smartphone in milliseconds [and] provides a real-time viewfinder at 1080p resolution […]”

The system is first trained on a vast library of photo pairs (one raw photo and one retouched one) — the researchers used 5,000 photos. Once it’s trained, it can operate completely offline using what it learned. The AI can then retouch images in real-time due to the fact that it does the bulk of the heavy-lifting image processing on a low-resolution version of the input photo first. Once it comes up with its retouching recipe, the formula is applied to the high-resolution photo to approximate the retouched image.

This is the new “shortcut” that allows this retouching to be used in a live view while a photographer is framing a shot. Here’s a 3-minute video that dives into the technical details of how this new system works:

“This technology has the potential to be very useful for real-time image enhancement on mobile platforms,” says Google researcher Jon Barron. “Using machine learning for computational photography is an exciting prospect but is limited by the severe computational and power constraints of mobile phones.

“This paper may provide us with a way to sidestep these issues and produce new, compelling, real-time photographic experiences without draining your battery or giving you a laggy viewfinder experience.”

(via MIT via The Verge)