This AI Can Convert Paintings Into Photos and Summer Into Winter

A team of researchers at UC Berkeley have revealed an ‘Unpaired Image-to-Image Translation’ technique that can do something really interesting: it can turn a painting by Monet into a ‘photograph’… also, it can transform horses into zebras and summer into winter.

The AI/algorithm/whatever-you-want-to-call-it is yet another addition to the photo style transfer research that it seems like everybody is working on—from the basic tech behind apps like Prisma that transform photos into paintings, to the incredible style-transfer tech from Adobe and Cornell we showed you last week.

What the team at UC Berkeley developed is a little like both of these examples, but their AI can actually do several of different things. First off, it can do a “reverse-Prisma” trick of taking a classic painting and transforming it into a “photograph.” In these examples, it transforms Monet’s impressionist paintings into something resembling real-world photographs:

But the tool goes further than that. It can also turn horses into zebras, or winter into summer, by transposing style elements and textures similar to the Adobe algorithm:

It can even do more practical effects, like applying depth of field:

The groundbreaking aspect of this research is that the UC Berkeley team isn’t using “paired examples.” As the researchers explain, this means they don’t have a photo of the scene Monet was painting from the moment he was painting it to use as a reference.

“We can imagine all this despite never having seen a side by side example of a Monet painting next to a photo of the scene he painted,” explains the paper. “Instead, we have knowledge of the set of Monet paintings and of the set of landscape photographs. We can reason about the stylistic differences between these two sets, and thereby imagine what a scene might look like if we were to ‘translate’ it from one set into the other.”

So, how will this technology affect the photographers of the future?

Like the Adobe/Cornell research, the most basic application we can think of is presets: capturing your distinct photo editing style and applying it intelligently to your whole portfolio will be a breeze. But that’s just the beginning. Taken a step further, you could turn summer into winter, cloudy skies into a beautiful sunset, and… well… horses into zebras.

(via Engadget)

Image credits: All research examples courtesy of UC Berkeley.