Snapchat code reveals team-up with Amazon for ‘Camera Search’

Snapchat is building a visual product search feature, codenamed “Eagle,” that delivers users to Amazon’s listings. Buried inside the code of Snapchat’s Android app is an unreleased “Visual Search” feature where you “Press and hold to identify an object, song, barcode, and more! This works by sending data to Amazon, Shazam, and other partners.” Once an object or barcode has been scanned you can “See all results at Amazon.”

Visual product search could make Snapchat’s camera a more general purpose tool for seeing and navigating the world, rather than just a social media maker. It could differentiate Snapchat from Instagram, whose clone of Snapchat Stories now has more than twice the users and a six times faster gro

wth rate than the original. And if Snapchat has worked out an affiliate referrals deal with Amazon, it could open a new revenue stream. That’s something Snap Inc. direly needs after posting a $385 million loss last quarter and missing revenue estimates by $14 million.

TechCrunch was tipped off to the hidden Snapchat code by app researcher Ishan Agarwal. His tips have previously led to TechCrunch scoops about Instagram’s video calling, soundtracks, Focus portrait mode and QR Nametags features that were all later officially launched. Amazon didn’t respond to a press inquiry before publishing time, and it’s unclear if its actively involved in the development of Snapchat visual search or just a destination for its results. Snap already sells its Spetacles v2 camera glasses on Amazon — the only place beyond its own site. Snap Inc. gave TechCrunch a “no comment,” about visual search but the company’s code tells the story.

Snapchat first dabbled in understanding the world around you with its Shazam integration back in 2016 that lets you tap and hold to identify a song playing nearby, check it out on Shazam, send it to a friend or follow the artist on Snapchat. Project Eagle builds on this audio search feature to offer visual search through a similar interface and set of partnerships. The ability to identify purchaseable objects or scan barcodes could turn Snapchat, which some view as a teen toy, into more of a utility.

What’s inside Snapchat’s Eagle eye

Snapchat’s code doesn’t explain exactly how the Project Eagle feature will work, but in the newest version of Snapchat it was renamed as “Camera Search.” If you remember, Snap used another animal name, “Cheetah”, as the secret word for its big redesign. The app’s code lists the ability to surface “sellers” and “reviews,” “Copy URL” of a product and “Share” or “Send Product” to friends — likely via Snap messages or Snapchat Stories. In characteristic cool kid teenspeak, an error message for “product not found” reads “Bummer, we didn’t catch that!”

Eagle’s visual search may be connected to Snapchat’s “context cards,” which debuted late last year and pull up business contact info, restaurant reservations, movie tickets, Ubers or Lyfts and more. Surfacing within Snapchat a context card of details about ownable objects might be the first step to getting users to buy them… and advertisers to pay Snap to promote them. It’s easy to imagine context cards being accessible for products tagged in Snap Ads as well as scanned through visual search. And Snap already has in-app shopping.

Snapchat’s Camera Search could become a direct competitor for Pinterest’s Lens, which identifies objects and brings up related content. Pinterest has evolved the product, embedding it inside the apps of retailers like Target. Beyond shopping, Camera Search could let Snapchat users find Stories that contain the same object they’re snapping.

Being able to recognize what you’re seeing makes Snapchat more fun, but it’s also a new way of navigating reality. In mid-2017 Snapchat launched World Lenses that map the surfaces of your surroundings so you can place 3D animated objects like its Dancing Hotdog mascot alongside real people in real places. Snapchat also released a machine vision-powered search feature last year that compiles Stories of user-submitted Snaps featuring your chosen keyword, like videos with “puppies” or “fireworks,” even if the captions don’t mention them.

Pinterest’s Lens visual search feature

Snapchat was so interested in visual search that this year, it reportedly held early-stage acquisition talks with machine vision startup Blippar. The talks fell through with the U.K. augmented reality company that has raised at least $99 million for its own visual search feature, but which recently began to implode due to low usage and financing trouble. Snap Inc. might have been hoping to jumpstart its Camera Search efforts.

Snap calls itself a camera company, after all. But with the weak sales of its mediocre v1 Spectacles, the well-reviewed v2 failing to break into the cultural zeitgeist and no other hardware products on the market, Snap may need to redefine what exactly that tag line means. Visual search could frame Snapchat as more of a sensor than just a camera. With its popular use for rapid-fire selfie messaging, it’s already the lens through which some teens see the world. Soon, Snap could be ready to train its eagle eye on purchases, not just faces.

In related Snapchat news: