Inside Facebook's plan to turn a scrapped AI project into a tool to reshape how millions communicate (FB)

fb messenger m suggestions 2x1 (1)

Facebook is on a mission to help people who speak different languages understand one another.
It's using M Suggestions, its virtual assistant in Messenger, to translate real-time conversations, and has just added support for French.
Business Insider spoke to Facebook employees working on the project about the tech behind it, and its potential to radically affect online communication.

In 2015, virtual assistants were the Next Big Thing.

Every major tech company had one, from Apple's Siri to Amazon's Alexa and Google Now, which would later become the more fully fledged Google Assistant. Facebook entered the space in August that year with the announcement of M — a chatbot that lived inside its Messenger app, and which users could ask for just about anything, from hiking recommendations to help buying flowers.

But while the likes of Alexa and Google Assistant have exploded over the last few years, finding their way into everything from fridges to cars, M — in its original incarnation — was never widely rolled out to Facebook's users.

Instead, it morphed into M Suggestions, an AI enhancement that hovers inside users' Messenger chats with their friends and offers contextual suggestions based on their conversations — from making payments to initiating video calls. These recommendations have historically been relatively incremental — send a funny GIF! Attach a sticker! — but Facebook is now leaning hard into translations in Messenger, and using M Suggestions as the engine to do so.

It's by far the most significant application of the technology behind M Suggestions to date — one that has the potential to radically reshape how hundreds of millions of people communicate in real-time on Facebook. Business Insider spoke to some of the team at Facebook working on the project, to learn more about the tech underpinning the ambitious project and their vision for its future.

"At Facebook we have a lot of different cultures and a lot of diversity in our team ... we are all speaking different languages, and we know how frustrating it can be not to be able to communicate ... in your own languages," said Laurent Landowski, a Facebook product manager. "So being able to also really provide M Suggestions for translation to the world is something that we're super proud of."

M is an AI assistant that lives inside other conversations

Every time you send a message to a friend via Messenger — assuming it's not an encrypted "Secret Message" — it's being scanned by Facebook.

That doesn't mean an actual Facebook employee is reading it, of course. Instead, Facebook's automated systems are parsing the message, trying to understand the intent of the message. This effort is partly underpinned by the tech Facebook acquired when it bought natural language startup Wit.AI back in 2015; cofounder Landowski is now the product manager of M Suggestions at Facebook.

If one of Facebook's AI neural net models identifies the message between you and your friend as something it can add context to, M Suggestions will automatically spring into action. If you mention a song, it might prompt you to play it on Spotify; if you're discussing multiple potential activities in a group, it might suggest creating a poll.

And it learns over time what different people utilize it for, and caters its responses accordingly; the version of M that a GIF-addict sees will be very different to what appears for someone who is more restrained in their messaging.

When it comes to quick replies — suggested responses M offers users in conversations to save them time — it even uses users' conversation histories as a training data to teach the AI how they speak, making the responses ("yes" versus "yeah" or "yep" or "yah") sound authentically like their voice.

Its unique positioning — inside users' existing conversations with other people, rather than dedicated human-to-AI chat windows — means it risks being invasive or jarring. As such, Facebook has moved slowly adding suggestions to M, Landowski said. "We have been working on trying to improve and really focus on the delight and relevance as opposed to the number of suggestions we could be suggesting."

He added: "It is super easy to lose user trust."

Flow_EN facebook m

Translations aren't easy — but the pay-off is huge

M Suggestions has thus far offered fun enrichments to conversations, but it's hardly transformative. Where that changes is translations.

Facebook has provided language translations on its core social network for years, first via traditional phrase-based translation techniques before migrating to a more advanced AI-powered neural net translation system in August 2017.

However, users who wanted to be able to talk across language barriers in real time were out of luck until earlier this year, when Facebook launched the first Spanish-English translations in Messenger, underpinned by M Translations. The company followed it up with the announcement this week that it was adding French.

While Facebook positions Messenger at least in part as a way to stay in contact with the people users are closest to, translations opens it up to assisting people who may never have interacted before — like in Marketplace, Facebook's peer-to-peer sales platform.

"You see more and more ways where translations can be applied, not only in your personal messaging but also like in the Marketplace, buyer-and-seller-type of messaging, that can unlock a lot of further opportunities for use," said Landowski, a native French speaker from Paris.

But translation isn't easy. Languages are always changing and shifting, evolving as slang becomes common parlance, and the problem is especially acute on an informal, real-time platform like Messenger. Facebook's language team has an "active taskforce working to adapt its models to the type of data that Messenger provides, said Necip Fazil Ayan, head of Facebook's language and translation technologies team.

"This is one of the hardest problems we have to deal with while working on translations at Facebook, and it's not a solved problem, making our systems more robust to informal language including slang," he said.

"It's a dynamic language right, and people keep inventing stuff ... my best example instead of just saying 'happy birthday,' [they] start introducing lots of P's or Y's or I's all over the place."

Facebook's unprecedented digital archives of billions of users' public and private conversations means it has a vast dataset with which to train its AI — but some languages have more material available than others. "One of the biggest challenges we're dealing with, both in terms of language understanding and translation perspective, is the set of what we call 'low resource languages.' As you can imagine, all the machine learning models require a lot of data to become accurate ... and we don't have that luxury for a lot of the languages we are dealing with," Ayan said.

And then there's the issue of bias. AI systems are only as good as the data they are trained on, and when there are human biases built into the data, it can creep into the results. "Our data is biased ... we are actively working on this at Facebook ... in machine learning this is a very hot area and it's a very difficult area, and it's going to take a while cleaning up the data from that type of bias or learning where the bias is."

But for all the challenges, automated real-time translations offer Facebook a way to have a have a fairly profound effect on the way people around the world communicate. "I really don't want language to become a barrier when people are expressing their opinions or when people are trying to reach other people to get different perspectives," said Ayan, a native Turkish speaker who grew up in the country.

"So that's the dream, we are going to break down the language barriers and that's my personal mission here."

'It's hard from an AI perspective to be able to create a fully automated assistant that can do everything'

Facebook now describes the original version of M as an experiment, one that provided valuable insight into the kind of things users utilise chatbots for but was never intended as a competitor to Siri et al, and never scaled beyond 2,000-odd users in the Bay Area. (In contrast, more than 100 million people interact with M Suggestions a month as of November 2017, a spokesperson said.)

"It's hard from an AI perspective to be able to create a fully automated assistant that can do everything,"Landowski said, "We basically realized that people, especially in Messenger, they really want to focus on the communication [assistance that M offered] ... it was really about trying to be where they are, which is their actual conversations. Instead of talking to an assistant directly it was all about focusing on the communications."

But there have been rumours circulating for months about Facebook building a smart speaker in the vein of the Amazon Echo of Google Home, with a voice-controlled AI assistant built in.

It was a no-show at Facebook's annual F8 conference in May 2018, but speculation was bolstered after a reverse engineer found references hidden in Facebook's code for an "Aloha" voice feature.

"We're exploring everything," Landowski said when asked about whether Facebook was currently looking at speech recognition.

"Speech is also an interesting way to actually interact with an assistant. We cannot comment more on exactly what we do and what we test, but definitely we are working and trying to improve this."

Contact this reporter via Signal or WhatsApp at +1 (650) 636-6268 using a non-work phone, email at rprice@businessinsider.com, WeChat at robaeprice, or Twitter DM at @robaeprice. (PR pitches by email only, please.) You can also contact Business Insider securely via SecureDrop.

Join the conversation about this story »

NOW WATCH: British Airways has a $13 million flight simulator that taught us how to take off, fly, and land an airplane