With hate speech becoming more difficult to tackle, this startup has created an AI-powered service to help businesses including TikTok flag offensive content

Timothy Quinn co-founder Hatebase

Hatebase, a Toronto-based artificial intelligence startup, has built a service that alerts companies and organizations when their users post offensive content.
The startup is building a database that includes hateful words and phrases from more than 95 languages that are used in more than 180 countries.
Its technology is used by the world's four biggest social networks, including TikTok; those companies and others face growing pressure to get hate speech under control.
Hatebase began as an initiative of the Sentinel Project, a Canadian nonprofit that uses technology to assist vulnerable communities that are potential targets of violence and genocide.
Click here for more BI Prime stories.

Hate and lies can spread so rapidly on the web nowadays that companies, especially social networks, are eagerly looking for help in getting a handle on the problem.

To date, many have tasked human moderators with the job of keeping an eye out for slurs, racist attacks, fake news, and other harmful content. But the work is excruciating, mind-numbing, and often soul-killing, sparking controversy for traumatizing those who are exposed to so much disgusting material.

"It's just a horrible, horrible job" for people who digest "the worst of the worst of social networks 12 hours a day," said Timothy Quinn, co-founder and chief technology officer of Hatebase, which offers a technology-based alternative to human content moderation.

"You want to get away from that," he told Business Insider. "You want to automate as much as possible."

That's what Quinn's Toronto-based startup is trying to do with its artificial intelligence-powered service. Numerous businesses and organizations, including TikTok and the world's largest social networks, use Hatebase's service to flag offensive content on their sites.

A database of hate words in different languages

Hatebase has built a database of hateful words, phrases, and expressions it's collected from more than 95 languages that are used in more than 180 countries. Clients are able to use the database to set up their own customized systems for flagging offensive content.

A network of volunteers and contractors keeps the database, which now includes more than 3,600 terms from around the world, up-to-date. As of December, Hatebase's service has sent more than 1 million alerts to its clients about the use of such words. A recent entry flagged the term "redneck" after it was used somewhere in the US, while another cited the use of "preto," Portuguese for "dark-skinned person," in Brazil.

Identifying hate speech can be a complex and tricky process, Quinn said.

"It's extremely hard to put boundaries around hate speech because everyone defines it a little bit differently," he said. "There's that blurry line between where hate speech ends and where ... insults and trash talking begin. These are all very thorny issues."

A broad definition of hate speech

For its purposes, Hatebase came up with a definition of hate speech that is broad and practical, Quinn said. As the company explains on its website, its database includes any term "that can be used to marginalize generally in a negative way or externalize a different population based on nationality, ethnicity, gender, sexuality, religion, disability."

"Anything that fits into that bucket we are going to broadly call hate speech," Quinn said.

Since many words and expressions can have different meanings in different countries or contexts, Hatebase's database includes terms that some people might not consider offensive. Quinn cited "paddy wagon," a term that was once considered a slur against the Irish, who were sometimes derisively called Paddies and were often portrayed as criminals in the 1800s.

"Now I would never be personally insulted if somebody referred to a vehicle as a paddy wagon," Quinn, who is of Irish descent, said. "But it is externalizing a population and associating a given ethnicity with criminal behavior."

That's why "paddy wagon" is part of the Hatebase lexicon, although the company also classifies it as "extremely low" in terms of its sensitiveness, Quinn said.

Hatebase's clients can tweak its service for their needs. They can add words or phrases to the database and adjust the service's rules to be more in tune with where they are based or the cultural sensitivities of their audience.

The startup's database itself is a work-in-progress and continues to evolve. That's because languages, including hate speech, are constantly changing, Quinn said. New slang words are always being invented and the meanings of particular terms often change over time.

Hate speech is not just a PR problem

In fact, some hate groups have actually started to intentionally obfuscate their messages in a way that only their members and sympathizers understand, he said.

"White supremacists have become masters of this in how they post content using code words using certain kinds of punctuation," Quinn said.

Increasingly, for many businesses, including social networks such as Facebook and Twitter, policing hate speech is no longer just about good public relations. Failing to do so can now have serious implications for such companies, as Facebook found out when it was hit with a $2.3 million fine for violating hate speech laws in Germany.

"If you're a big social network, you're concerned about people posting material on your site that is potentially alienating," Quinn said. But, he added, "increasingly there are stiff fines now for online ecosystems that allow hate speech to flourish on their site."

Hatebase is bootstrapped rather than venture-funded. It has a paid staff of about 12 who are assisted by dozens of volunteers. It's also profitable, with hundreds of thousands of dollars in revenue, Quinn said.

One Hatebase client is Fiksal, a Utah-based startup that developed an app that helps individuals and organizations communicate more effectively and in ways that avoid conflict. Hatebase's system helped Fiksal better understand cultural differences, particularly in the meanings of particular words and phrases, Fiksal CEO Krista Andersen told Business Insider.

"They're really a stellar company with a big vision and clearly passionate about helping people understand how damaging hate language is," Andersen said.

Hatebase started as a part of nonprofit anti-hate group

Launched in 2012, Hatebase began as part of a campaign to prevent hate speech from spinning out of control. It was created by the Sentinel Project, a Canadian nonprofit that's working to assist vulnerable communities that are potential targets of violence and genocide.

Two years ago, the Sentinel Project team used technology, including a heat map, to monitor false reports that a gang was about to attack some slum communities in Kenya.

"We were able to tailor our response to those communities; calming people who contacted us worried about their safety, diffusing the potential for vigilante mobs to form, and directing concerned individuals away from higher risk areas, such as where protests were forming," said Drew Boyd, a member of Hatebase's leadership team and the operations director of the Sentinel Project, told Business Insider.

Christopher Tuckwood, another member of Hatebase's leadership team, pointed to the company's Kenya experience as an example of how its service can play a critical role in neutralizing the potential impact of hate speech.

"We envision hate speech data being used as a warning indicator alongside other types of data — misinformation, violence rates, economic changes — which give an overall picture of the risk of instability and conflict in a given place, including severe cases like genocide and mass atrocities," he said.

Got a tip about Hatebase or another tech company? Contact this reporter via email at bpimentel@businessinsider.com, message him on Twitter @benpimentel or send him a secure message through Signal at (510) 731-8429. You can also contact Business Insider securely via SecureDrop.

Join the conversation about this story »

NOW WATCH: A podiatrist explains heel spurs, the medical condition Trump said earned him a medical deferment from Vietnam