Why Netflix's 'thumbs up, thumbs down' ratings system is fundamentally flawed (NFLX)

Stranger Things

For years, Netflix would let you rate movies and TV shows anywhere from one to five stars. That changed in April, when Netflix removed the star system in favor of a new system that simply asked users: “Thumbs up, or thumbs down?”

Netflix’s explanation at the time was that the previous star-rating system was confusing to customers.

In that old system, if a movie on Netflix had four stars, people assumed that was the average rating among all users. After all, that’s what stars usually mean on online services that feature reviews, like Yelp. In reality, though, a movie with four stars meant Netflix was predicting you would rate that movie four stars based on your habits (and the habits of similar users).

With so much confusion around how the star-rating system worked, Netflix argued people were less motivated to rate titles since they thought their vote would be a drop in the bucket among the millions of other votes. People didn’t understand that using star ratings would only improve Netflix’s ability to understand your tastes. So Netflix introduced a new, more straightforward binary system earlier this year: Just give any title a thumbs up or thumbs down and Netflix would learn from your ratings.

Netflix made one other change, too: You would no longer see how many "stars" a movie had. Now, you only see a percentage — a number that represents the likelihood you'll enjoy what Netflix is suggesting.

This system might sound good on paper, but the flaws start appearing upon closer inspection.

Forget the smaller issues, like how titles that get a thumbs-down continue popping up in your various suggested lists (seriously, Netflix, I don't want to watch "Disjointed"). The bigger issue here is that the percent-match system is no better than the star system before it; if anything, it's just as confusing as before, and it still turns up results I don't agree with.

disjointed For example: On my account, Chelsea Handler’s Netflix talk show "Chelsea" is a 90% match. A Netflix show called “White Gold” is a 97% match. “Fuller House” is a 92% match. I tried all three of these shows, and I couldn't get past the first 5-10 minutes; I'm simply not interested in these shows.

Meanwhile, movies I’ve seen and really enjoyed are given comparatively lower percentages. I loved “Dawn of the Planet of the Apes,” but Netflix says we’re only a 79% match. “Guardians of the Galaxy” is easily my favorite Marvel movie, but Netflix says we're only an 84% match.

stranger things netflix Netflix would be better off with a more straightforward system, where a percentage or stars don't represent the likelihood you'd enjoy a film or a show, but represent how all other users rated that content. That's how most online ratings systems work — Amazon, Yelp, you name it — and it would make sense for Netflix.

The problem is, Netflix won't abandon this new system. The company is very particular about the information it wants to show users (since it won't show much), and Netflix likely has some statistics that support how well the current percent-match system works. For instance, Netflix has repeatedly said in the past that there's a difference between what you rate highly and what you actually like watching; the percent-match system probably supports this fact.

That said, there's a bigger underlying issue here: The mechanic that powers the new percent-match system is a binary thumbs-up or thumbs-down decision, which is inherently flawed.

This boils down to a philosophical issue: "Good" is not the same word as "great," just like "bad" is not the same word as "horrendous." Simply saying a movie or show was "good" or "bad" simply doesn't accommodate the myriad shades of grey. As one Reddit user put it, "there is now absolutely no difference between 'I guess I didn’t hate this movie' and 'This is the greatest movie I’ve ever seen in my life, and I want to see more like it.'"

daredevil This is also conventional wisdom for the film industry at large: Almost every single movie or TV show review you'll find is rated on some kind of gradient or scale, a number between zero and 10. Even Roger Ebert, the inventor of giving movies "thumbs up" or "thumbs down," eventually rated movies from zero to four stars when he launched his own website. And before that, when "Siskel & Ebert" was on the air, both critics always backed up their binary choices with a ton of explanation and nuance that Netflix's system simply doesn't capture.

It's fine if Netflix wanted to change stars to percentages — they still mean the same thing: the likelihood Netflix thinks you'll like what it's suggesting — but people should still be able to rate movies from zero to five stars instead of a simple thumbs up or down. It feels almost antithetical to deprive people of choice in an age where people crave more nuance and information, where even Facebook's "Like" button is now supplemented by other emotions. I understand Netflix invested a significant amount of time and money into this current setup, but at the very least, I think it's time to get rid of the thumbs.

Join the conversation about this story »

NOW WATCH: The 5 best hidden features from the latest iPhone update