A lot has been said about The Last of Us II's Metacritic user review score. Fans of the game claim a group of "manbabies" review bombed the game, lowering the score significantly. People critical of the game claim "shills" love bombed the game with 10/10 scores and the game sits at a higher score than it should. As of the time of writing this article, the score sits at a 5.4 based on 123,492 Ratings.
There have been attempts to scrape the user review scores by several people, but most either failed to include a large enough sample size or made some error and pulled their data. One person, AyaNeko on Twitter, was able to scrape 63,802 user reviews. All user reviews that were up when the program was run. You can view that data at the following link Neko's Datamine. The data can be viewed with all the reviews(63,802) included, only the reviews(53,803) by people with 1 review under their belt, only the reviews(9,999) with more than 1 review under their belt, or all the reviews(536) with identical text.
For those unfamiliar with data scraping I had Aya Neko explain it in plain language, and he said the following. "I looked at the user reviews in Metacritic, you sort those by date and make it show 100 reviews for each page. My script goes through all those pages from page 1 to 640+, then writes those user reviews and some info into a database. I took the review date, score, username, and the url so i can identify which page i got this review from. I got 63802 reviews as of yesterday(7/10/2020). Later I also wanted to filter out those users who only wrote 1 review, so i had to crawl through all user profiles and check how many reviews they made. That took long because the reviews were only 640+ pages to crawl, but each user profile was a page each, so i was crawling through 63802 profile pages. After that it was just a matter of aggregating the data, make some calculations, how many % which scores, how many % only have 1 review in their profile, etc."
Now, before I explain some of the finding that has been learned by analyzing the data I have to make you aware of a few caveats. First off, there are more user reviews today then when the data was scraped and compiled. Second, there are 59,690 ratings that contribute to the user review score that do not have a written review. There is no way to know how many of these ratings are bots, review bombers, love bombers, or genuine ratings. Third, there is no way to know if people who only have 1 review under their belt are in fact review/love bombing or if they are simply people who never had a reason to use Metacritic prior to TLOU2. Fourth, there is no way of knowing how many people gave the game a 1 or 9 in an attempt to avoid being seen as a review/love bomber. The fifth caveat is we don't really know how many 10/10's or 0/10's are legit scores and not review/love bombs. The final caveat is there is no way to know if reports about reviews being removed or disallowed to be posted are true or just internet rumor.
With all that said, what did the data reveal? With all the data being used, the user review score would be a 5.145. So, the 59,690 ratings Aya Neko was unable to compile only bumped the score up a measly .255. Possibly less, depending on how Metacritic rounds its scores. Of these reviews, 27.28% were 0/10 reviews. The 10/10 reviews were a bit higher, clocking in at 35.18%. That said, 62.46% of all the user reviews in the data were either 10/10 or 0/10 reviews. Seems pretty sketch to me.
Looking at the reviews done by people with only 1 review under their belt, 83.44% of all the reviews, shows a very similar distribution in review scores. The overall score is a 5.222 with 27.32% being 0/10's and 36.37% being 10/10's. This means 63.69% of the reviews made by people with only 1 review under their belt were either 10/10's or 0/10's. This makes sense as the overwhelming amount of reviews for TLOU2 were made by people with only a TLOU2 review on their profile, so the data lining up so close was expected.
If we look at the data of people with more than 1 review under their belt, 15.67% of all the reviews, then you can see review score distribution is very similar to the other data sets. The overall user review score drops to a 4.729 with 27.08% of the scores being a 0/10 and 28.76% being a 10/10. This set of data has 55.84% of it's reviews being either a 0/10 or a 10/10 which still seems pretty odd to me.
If we completely remove the reviews that came from people who only reviewed TLOU2 and disregard the 0/10 and 10/10 we come up with 4,415 total reviews with an overall user score of 4.195. This is a 1.205 difference from the current score sitting on Metacritic. This tells us that if the 10/10's and 0/10's are illegitimate scores than the game has been love bombed more than it has been review bombed. In fact, for ever three 10/10 reviews we only get about two 0/10 reviews.
What does all this mean? Well, people who love the game as well as people who hate the game have gone out of their way to try to manipulate the user review score of the game. As of now, it seems the ultra fans of TLOU2 are winning. Having never played The Last of Us and with no interest in The Last of Us 2 I have no horse in the race and I am only writing this to provide accurate data and relevant information to the discussion. Thanks for reading. Below you can find the video Aya Neko made on his process of scraping TLOU2 review scores.
Comments
Post a Comment