The way that I will tend to read up on a game, is to go to Metacritic and select three of the top-scoring reviews, three mid-scoring reviews, and three low-scoring reviews – I couldn’t easily sample the full spectrum of opinion without the existence of arbitrary review scores.

However, with thousands of comments on reviews being outraged at a number and companies giving bonuses based on metacritic scores, it is certainly out of hand.

How I tend to utilize reviews is by saying “What does this reviewer NOT like about the game and can I personally overlook enough of those things/do enough of those things not bother me to justify playing the game?” And of course I read multiple reviews when I do this. Usually all the good points a game has to offer are things I already know or else I wouldn’t have been interested enough to read the review.

As a thought exercise, say Game A had an average score of 7.2/10 and an SD of 0.6 (indicating a decently big spread of review scores, assuming a large sample size) while Game B had a mean of 8.1/10 but an SD of 0.1 (meaning everyone roughly agrees on the score). The easiest thing to see is that virtually everyone agrees on how good Game B is, but Game A seems to only strike a chord with a smaller segment of reviewers.

Looking a little deeper into the stats, you would find that the games are not significantly different by research standards. Many times, people use 2 times SD for significance so:
7.2 + (2*0.6) = 8.4
8.1 – (2*0.1) = 7.9
And because these two scores cross over, by definition, they aren’t that different in score.

If the numbers were, say, 3.8/0.4 and 7.1/0.6, you’d get
3.8 + (2*0.4) = 4.6
7.1 – (2*0.6) = 5.9
And because they don’t cross over, you could safely say that one game is better than the other.

Of course, sites like Metacritic would never go through enough effort for something like this, and only a small segment of the population would probably even give a crap. But to me THAT is really where review scores are helpful. Not as a floating data point that some fanboy can trumpet in a forum somewhere.

I’ve seen the Sony and MS units before, but they bear reposting. That 3DS XLpro mockup, however, is pretty damn funny. Put that d-pad right next to that other d-pad!

The Circle Pad Pro XL!

Sony’s Absurd Racing Harness!

Microsoft’s Double-Ended Pleasure Device

There is no such thing as a 9 game or an 8 game or a 5 game. There are games. They have good points and bad points. The idea that there is a way to standardise ‘scores’ is ludicrous, but far worse and more subtly insidious is the idea that a gaming experience can (or, indeed, should) be reduced to a single numerical value.

I stand by what I said: review scores are terrible and they should be scrapped entirely. They are based on an obviously false premise–one that is harmful to gaming.

@Lusi: I don’t think that the numbers themselves are necessarily the problem, and I’m not sure just flat scrapping the system is the best answer. Much like nuclear energy, it’s all in how you use it, and right now review scores aren’t being used productively, they’ve been weaponized. Unfortunately, human beings have an innate ability to find the worst way to use a good idea. (This is why we can’t have nice things!) If things were standardized from site to site and reviewer to reviewer, Metacritic would be a great resource because it would show whether a game is really divisive or everyone agrees that it is either terrible, mediocre or great. Unfortunately, that’s just a pipe dream.

Review scores and their constant re-tooling and re-thinking are perhaps people attempting to build a better mouse trap in a world where mice don’t exist. In general, classification of things like movies and games are a limiting factor. More often people turn things down based on their classifications than take them up. OC Remix, which you link to on the bottom of the site, doesn’t offer a way to search their remixes by genre because the guy who runs the site (DJ Pretzel) thinks that this would only lead to people listening to fewer remixes than more.

The best way to fix the review system is to abolish numerical/letter/etc. systems and just write a review. Let the reader decide, in their mind, based on the content of the review, what sort of grade it deserves, if they so choose. For years, book reviews did this and it worked great. Then, at some point, the laziness of stars and thumbs and so on hit the scene and now we have the miserable situation we live in today.

Scoring systems are laziness on the part of the reviewer. Full stop.

@Mel: A lot of sites are inclining towards this, albeit not necessarily gaming reviews. Things are now delineated into Like/Dislike, because what we have seen on sites like Metacritic is that the vast majority of users are inclined to either give something a 10 or a 1. I still think it is laziness.

I think I have 4 different copies of the original Final Fantasy. I might have a problem.

What is it about Namco’s name that makes it so easy to Spoonerize? I always want to call them Bamco Nandai.

The interpretation of the review system has become broken. So many sites now grade on an academic curve so a 7/10 is seen as mediocre when that should still be seen an above average game. This is the same problem that sealed the fate of Alpha Protocol. Unfortunately, I don’t see how this is going to be fixed anytime soon, though.