Review scores have always troubled me. Ever since I found out how the industry uses them to judge bonuses for developers (something I knew long before that article was published thanks to friends working in the industry) the score I ended up giving the game always had a level of gravitas attached to it, even if I knew only a couple dozen people read the review. I tried to clarify my position in an attempt to give more insight into how the final score was constructed but still that nagging feeling remained. Since then I’ve had many discussions with friends about review scores in general and I’ve come to the conclusion that they’re staying around but there’s a set of rules around their use.
For starters I think the biggest issue with review scores in general is when they’re used in aggregate to compare titles on the same medium (games/movies/books/etc.). The problem with this is that it ignores the unique perspective and thought process that goes into curating that score, something that’s intensely personal and becomes meaningless when stripped of its context. For instance my preference to start games out with perfect scores and then take points off is likely not the same process other reviewers go through and thus my 8.5/10 does not compare to the same score elsewhere. Indeed I try incredibly hard to lay out any personal bias on the table so that when you do see that final score you can make a decision as to how valid that is compared to your viewpoint or other reviewers you might follow.
Following on from this its logical to then assume that scores bear no correlation between different game genres. It’s impossible to compare something like MirrorMoon EP to Payday 2 using only the review score because their commonalities end almost as soon as they begin. However I feel comparing MirrorMoon EP to say Kairo quite valid as they’re quite similar in many regards and whilst their differences are nuanced if you wanted a general idea of how they compared to each other the review score is then appropriate. I’ve been told that this kind of philosophy is what drove the late and great Roger Ebert’s review scores for movies and I believe that it’s very applicable to the world of gaming.
Of course I can really only enforce these rules here where I have total control over how the content is presented but I think some generalization of these ideas applied widely would go a long way to reversing some of the damage that review scores have done to various parts of the industry. They’re still not ideal of course, nothing that boils down hours upon hours of invested time to a single digit is, but if we as consumers become more nuanced in the way we use reviews then they might start to become meaningful. This would have to go hand in hand with turning down our usage of review aggregators as they’re arguably the primary source of most of the complaints that center around review scores.
I don’t see that happening anytime soon but I can at least do my part to improve the situation.
Welcome to review guide for The Refined Geek. If you don’t need the full explanation of why I review the way I do feel free to skip to the TLDR section at the bottom.
It’s come to my attention that whilst I think my method for reviewing games is somewhat transparent it’s really anything but. Although I’ve tried to keep the same format and style over the past couple years I’ve noticed that my initial reviews are really worlds away from my current format. So in the effort of transparency (and hopefully adding some clarity) I thought I’d give you a brief overview of the process I go through in order to bring you game reviews every other week or so.
The first, and of course most enjoyable part, is that I’ll play the game that gets reviewed. Now my preference for games is the PC as that’s the best platform I’ve found for conducting reviews. I’m not adverse to playing on other platforms (as shown by my reviews of games like Assassin’s Creed, Heavy Rain, Red Dead Redemption, etc.) but I prefer it since I can take my own screen shots (all console games have thus far used press kit screen shots) and I’m not taking up the main TV which I have to sometimes fight my wife for. I am working on a solution for reviewing console only titles but I haven’t found a solution that works exactly the way I want it to.
One big point I make is to finish all the games that I play before reviewing them. This can be something of a burden with games that pack on the game time (like Skyrim, for instance) but I feel reviewing a game I didn’t complete doesn’t do it justice. The problem with this of course is that some games will drag out too long and I won’t finish them or, what usually happens, is that the game has nothing redeeming about it and I won’t attempt to finish it. I’ve had several games like this and unfortunately since my time is somewhat limited I don’t like to waste it on titles that just aren’t fun to slog through.
My review scoring system is quite simplistic: all games start off with a perfect 10/10 score and then will lose points for things that detract from the game experience. Think of it like an innocent until proven guilty sort of scenario where I like to err on the side of a game being good rather than it being bad. The issues are mostly subjective but there are also some objective things like game breaking bugs, bad performance or poor game design. The amount I take off is somewhat subjective, usually based around how bad something detracts from the game, and I haven’t bothered itemizing it since if you read my review you can easily tell where the game lost points. Still if there’s interest in seeing a breakdown of why I took points off for certain things I’ll be sure to include it in future reviews.
I’ve had some complaints that my review scores are too high and hopefully the last 2 paragraphs have explained the reasons as to why this might be. The high scores are a combination of my scoring scheme (everything starts out high) and the fact that I simply don’t finish crap games (so everything that gets reviewed meets a certain standard). I’m not one of those 7/10 guys nor are the game developers paying me for their reviews as all games (bar one exception, which I made clear at the time and will always do so) has been bought with my own money and played just like any other retail customer. Whether that makes the final score useful to you or not is left as an exercise for the reader but it is a useful guide to how I feel if you’re not in the mood for reading the entire review.
I try to keep the structure consistent if for the sole reason that it stops me writing an incoherent mess. For the most part I’ll start with some background of the game, genre or developers depending on what kind of history I have with those aspects. Then I’ll usually set the scene for the game to give the following sections some context. Then I’ll usually break it down into specific sections like graphics and performance, game and set design, combat, game mechanics, overall plot summary and finally multiplayer. The multiplayer section is a relatively recent addition to the reviews as I had avoided it previously but since I’ve had many games that are defined by their multiplayer experience I found it necessary to include it.
If you feel like I’m missing something important or you’d like to see a particular aspect included in future reviews feel free to drop me a comment here, send a tweet to @davidklemke or drop me a line at [email protected]