Posts Tagged‘quality’

Why User Reviews Are (Mostly) Useless.

The whole idea of boiling down an entire game experience to a single number is something that’s started to wear on me of late. It’s supposed to make for an easy judge of the overall quality of the game, combining all aspects of it together to give you something that makes it easy to compare it against other titles. However there’s usually far too much nuance in any particular game for that single number to be meaningful and whilst I still give them overall ratings I hope that the readers go through the whole review before reading the score so they can understand what lead up to it. There’s also the question of innate quality and how that should be reflected in the overall score, something which has lead to many questioning why critic reviews scores tend towards the upper end of the spectrum.

For me the explanation is simple, it’s survivor bias for the games that I actually finish (I hate not finishing games before reviewing them and will point that fact out when I do). This means there’s a certain quality bar that has to be hit for me to make it the whole through before giving it up and that pushes my average scores upwards. At the same time the objective quality of games (things like bugs, how the game plays, performance, etc) is actually quite high when you compare it to the past and thus it’s hard for a reviewer to give a game an absolutely terrible score when for the most part it’s a well done game.

Users on the other hand aren’t so sympathetic. Take for instance the current review scores for Call of Duty: Black Ops 2:

Taken at face value that’d have you thinking that it’s an absolutely atrocious game, one that the vast majority hates with an unbridled sense of passion. Contrasting that with the critic reviews and it’s easy to see why a lot of people jumped on the DoritoGate bandwagon, proclaiming loudly that all the reviewers where in the publisher’s pockets. The complaints are almost identical to all the previous releases most stating that it’s the same game with nothing new to offer or saying that the previous ones were better and they can’t believe they shelled out for it. Digging into the reviews however I started to notice some patterns that tell you that these user review scores are for the most part, total crap.

The first type of review pattern I came across was what I’ll call the Negative Nancy who’s long history of reviews are dominated mostly by scores in the 0-2 range. I’m not exactly sure what these users get out of writing and rating all these games so lowly but it seems like they’re dedicated to sending the user review score down as far as they can go even if they admit that the game has some redeeming features. Suffice to say it’s hard to take someone’s opinion seriously if all they’re doing is rating everything badly as you have no reference point to determine whether their style of reviews lines up with yours (unless you too, hate every game released).

The second, and most telling, are the Extremists. They are capable of dishing out both positive and negative reviews but do so only on the extremes of the spectrum with everything either being perfect or nothing at all. This kind of reviewing can be fine, if you give a rating in the form of something like recommend/don’t recommend, however their scores aren’t really any indication of the relative quality of the games due to the lack of graduation between the good and bad titles. That might be enough for some people but in all honesty if you want to know if a game is worth playing or not these kinds of reviews aren’t great indicators of that.

The final piece of information I’ll leave you with is the fact that there’s many people who are reviewing Black Ops 2, and in fact any of the most popular games, have just a single review. Now I’m not saying that only having one review discredits them completely but the fact that most of them have signed up just for the purposes of giving a negative review says a lot about their motivations for doing so. Indeed I believe many people will probably see an article like this one posted on a website and will immediately hop on the negative review bandwagon simply to be part of the crowd.

After saying all that though there were some negative reviews on there from people with long histories of reviews with varying levels of scores and those are the kinds of user reviews you can put some weight behind. They’re annoyingly rare unfortunately with most falling on either side of the extreme, rendering the overall score completely useless. The critic reviews are only better due to the long articles that come along with them (and not because of their overwhelmingly positive scores) and not the 2 sentences that accompanies the user reviews.

If you’re actually interested in proper reviews though I’m probably preaching to the choir here. We all know that games are incredibly hard to boil down to a single number and that’s usually heavily influenced by the reviewer’s biases. I try to lay all mine out on the table so you can get a feel for what led me to give the final score but I can’t say the same about the vast majority of user reviews on Metacritic I’ve read over the years. There are some good people on there but it’s akin to finding that elusive needle in a haystack, something that’s just not worth doing all the time.

Samsung’s Galaxy S2: Powerful, Elegant, Sublime.

I’m pretty fiscally conservative when it comes to my own cash, agonizing over purchases for sometimes weeks at a time before I take the plunge. It’s enough to outright kill some purchases entirely like the Motorola Xoom that I was convinced was worth at least having around just for the “tablet experience” but couldn’t seem to pass my financial filter. There are however times when my inner geek becomes so impressed with something that it overwhelms any sort of fiscal responsibility and I’ll find myself in possession of my object of desire well before I realize that I’ve taken my credit card out of my wallet. The Samsung Galaxy S2 is a brilliant example of this as I had been looking for a new phone for a while (and the Windows Phone 7 handsets available weren’t wowing me) and a quick trip to the specification sheet had me deep in geek lust, and 3 days later I had one in my hands.

The Galaxy S2 is really another world away from any other handset that I’ve had the pleasure of using. It’s quite a wide unit with the main screen measuring an impressive 4.3″ (10.92cm) across the diagonal but it’s also incredibly slim, being only 8.49mm thick. It’s also incredibly light weighing in at a tiny 116g which you’d think would make it feel cheap when compared to other similar handsets (the iPhone 4 is much more meatier) but the construction of the handset is very solid despite it being entirely plastic. The front screen is Gorilla glass which is incredibly resistant to scratches. I haven’t had a single scratch on it despite dropping it a couple times and putting it in my pocket with my keys by accident, something that would’ve ruined a lesser phone. To say that the first impressions of just holding the handset are impressive is putting it lightly, it’s simply an incredible device to hold.

In fact coming directly from an iPhone to the Galaxy S2 I can see why Samsung is in hot water with Apple over this particular device. I’ve covered the TouchWiz interface being strikingly similar to iOS in my Android review but the handset itself is also very Applesque, sporting the same single physical button on the front right in the same location that Apple has. Although its hard to accuse them of outright copying Apple since you can only get so creative with large touchscreen devices, especially when some of the required buttons are dictated by the underlying OS.

Under the hood of this featherweight device lies immense processing power, a multitude of connectivity options and enough sensors to make privacy nuts go wild with lawsuits. To give you an idea of just how jam packed the Galaxy S2 is here’s a breakdown of the specifications:

  • Compatible networks: GSM/GPRS/EDGE: 850, 900, 1800, and 1900; MHz UMTS: 850, 900, 1700, 1900, and 2100; MHz HSPA+: 21 Mbit/s;  HSUPA: 5.76 Mbit/s
  • CPU: 1.2 GHz dual-core ARM Cortex-A9 SoC processor; Samsung Exynos (GT-i9100)
  • GPU: ARM Mali-400 MP (GT-i9100)
  • Memory 1 GB RAM
  • Storage: 16 GB flash memory expandable with microSD (up to 32 GB)
  • Data inputs: Multi-touch touch screen, headset controls, proximity and ambient light sensors, 3-axis gyroscope, magnetometer, accelerometer, aGPS, and stereo FM-radio
  • Rear camera: 8 Mpx with auto focus, 1080p HD video, LED flash; Front camera: 2 Mpx for video chatting, video recording (VGA)
  • Connectivity: 3.5 mm TRRS; Wi-Fi (802.11a/b/g/n); Wi-Fi Direct; Bluetooth 3.0; Micro USB 2.0; Near field communication (NFC); DLNA; MHL; HDMI; USB Host (OTG) 2.0
  • Screen: 800×480 px at 218 ppi WVGA Super AMOLED Plus
It’s this very list that sold me on the phone initially. In terms of computing power the Samsung Galaxy S2 is roughly equivalent to a full PC I owned just 8 years ago, except that this one fits in my pocket and can also make calls. The in built storage is extremely generous, even if 5GB of it isn’t really available to you. What really impressed me was the inclusion of an 8 megapixel camera that was also capable of recording in full 1080p HD, something I had only just recently been able to do after shelling out $400 on my Sony DSC-HX5V. There’s also a pretty decent camera on the front of it too, although I’ve yet to use that for anything. You might then be wondering how the picture and image quality stacks up against my little pocket camera, well I did a quick comparison video below showcasing both of them side by side:

As you can see it actually stands up quite well when compared to my Sony. The video and picture quality is very comparable, especially in well lit situations. However it does fall down in low light and any time there’s motion due to the smaller CMOS sensor and lack of image stabilization. The LED flash on it is also incredibly harsh and will likely wash out any low light photo you attempt to take with it, but it does make for a decent little flash light. It won’t outright replace my little pocket cam any time soon but it’s definitely a good stand in when I don’t have (or don’t want to carry) it with me.

The everyday usability of the Galaxy S2 is also quite good for someone like me who has large hands (…ladies 😉 and used to struggle somewhat with the smaller screens on other handsets. However one gripe I do have with the handset is the lack of physical buttons for the options and back buttons for Android. The Galaxy S2 opts instead for 2 capacitive buttons either side of a the physical home button which does give the device a much sleeker look but can also mean accidental button touches should you brush against them. Samsung has also opted to put the power button on the side of the handset instead of the traditional placement on top near the headset port, which takes a little getting used to but is quite usable.

Where the stock Galaxy S2 falls down however is in its battery life. With moderate usage the battery wouldn’t make it through a second day requiring me to keep it plugged in most days whilst I was work lest it die on me overnight when I went home. This could have been the deal breaker for this phone as whilst I’m not the forgetful type I do like to be confident that I can make it through the day without having to watch the battery meter like a hawk. Thankfully the guys over at XDA Developers came to the rescue again with their custom ROM for the Galaxy S2 called VillainROM. After going through the process of doing the upgrade my battery now lasts about twice as long as it used to, only needing charging once or twice a week. I’ve yet to run Advanced Task Killer to attempt to squeeze even more battery life out of my handset, but it’s good enough for the time being.

It should come at no surprise then that this has been a wildly popular handset with both the tech and non-tech crowd a like. In the 3 months since its release the Galaxy S2 has sold a whopping 6 million units and just anecdotally it seems nearly every single one of my friends who was looking for a new phone has got one as well as almost half of my workmates. I used to laugh at anyone who touted any smartphone as an iPhone killer but with the Galaxy S2 not even being available in the USA yet and already garnering such a massive reception it might be the very first single phone that will be able to come close to touching Apple’s numbers. Of course I don’t believe for a second that any single Android handset will be able to take down the iPhone, not for a while at least.

The Samsung Galaxy S2 has set the bar as to what smart phones should be capable of and it will be the gold standard with which all are compared to for a long time coming. The combination of elegant design, incredible power and features galore make the Galaxy S2 stand out from the crowd in a big way, so much so that buying any other handset seems illogical. For many it has the potential to replace several other devices with its top notch multimedia components, further improving the overall value that you can derive from this handset. Overall the Samsung Galaxy S2 is a wonderfully impressive device and if you’re in the market for a new smart phone I really can’t recommend it enough.

The SEO Line I Won’t Cross.

 One of the first things you’ll come across after starting a website is the wonderful world of search engine optimization. In essence it’s the idea of making your website more appealing to search engines by following certain sets of guidelines in the hopes of getting your site higher up in the search ranks. Whilst I won’t go as far to call it black magic (search engines are by definition deterministic) there’s still enough mystery about how search engines work for the snake oil peddlers to work their craft in this field. If you see any of those “Mum makes $70/hour online! Find out how today.” type of ads on websites I can almost guarantee they’re some kind of SEO based idea that more than likely ends up being a scam or fails to deliver on any one of their promises.

This is not to say that I don’t employ SEO techniques on this site though, far from it. I run a couple WordPress plugins to make my site easier for search engines to crawl so that my posts appear in Google no more than 5 minutes after they’ve been posted. Additionally many of my articles have been written in such a way as to ensure that they’re more favourable to certain search terms. Indeed I’m guilty of writing articles specifically for people who land on my blog with certain search terms, mostly because I know how frustrating it can be trying to find something so simple yet be lead up the garden path repeatedly by countless blogs.

Those SEO experts amongst you would also point out that this blog is not the only one running on The Refined Geek domain, in fact there’s 26 more of them! Why would anyone have these if not in an attempt to try and boost the primary site search listings? Well I can’t deny that they’re part of an experiment I started over a year ago to see how one particular SEO technique faired, mostly to see if a blog that’s automated (read: none of the articles on them are written by me) could garner a higher readership than one from a genuine person. The result was mixed as for the longest time none of them attracted more than a casual passer by but at the beginning of the year at least 4 of them soared past this blog in terms of readership. More recently however this blog has overtaken them yet again and whilst no hard evidence of the cause (the experiment has been woefully unscientific) I believe it’s because Google has figured out how to track down blogs of this nature and is beginning to punish them in the search results.

However whilst I might be doing quite a bit of SEO based work there are some techniques I just can’t make myself do. Take for instance Tim Ferriss’ (of Four Hour Work Week fame) guide to writing titles for articles that will get you retweeted:

Into trapeze or German techno? Our starting headlines might be “How to Perform 5 Tricks on the Flying Trapeze” or “German Techno 101.” That’s just a starting point. Then we expand to what your wider circle of friends or co-workers might be interested in. For example:

“How German Techno Can Make You a Better Agile Programmer”
“5 Principles of Flying Trapeze for Better Hiring Decisions”

See how that works? This recipe works, and it’s a plug-and-play format for getting started, and getting traffic.

Once you’ve had a bit of practice, it’s oftentimes easier — and more scalable — to imitate what works elsewhere.

In essence this one of the more classic SEO techniques of putting the search term you’re targetting at the start or early in the title of the blog post. I’m guilty of doing this too, most notably with my most visited posts in the forms of game reviews. However those kinds of titles actually suit the articles that follow them since they first tell you what I’m going to be talking about usually followed by a somewhat whimsical statement that reflects my overall feelings about the game¹. Ferriss’ idea of creating these kinds of titles for posts, whilst nothing new to the SEOs out there, still managed to rub me the wrong way.

I won’t lie that getting people to read your blog is the main reason why a lot of us bloggers do what we do. For many of us the main source of readers are search engines so it makes sense to try and sculpt your posts in a way that will push them up the rankings. For this site the breakdown is about 41% from search engines, 47% from referrals and the remaining 12% coming from bookmarks or typing the address into the bar. Still writing just for the purposes of running up the search rankings never feels quite right as the titles always feel sub-par and don’t match my style. I will admit that they are quite effective but that doesn’t stop it from feeling like selling out.

I guess my feelings stem from the idea that the title is just that, barely even a begging to the actual content. An article or post should really stand on its own with its success being determined by the content not by the way in which its title was crafted. Unfortunately while search engines continue to value titles over content this kind of behaviour will continue. Sure you can still be creative within the bounds of certain rules but for myself it still just doesn’t sit right and I’ll continue writing just the way I’ve always done.

¹I often wonder if anyone picked up on this as no one ever commented on them. People have commented on some of my other post titles though (usually when I’m trolling). I’ll bet you’ll notice now! 😛