Saturday, September 22, 2012

How to Judge an MMO

I have had very limited time to play/write recently.  And when I do get those small breaks, I almost always choose to play versus writing.  So if any of you are wondering for the large lulls recently, I am to blame, and I apologize for that.

That being said, I wanted to take a moment to analyze how we judge an MMO. Afterall MMOs, cannot be judged like books, movies, are even other video games.  MMOs are normally designed to be played over several years and are constantly being updated. To make the matters worse, they are far more complicated.  They have varying arrays of large scale systems that imply intricate virtual worlds.  How do you judge an MMO, a style of game, that is so vast, so self-perpetuating, and so incredibly complex?

Everywhere you look people have a different preference.  IGN, Metacritic, MMORPG, or just your daily forum posters.  With Guild Wars 2 having such a massive lead-off of hype prior to it's release, millions of people have been forming opinions of the game.

The problem really begins when those people start trying to write up their opinions as wide-spread fact and try to review the game.  I actually made a similar mistake.  I tried to write up a lengthy review of all of the major aspects of a game, assigning numerical values to each, and coming up with a total score.

The first and most obvious question is: When is it fair to write the review?  At what time point (since beginning playing) can I write knowledgeably about an MMO?

I will largely ignore this first question.  The answer is too subjective for each reader to decide. Some suggest with in a few days, other a few weeks, and some even a few years.  The lines have been drawn depending on the person, and there's not much you can do to change someone's mind on the issue.

I'm of the belief that you should try out every game mode possible, and have substantial experience with each to know the major aspects of the game.  I also normally believe you should reach level cap before being fully capable of writing a lengthy review.

But then the second problem becomes:  How do I actually judge an MMO?  In small categories that sum up to be the review of the entire game?  And if so, what and how many categories?

My first attempt was to base it on the actual pieces of the game:  World, Dungeons, sPvP, WvW, Personal Story, Combat, etc.  But then that brings up the problem of comparison between games.  Many reviewers use numerical scores, and those scores lose significant value if they are incomparable to multiple games.

So then the second attempt was:  Visuals, PvP, Story, Combat, World.  But then the question became, should I include Customer Support, UI, game instability, or should I incorporate overall value into the score?  Afterall a B2P game is almost always significantly cheaper than a subscription based game.

However, let's continue playing this game and say I chose my categories, and we all relatively agreed on them.  Then even more problems arise.  Now more than ever, games can change relatively fast, particularly in MMO space.  With significant patches or expansions, do you need to re-review the game?  If so, when?  And how do you fairly compare two games that came out at different times?  Is it fair to judge WoW-dated graphics to Guild Wars 2 graphics simply because both are releasing some form of content at similar times? And then should you form numerical values for the sub-components of the game, and should they be judged equally?  If you do, what scale should the values be on, and do you include tenths of a point, or only halves?
If your scoring system changes over time specifically on a new release, are you obligated to change your past scores?

I will say this for now.  I do not envy anyone who tries to review a modern day MMO.  No matter what your system is, there will be substantial flaws in your model.


  1. im sure none of my fellow readers hold the slightly reduced content against you mate, we all know the constraints RL can place on these things and we all are gratefull for the articles you do post, its a damn fine highly informative site, one of the best i found on GW2 id say(why i check it every day).
    your right, game reviews is a difficult thing to judge, ive found among myself and my friends, that we no longer pay as much attention to the actual scores on sites such as ign/gamesradar, increasing complexities in games has rendred such scores meaningless is what i hear a lot, now its more reading the review for a idea of the games structure/design, for example a friend refuses to play a game with a monthly fee, regardless of its score, and i would never play a play-to-win game regardless of how good it actually is, for many ive spoken to the fact of GW2's no monthly fee (while still being a full MMO)is something that on a score system jumps it up a few places, nothing to do with the game itself technically but still impacts overall buying decision.

  2. Agreed.......I pre-ordered, played some beta and stress tests and have been taking my time on 5 different toons (one of which is L50). I like dropping in here weekly to see how things are progressing and your thoughts on the game. I like to think of reviews of MMOs much like I do the MMO themselves, ever evolving ever changing......and hopefully ever improving.

    I think there is always a first impressions review (whether that can be considered a true review or not) within the first few days of playing. Looks, feel, flow, toon creation, graphics, bugs, crafting, pvp, nuances, etc.....and even those can develop and change over weeks and months. Within 4-6 months we should have a good base to develop a high end overview of the game but even then it is subjective and doesn't necessarily dive deep down into the each individual players nuts and bolts.

    I am having a blast playing GW2 and have dabbled in about everything it has to offer within level of course (jumping puzzles, 5v5, WvW, crafting, PVE, guilds, TP, exploration, dailies, etc........) but even there it has merely been dabbling and there is so much depth that I look forward to personally evaluating the game from my viewpoint 8 months from now and all the reviewers as well. Thanks.

  3. This comment has been removed by the author.