Of Castles and Consoles

All things considered, the landscape in Scotland is not drastically different from the scenery I'm used to seeing in upstate New York. The most striking contrasts actually stem not from natural, but from architectural differences - most notably in the case of the various castles which adorn the countryside.

I recently had a chance to get a closer look at one of these awesome structures when I attended a bonfire at Balintore Castle. Standing in the glow of the firelight, gazing up at the castle's high towers, I felt the distinct pang - no, not the fun kind - that serves as an acute reminder that I am currently living in an unfamiliar country.

Adjusting to life in Scotland has occasionally been challenging, but the day-to-day hurdles are infinitesimal in comparison to the monumental ordeal of getting here in the first place. What should have been the relatively straightforward task of obtaining a visa morphed into an arduous, months-long battle of attrition characterized by more migraine-inducing, arbitrary setbacks than I could have ever fathomed in even my most pessimistic contingencies.

Reflecting on these visa struggles during that cold night at Balintore, it occurred to me that the only other comparably vexing experience that I had ever previously endured also resulted from an eerily similar approval process, one that most commercial game developers know all too well: namely, that oft-maligned mug's game known as console certification.

The more disturbing realization, however, was that in comparing my visa approval and console certification experiences in hindsight, I honestly don't know which was more frustrating.

As with all bureaucratic systems, both visa approval and 'cert' deal in obfuscation and red tape as their primary stock in trade. Functioning bureaucracy, like software development, relies on hallmarks like modularity and indirection (some would argue misdirection) in order to manage large influxes of requests and exert tight control over information flow. Unfortunately, these same properties can easily malfunction and lead to nightmare scenarios where lack of accountability and rigid adherence to protocol transforms the bureaucratic system into a mindless hydra that utterly demoralizes anyone trying to obtain something as simple as, for instance, a minor clarification.

In the case of cert, one needn't search far to uncover numerous tales of well-intentioned developers who became disheartened while wrangling with the inherent vagaries that must be surmounted before that ever-elusive ratification becomes a reality. Unsurprisingly, the adverse effects are felt most severely by independent developers - people like Jeff Minter, Jonathan Blow, and Jamie Fristrom; incidentally, the same folks who are producing some of the most unique and intriguing (if not always the top selling) output on consoles today.

Before I proceed, a few clarifications. Although I am being critical of the certification process, it is in the interest of reform rather than abolition. I have mixed feelings about the conflicting motivations that shaped the evolution of the modern cert process (further elaboration on this topic to follow in an upcoming post), but I don't think any game developer will dispute the importance of cert's ostensible purpose: namely, ensuring that all releases on a particular console are as stable and user-friendly as possible.

Nor is this discussion intended as an objective, definitive examination of all relevant factors from every possible perspective. I do not have any experience working for a console manufacturer; I am a developer and a gamer, and I will be viewing the issue primarily through these lenses. If you're into that whole "coherent and balanced" thing, I recommend checking out Simon Carless' thoughtful dissection of the matter over at GameSetWatch. For my part, I'm more interested in how the approval process can be improved and streamlined to better facilitate independent developers, since their experiences often seem to contrast sharply with those of more established names.

Alright, back to Balintore:

Perhaps due to the surroundings, the first game that popped into my mind with regard to the certification process was Castle Crashers. I am a big fan of The Behemoth's previous effort, despite the fact that (as I stated in my review of the game) it did contain an occasional bug or two. As a gamer, I did not consider these a hindrance to my enjoyment of the title, instead chalking it up as a natural side effect of the game's outsized ambition relative to its compact development team and comparatively meager budget.

When Castle Crashers was first announced I was thrilled, but part of me remained slightly apprehensive - here was a game that not only promised to be much grander in scope than Alien Hominid, but that also intended to dive headfirst into the certification abyss known colloquially as 'network play'.

The primary mitigating factor for these concerns was that Microsoft appeared to have a reasonably solid testing infrastructure in place for networked games. This is, of course, especially crucial on consoles, where procedures such as public beta tests and patching (pillars of online game development on the PC) carry a greater associated overhead and can be cumbersome to mobilize. Surely the XBLA certification process, with all its attendant rigors and promises of localization and testing assistance, would serve to expose a majority of the potential issues prior to the game's release.

Alas, when the fateful date finally arrived it quickly became clear that this was not the case.

These problems are currently being addressed, but it remains to be seen whether the resultant blight will have a detrimental effect on the game's overall sales. Regardless of the financial impact, it is clear that the stigma now associated with Castle Crashers in the collective consciousness of gamers will continue to haunt The Behemoth for some time.

While it's difficult to surmise exactly what lead to this unfortunate circumstance (much to the chagrin of the aforementioned gamer hivemind, which typically yearns for easy scapegoats), I believe the results provide a potent indication of how the certification process has gone awry, simultaneously failing both gamers and developers. That's right - in the same sentence where I decry the hunt for scapegoats, I'm turning around and blaming cert. The gamer in me will not be suppressed!

This is where much of the antipathy indie developers have expressed toward cert seems to arise. Developers will gladly agree to wade through even the most egregious morass of onerous requirements and burdensome mandates, but only if this effort will ultimately yield a corresponding benefit for their final product.

To hear the console manufacturers tell the tale, this is simply a problem of perception on the part of naive developers who underestimate the importance of factors like intuitive gameplay and stability. But how, then, does a game with such a heavy emphasis on online play manage to slip through the certification system with such glaring networking flaws intact? At the very least, the story of Castle Crashers would seem to indicate that the accusations from indie creators regarding cert's misplaced focus are not entirely unfounded.

It doesn't need to be this way. There are plenty of feasible measures that can be taken in order to improve the certification experience - not just on the part of the console manufacturers, but developers as well.

As developers, particularly those of us making networked games, it is our responsibility to devise an adequate test plan that can scale appropriately within the scope and budget of our projects. If your budget doesn't include support for testing, it's time to consider either reducing the scope, as Jamie Fristrom pragmatically suggests in his Schizoid postmortem, or even - gasp! - signing a limited agreement with a publisher, an alternative that Simon Carless has previously proposed (albeit hesitantly). In any case, relying primarily on cert for testing is clearly a recipe for disaster.

At the same time, game creators must also strive to eschew the mentality that the initial cert submission is the end of the development process. No matter how anxious we are to reach the finish line, pausing for a self-congratulatory high five before actually clearing the final hurdle doesn't help matters, and in fact only serves to heighten our fans' (already considerable) impatience. As my own struggles with the visa approval process taught me, bureaucracy is never beholden to our desired timeframes.

Ultimately, however, individual developers can only change their own approaches to certification; widespread reform of the cert process itself lies chiefly within the domain of the console manufacturers. To that end, improved transparency is essential, alongside expanded partnerships with external QA providers that can help fill in the gaps in testing that are currently allowing major issues to slip under the radar. Alternatively, console manufacturers that aren't able to provide an adequate safety net should utilize other methods to facilitate smaller developers and budgets, either by providing more flexible business arrangements (don't hold your breath) or lowering the priority of cert requirements related to feature additions in order to allow indie developers to focus on stability first.

While necessary, these steps are, in fact, only a prelude to what I believe is a more fundamental shift that console manufacturers must embrace in the near future, namely the impending transition from "gatekeeper" to "community conduit" - more on that in a future entry.