The Strategy of Playtesting

Hey all, just a short update on recent goings-on, plus a special a peek behind the curtain regarding how developers decide when to pull back said curtain. I know many of you are very eager to get your hands on the game, and we’ll be kicking the alpha off in a few weeks. So what have we been up to, and why the wait?

AI Status Report

I’m in the process of finishing up the first batch of key AI functionality. The basic framework for all AI systems (except for diplomacy) is now in. The AI knows how to explore, account for supply, train new units, defend itself, and survey/improve resource deposits. The last two items on my list are teaching the AI how to take out nearby hostile camps, and migrating once an area’s resources start running out. At that point they’ll be at least semi-competent at the early game, and – officially – able to play the game.

On a related front, Jonathan is almost done untangling the complex web of pain that is pathfinding, a feature needed by the AI and for the issuing of multi-turn move orders. Among many other benefits, I’m particularly excited about the ability to see which tiles you’re able to move to during the turn. Blind trial and error is not most people’s preferred method of learning!

Once these tasks are wrapped up we’ll be doing a week of internal playtesting and tweaking to ensure the game can be played for at least a few minutes before devolving into a charred mass of fail.

Okay, sure, the game isn’t done yet – this is neither news nor a concern if you’re a member of our dedicated initial test group. After all, if the game were ready then it wouldn’t be an alpha test! So what’s the rationale behind holding off for another few weeks?


Deciding When to Start Playtesting

First impressions are incredibly valuable. In fact, probably the most important feedback a developer can get. They offer a nearly-unbiased metric for what’s actually working and what needs the most attention. It’s tempting to shrug off early complaints and say “that’s no big deal, we’ll fix it later.” But sometimes “later” never comes. And even if it does, that seemingly minor problem might have been masking one far more insidious. And now that you only have a month left it’s far too late to do anything about it. Woops!

Veteran playtesters are crucial in a variety of other ways, but after playing for a while you become blind to certain high-level flaws that newbies spot easily. Your stock of these bullets is limited, and it’s unwise to start firing them at an early experience you know to be completely unrepresentative of your final vision. This is true with any game, but let’s take a look at a couple tangible examples from AtG.

The supply feature has yet to be tested rigorously and it’s pretty much guaranteed the amount provided by each terrain, consumed by each Unit, the effect radius of Supply Camps and so on are all completely out of whack. If we started testing right now most players would find the system completely brutal and (gulp) unfun. But is this because our design vision for supply is completely broken? Or is it simply the result of a few numbers slightly askew? This determination is hard under ideal circumstances, and at this early stage you honestly have no clue until you’re sure reality matches up at least roughly with the idealized design that lives in your head.

Exploration might be an even more poignant illustration of this. Uncovering a shrouded map is fun because you’re constantly discovering something new – but after you’ve seen it all that excitement diminishes somewhat. We’ve invested heavily in making this aspect of AtG very strong, but does it really work? Until the AI is playing the same game as the human there’s no way you can accurately gauge this. If you’re able to grab all of the goodies you’ll develop a highly skewed opinion of the reward pacing. And unlike supply, once you’ve seen the surprises there’s no going back. Sure, exploration is still fun the hundredth time you play, but in a very different way. That’s not the kind of blind spot you want to just shrug off.

Playtesting too early isn’t even the biggest risk, as it’s much easier – and far more dangerous – to pursue the opposite extreme. Everyone is sensitive to criticism and there’s always a voice in the back of your head encouraging you to spend too much time polishing your work before opening it up to the world. The problem is that the more you invest the more you waste when you change gears. And trust me, change gears you will. A lot. In the best-case scenario you’ve wasted a lot of time, and in the worst your game is far worse than it could – and should – have been.

Developers have a tough balancing act in deciding when to reveal their game to the outside world. There’s no formula that works in all situations, but there’s no doubt in my mind that it’s better to err on the side of early.

And lean that direction we shall. Just… not quite yet! Thanks again for your patience and support!

– Jon

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

search previous next tag category expand menu location phone mail time cart zoom edit close