Coverage-Based Playtesting

OK, following up from last post, here's another approach to playtesting that borrows from software testing.

Coverage-Based Software Testing

In Software Engineering, Coverage-Based testing means running your tests and then, usually with a specialized tool, seeing what parts of your program the tests "covered" and which were left-out of the test. It's not a perfect technique: you might "cover" a piece of code and still miss the bug that is there. However, if you didn't cover a piece of code, then you definitely won't find the bug there. So it's one of the quicker methods to find holes in your testing, albeit an imperfect one.

Coverage-Based Playtesting

Get a game together and run a playtest on your game artefact, be it a game system or module. Now, when the playtest is over, ask yourself what parts of the game did your playtest "cover". The simplest way to do this is to skim through the text and circle any rooms, monsters, magic items, etc. that didn't get used.

Okay, so you've got your list of "holes". What do you do with them?

The Brute Force Approach

The ideal option is to continue running playtest after playtest until most or all of the holes are covered. But that can take a lot of time, especially since, along with the holes, you will no doubt be re-hashing a lot of the same covered material as in previous games.

Filter

For one, you can decide which holes are really a problem and which aren't.  Empty rooms might not be so important to playtest, for instance. You might also filter them based on what concerns you have for this artefact.

The Parallel Universe Approach

Now that your group finished their playtest tell them "OK, now let's go back in time to when you went right at that passage and instead say you decided to go left". Continue to reset them to past locations until most or all of the holes are covered.

The Proofreading Approach

You're not going to get around to covering everything via actual play. Instead, give the holes an extra round of proofreading, to make up for them not being covered in the playtest.