In a talk at GDC Canada on Friday, Blue Castle Games' Tom Niwinski and Dee Jay Randall discussed the telemetry behind Dead Rising 2's zombie killing theatrics, focusing on remote reporting to help hone Capcom's awaited sequel.
Randall, introducing the tech behind the talk, explained that Dead Rising 2 is using remote telemetry from people playing the game in Blue Castle or Capcom-affiliated cities/events, including the UK, Italy, Japan, Hawaii, Vancouver, California, and Blue Castle's main QA location in Montreal.
The game simply uses HTTP GETs to pass a multitude of information over the Internet as testers, producers and even journalists at the Captivate press event play through the game. The data includes in-game events, data on crashes, and a multitude of other things.
As a result, the detailed data -- which is only captured in the pre-release version of the game, and will be removed before the game ships later this summer -- is captured and visualized for the coders, artists and designers.
Of course, visualization is a major issue, so Niwinski explained some of the best ways to interpret information, from a design perspective. For example, the Dead Rising 2 creators look at simple text data, like how many times an item has been picked up or how long a particular build has been played through.
Niwinski says that stat-based outliers are important to look at, because they might be bugs in the game. They found instances where Chuck, the main Dead Rising 2 character, took 2 billion points of damage, and traced it to a bug with an animation state, even before any testers found and reproduced it.
Other methods Blue Castle uses to visualize include the 'SQL to chart' converting software Confluence, which does percentage charts, bar and line graphs, as well as custom in-house tools.
In fact, they've made an impressive-sounding tool called ChucksStash, which allows the Dead Rising 2 creators look at trends within a 3D visualization of the game world, overlaying multiple data sources. The creators can then use the data to decide whether weapons are placed in the right places, whether zombies died as a result -- and the design ramifications of that.
This means you end up getting information on what killed the zombies, where the user went, how long people played for, what survivors were rescued, and so on, which is pretty much standard for some high-end games, they note. So what unexpected things did they find using telemetry?
For one thing, if there's bad collision and the user falls through the floor, there's a safety floor underneath it, and it'll be captured by the telemetry. Then ChucksStash will group those areas together, and the programmers or designers can jump to those places and find the geometry error.
In addition, design decisions can be definitely resolved. For example, doors are difficult to close -- did the designers care? They looked and discovered that doors were opened almost half a million times during the play testing, but only closed 2,000 times, with no tester complaints -- so it appeared that people didn't really mind.
Soak testing for memory leak-related crashes is also easy with telemetry of this kind, since you can just leave the game running in a demo mode, go home, come back in the morning, and see if it crashed, checking out the stats via the telemetry.
The Blue Castle team has also set up bug reporting for their testers using a custom interface which adds game data such as world location, health, and the ability to attach screenshots. You can even see nearby bugs -- from a world location perspective -- before you submit yours.
Wrapping up, the Dead Rising 2 team really believed that telemetry was a "win" for them. And they warned on dispelling 'telemetry FUD'. For those thinking it would take too much memory, for example -- the telemetry only took up 65kb.
In addition, telemetry gathering doesn't take away from the main CPU thread. In terms of bandwidth, it takes less than 1kb per second, and discards data if overloaded -- so they definitely feel like the advantages outweigh the disadvantages.
|
