One of the principle concepts behind our development process is referred to as the SVA method. SVA stands for "simulation, verification and automation". This method is a process that plays out in three stages:
Simulation
The first step in using the SVA method is to build a "Monte Carlo" simulation of the project in a framework that permits automation. Microsoft C#, Oracle Java or Mozilla Javascript are examples of tools that have been used previously for the simulation process. The game rules, the game materials and basic player decision-making are simulated in a faithful single-iteration test of the actual game experience. This can be done without investment in any art or visual elements, allowing prototyping to be performed in a meaningful fashion without having to struggle with the challenges of creating a functional model first. There is no replacement for real human play testing but the benefits of SVA are difficult to match reliably with other methods. Manual play-testing and SVA, one novel and one traditional, can work together to make games better than they have been with just human play testing alone.
Verification
During the development process, there needs to be logging, and lots of it. The key purpose of this step is to verify that the behavior of abstract code matches our real-life expectations. The simulation will be designed as a single iteration but structured discretely to allow an automation structure to be built around it. Exhaustive logging around that experience is crucial. This step should be approached with care, as the eventual goal of automation is meaningless if are collecting and aggregating data for the wrong process. Logging of this nature is usually somewhat burdensome for some frameworks so development should include design choices that make disabling logging possible.
Automation
The final step of this process is to get the roller coaster rolling: automation! We will choose an arbitrary threshold for how many game sessions to run in a testing session (usually based on duration of testing, a function of programming efficiency and available computing power) and then automate the process programatically. It is important to incorporate software design elements that collect and organize raw data regarding key metrics for your project. The frequency with which asymmetrical elements produce certain outcomes, how often players are winning using certain strategies, thresholds for reaching certain states for game play change. . .these are just some examples of the sort of crucial data points that can be aggregated and observed.
Benefits
Air-Tight Rules System
Frameworks for writing computer programs are truly unforgiving in their analysis of your code procedure. Especially with re-iterations of the magnitude described here, if a possible course in the logic of your rules set is indeterminate or unhandled, this process will 100% sniff that dead end out and call it to your attention. If the rules you have written are so robust that you can base a computer program on them, that gives you a strong measure of confidence in design.
Balance Beyond Measure
There is no substitute for human player interaction with a game project. However, balance is not always entirely about player choices or the feels around the experience itself. With this method, we can see systems of chance and randomization play out on a macro scale and observe the probabilities without having to "know the math" and perform interminable lists of statistical equations to model the math behind it. Using the SVA process, we can simply dream up methods for building probabilities, model them in a simulation, and run that simulator for testing. Once we can see the shape of the data, we make changes to our methods and story-telling accordingly. To do so manually would take years in most cases, but automation can give this to you in months or weeks or even days if the project is simple enough.
Solid Foundation for Growth
If properly designed, a simulation can be a basis for future work expanding on your game system. If the simulation is designed in a fashion that has data sources representing the game materials that are discrete and decoupled from the code that handles player decision-making and rules, the simulation can be used as a basis for building and balancing future game concepts for expansions. In other words, you have a machine for making more content, and for matching the performance against existing game elements!
Costs
This ain't easy. You have to design and write a computer program. So it is work intensive and frankly, requires a bit of experience. However, the results are absolutely worth it. In fact, this method arguably provides opportunities to build with probability structures and methods that are otherwise infeasible to work with. A whole new way to do game design.