Fantastic Tests and Where to Use Them
As a QA tester myself, I know some of us aren’t that familiar with different programming languages or the different principles of UX design. I’m also acutely aware that there are some developers who are not familiar with the different types of QA tests available, and what they are used for. So in this piece, I will be walking you through five different QA tests that we use at Mighty Bear, and how they benefit everyone!
We shall start things out with the most basic of QA tests: the Boot Test. This test is so basic that it’s actually automated in most companies! A boot test is required when developers want to know if the product can be installed and launched. More requirements can be added, such as entering a world or playing a multiplayer mode, but that’s up to the product owners and QA teams to decide on together.
I usually timebox a boot test to last for 30 minutes at most, inclusive of everything from installation to end of execution. A boot test can also be considered a part of (or a prerequisite to) a smoke test, which will be explained in the next paragraph.
The Smoke Test is the most common and most executed of all QA test types, because it is a necessity in any continuous integration continuous delivery (CICD) environment. A smoke test ensures that a product is functional at its most basic level, covering most, if not the entire scope of the game, but only as a vertical slice benchmark. Essentially it is a green light for others to obtain the build for further testing! Debug is usually (and strongly encouraged to be) a part of smoke test execution, as developers rely on the outcome of this test the most if they need to continue work on a product build. A good timebox for a smoke test is usually one hour, though if the scope requires and time permits, two hours is okay to ensure more features are covered.
Next on the list is the Validation Test, also known as the Acceptance Test. This test is essential for most features going into a product as it ensures the features operate properly and meet the needs of the product’s end users. In other words, testers use the validation test to ensure that a feature is behaving as intended. The duration of this test execution depends greatly on the scope of the feature, so it could range from 30 minutes to 1 week, or even longer! Debug is not usually used to validate these intentions unless debugging tools are needed as part of validation of the feature, especially if it’s a big feature like seasonal events or weapons damage levelling, which could take forever to test if there’s no debug to assist testers.
Our second last item on the list is called the Functional Test, or Functionality Test to some. To put it simply, this test is fundamentally a smoke test, but without the vertical slice requirement. It’s testing EVERYTHING in the product, with or without debug, to ensure that all things are functional (thus its name). Similar to above, the duration of this test depends greatly on the size of the product and its contents. It could take 1 to 2 weeks on average for a team of 10 testers, working uninterrupted, to complete a functional test!
And finally, we have the Compliance Test. This could arguably be the most important test out of this list because without passing this, the product might not even see the light of day (and by that I mean reaching the consumers)!
This test is the key to ensure that the product ends up where it should be: in the hands of end users. It starts with having a very in-depth documentation of a publishing platform’s list of rules and needs of how the product is intended to work. The development team then converts this list into a set of testable and non-testable cases. The testable cases go into testing, while the non-testable cases get disseminated to different departments, who then certify that the documentation for everything is accurate.
Any failure within a build could result in a rejection from the platform, and that would waste days, or even weeks of development time to fix and resubmit. Therefore, passing the Compliance Test will ensure that the product goes live on the platform as intended, without any additional resubmissions, and that is always a big win!
And that’s it! Five different types of tests that we use at Mighty Bear Games to guarantee that players’ experience with our games are as smooth as possible. The tests mentioned can also be summarised into a simple table, seen below (note that this is not a hard definition; it depends on your company and scope).
Of course, this list is not exhaustive; there are more types of tests we also utilise for our products. There’ll be a next part to this article, so keep an eye out for it!
If you’ve enjoyed reading about our QA tests at Mighty Bear, drop me some claps, leave me a comment, or follow Mighty Bear Games on Medium!
Fantastic Tests and Where to Use Them was originally published in Mighty Bear Games on Medium, where people are continuing the conversation by highlighting and responding to this story.
