Friday, May 23, 2008

Step Three: List Test Case Names

After you have outlined your test suite, this step becomes much easier to do well. Having an organized system test suite makes it easier to list test cases because the task is broken down into many small, specific subtasks.

Put your finger, or cursor, on each list item or grid cell in your test suite. Then, for each one, ask yourself about the relevant system requirements. If you have a written use case document, you will often be able to turn each use case into one or more test cases. There may be some list items or grid cells that really should be empty. For example, an e-commerce application might not have any delete operation for the Customer Order business object. Explicitly mark with "N/A" any cells that logically should not have test cases. If you cannot think of any test cases for a part of the suite that logically should have some test cases, explicitly mark it as "TODO".

The name of each test case should be a short phrase describing a general test situation. Append a unique number to each test for the given test situation. For example: login-1, login-2, login-3 for three alternative ways to test logging in. And, sales-tax-in-state-1 and sales-tax-out-of-state-1 for two different situations where collected sales taxes are reported to the government according to two different procedures. Use distinct test cases when different steps will be needed to test each situation. One test case can be used when the steps are the same and different input values are needed.

As you gradually fill in the test suite outline, you may think of features or use cases that should be in the software requirements specification (SRS), but are not there yet. Quickly note any missing requirements in the SRS document as you go along.

Before moving on to the next step, it is worth highlighting the value of having a fairly complete test suite outline. The test suite outline is a useful asset that can help your project succeed. At this point, you can already get a better feeling for the scope of the testing effort. You can already roughly prioritize test cases. You are already starting to look at your requirements critically and you may have identified missing or unclear requirements. And, you can already estimate the level of specification-based test coverage that you will achieve.

Step Two: Outline the Test Suite

Once you have prioritized your QA goals, it is time to outline the system test suite. A test suite document is an organized table of contents for your test cases: it simply lists the names of all test cases that you intend to write. The suite can be organized in several ways. For example, you can list all the system components, and then list test cases under each. Or, you could list major product features, and then list test cases for each of those.

One of the best test suite organizations is to use a grid where the rows are types of business objects and the columns are types of operations. Each cell in the grid lists test cases that test one type of operation on one type of object. For example, in an e-commerce system, a Product business object would have test cases for each of the following operations: adding a product to the system, listing or browsing products, editing products, deleting products, searching products, and calculating values related to the product such as shipping cost or days-until-shipment. The next row an e-commerce test suite grid might focus on the Customer Order business object and have test cases for almost all the same operations.

The advantage of using an organized list or grid is that it gives you the big picture, and it helps you put your finger on any area that needs more work. For example, in the e-commerce grid, there might be a business object "Coupon." It is obvious that shoppers use coupons, but it is easy to forget to test the ability for administrators to create coupons. If it is overlooked, there will be a clearly visible blank space in the test suite document. These clear indications of missing test cases allow you to improve the test suite sooner, make more realistic estimates of testing time needed, and find more defects. These advantages allow the found defects to be fixed sooner and help keep management expectations in sync with reality, which helps keep the project out of crisis-management-mode.