For each test performed, clear and unambiguous acceptance criteria needs to be defined.

This can take multiple forms.

Provide the Range

If a quantitative value is defined, if there is any tolerance, be sure to provide the range. Ideally, this is defined in requirements but if not, needs to be defined in the protocol (remember that appropriate personnel review and approve the protocol so such definition is acceptable but, again, should be considered a ‘last resort’ for specifications).

In and Out of Scope

In some cases, a screen shot that the tester can compare against can be included in the protocol. Be sure to specify any portions which are in or out of scope in case there is, for example, time or date displayed (which would fail a bit-by-bit comparison).

Objective Evidence

The above example relies on the tester to make a pass/fail judgment based on what he or she sees. An alternate approach might be to have the tester take screen shots to provide objective evidence of passing steps 4 and Judicious use of screen shots provides good supporting data.

Too Many Screenshots

Gratuitous use of screen shots, though, can be a real headache. Each screen shot needs to be properly annotated to link the results to the test. It’s easy to overdo screen shots so be aware of the overhead. It’s recommended to give the user explicit instructions when screen shots are called for. An example of how you could do that is shown on the screen:

Second Source

It’s easy to tell that the values are “properly added” given these are just integer values. If these were floating point values or if the computation was more complex (e.g., square roots), a second source of verification may be warranted. Should a second source be required, it would constitute “test equipment” and will be discussed below.

Objective Evidence

This might be a better spot for a screen shot. It’s not uncommon for testers to see what they want to see. After a full day of grueling testing, it’s easy to not always pay close attention. Recording the result as in step 7 is helpful and is a readily-accepted method for recording objective evidence.