What should a test protocol contain? What is the typical content of a test protocol? Before any test execution takes place there are a number of checks you must do before entering into the world of test execution. “Fail to prepare, prepare to fail” no truer words can be spoken in relation to protocol execution.

How many times have you seen executed protocols with deviation after deviation or observation after observation?

Can this be prevented or do you have to have executed protocols with mark ups everywhere on the document. Its one thing to have a clean protocol with no deviations or mark ups (rarely happens and would raise suspicion with the FDA) but its quite the opposite to have a protocol littered with mark ups everywhere to the point where someone has to ask the question “Has this been given a dry run”.

The article below details some great tips before entering the dreaded execution step of your validation project.

Test Protocol – Are You Prepared for Testing?

What should a test protocol contain? What is the typical content of a test protocol?

Below is a checklist of typical headings that should be found in any test protocol.

  • Introduction
  • Purpose
  • Scope
  • Rational
  • Roles and Responsibilities
  • Team
  • Validation Responsibilities
  • Test Team Responsibilities
  • Location of Testing
  • Testing Requirements
  • Hardware Requirements
  • Software Requirements
  • Documentation Requirements
  • Test Execution
  • Order of Testing
  • Acceptance Criteria and Observed Results
  • Hand Written Data
  • Recording of Test Results
  • Corrections
  • Incident Recording and Management
  • Fault/Deviation Reporting
  • Observation Reporting
  • Outcome of Test Documentation
  • Change Control
  • Document Changes
  • Glossary
  • Definition
  • Abbreviations
  • Associated Documents
  • International Standards
  • International Guidelines

Test Specifications

The test specification contains the test instructions, test acceptance criteria and results for the items under test. It is very important to read the test specification carefully before entering into any test execution. Time spent dry running or becoming more familiar with exactly what you have to do will be time well spent.

An entire dry run (time permitting) should always be performed on any test script before official execution.

Design Specifications (Typically applies to Software Testing)

The design specification contains the design information for the module/application under test and is the document from which the Test Specification is written.

There is typically one design specification per Test Specification but occasionally there will be the situation where one Test Specification will refer to multiple design specifications.

Make sure that you have a copy of the required design specification(s) before commencing test. You need to record the revision of this document in the Test Specification before testing and you will need to refer to it during testing.

Related: Become a Protocol Execution Ninja!

Fault and Observation Forms

These forms are critical to any test process as they will record the details and resolutions for any incidents noted during testing. One form is completed for each incident noted and this is referenced in the test specification. Make sure that on the test protocol that any reference to a fault or observation form is clear, and they are included in the attachment listing section.

Test Report

This is the summary document that reports the findings of the Test Phase. The test report is an important document in the sense that an auditor should be able to see quite clearly from the report how successful the test phase was for the particular module/equipment/application in question.

Completing the Test Documentation

There is one simple rule in relation to test documentation:

If it’s not written down then it didn’t happen

Signature Log

Before commencing test, all testers, witnesses and reviewers are required to sign the Signature Log which is included in the test protocol, usually located at the back of the document.

Test Specification

Read the test specification carefully; understand exactly what you are testing. Check to ensure the pre-requisite steps make sense and are performed correctly before launching into test mode. Read the test step instructions carefully.

Test Results

Fill in the acceptance criteria sections correctly, if the acceptance criteria asks for a PASS/FAIL response then write down either PASS or FAIL no other acceptance criteria is allowed.

The tester must sign and date at the bottom of each page.

Witness must sign and date at the bottom of each page.

Completing the PASS/FAIL column

If the results match the expected result then mark as pass. If the results do not match the expected result then mark as fail and reference the corresponding observation/deviation number. If the test section does not apply to the test step then mark this section as N/A and give a brief explanation of why this particular step does not apply.

Attachment List

The following is a guideline detailing how to handle attachments associated with your test protocol.

Each attachment associated with the protocol must have the following information.

  • Test Reference
  • Test Section
  • Attachment Number
  • Page No _ of _
  • Initial and Date
  • Complete the table in the attachment list section of the test specification to list all Test Attachments

Result of Test Specifications

If all tests were passed without incident, then mark clearly on the protocol passed.

If any test failed and faults or observations were raised, state whether the faults or observations are closed or resolved, if resolved mark as pass.

Mark the overall test protocol as passed if all the faults and observations have been closed out, if not indicate that observations or deviations are still open.

Common Testing Mistakes Associated with Software Validation

Test Script Generation Errors

  • Writing the tests based on the system and not on the specifications
  • Inadequate security testing
  • Using the “Shotgun Pattern to Testing”
  • The test system doesn’t match the production system
  • Non-compliant ordering of tests
  • Not doing adequate pre-test setup
  • Not ensuring test step repeatability
  • Inadequate field verification
  • Poor selection of test data
  • Not testing the error handling capabilities
  • Not handling calculation verifications correctly
  • Not testing for known problems
  • Insufficient audit trail testing
  • Insufficient depth of testing
  • Insufficient stress testing
  • Writing test scripts for functions that are not used

Test Script Generation Errors

  • Not training your testers in good documentation practices
  • Assigning testers that just don’t care
  • Tester error as a result of unfamiliarity with the tests
  • Relying on inexperienced test script writers

Execution Mistakes

  • No pre-execution approval of tests
  • Poor organization
  • Mandatory order of execution not followed
  • Preprinting and replacing lost of damaged pages
  • Not explaining mistakes or errors thoroughly
  • Not allowing for ad-hoc testing
  • Not retaining the entire report on huge printouts
  • Not writing up a deviation
  • Deviation records not complete
  • Changing a FAIL to a PASS
  • Showing insufficient deviations to support comprehensive testing
  • Not closing out deviations
  • Not having adequate sign offs of deviations
  • Not creating a deviation log book
  • Individuals writing and executing the validation protocols are not identified or trained
  • Uncontrolled changes to approved test scripts
  • Not archiving off the validation database before and after validation
  • Original validation documentation can’t be located

Conclusion

I hope this article provides some useful tips on how to execute validation protocols successfully. Remember it’s all about pre-planning and making sure that you have the time to do a dry-run beforehand. Please use the comments box below to add any of your real life experiences when it comes to protocol execution.