Model Systems Engineering Documents for Adaptive Signal Control Technology (ASCT) Systems

F. VERIFICATION PLAN GUIDANCE

The chapters required for the Verification Plan are:

1. Purpose of Document
2. Scope of Project
3. Referenced Documents
4. Conducting Verification
5. Verification Identification

This verification plan describes the activity of verifying that the system being built satisfies all the requirements set out in the requirements documents. The verification documents will include:

• A plan to initially lay out the specific verification effort
• The verification plan that defines the detailed mapping of the requirements to verification cases
• A report on the results of the Verification activities

To ensure that all requirements are verified by this activity, trace each requirement into a verification case, then trace this in turn into a step in the Verification procedure.

The Verification Plan does not need to include verification procedures. These may be prepared by the vendor, but must be reviewed by the agency to ensure each verification case will be tested and appropriate results recorded. In relatively simple cases, both the Verification Plan and the procedures may be prepared by the vendor. In this situation the agency must ensure that each requirement is mapped to verification test.

Preparation of a stand-alone verification plan is strongly advised if:

• The system is complex
• There are several separate verification activities being performed on the system
• Multiple deployment sites are involved
• Multiple stakeholders have to be satisfied

There is also the question of how comprehensive to make the verification effort. It is impossible to validate all possible combinations of actions under all possible operational situations. A good rule of thumb is: if it was important enough to write down as a requirement, then it should be verified, at least once. In-process⁴ verification performed on the needs and requirements will help ensure that the correct requirements are being verified.

Once the verification plan is completed, use the following checklist to ensure all critical information has been included.

Does the Verification Plan answer all the questions of who, what, where, and when?

Does the Verification Plan make clear what needs to happen if a failure is encountered?

Does the Verification Plan document the configuration of the hardware, software?

Are all requirements traced to a verification case?

1 Purpose of Document (Chapter 1 of the Verification Plan)

This chapter identifies the type of verification activity to be performed within this Verification Plan. For instance, this activity may validate the entire system, a sub-system, the deployment at a site, a burn-in, or any other verification activity called for in a relevant Program Plan.

2 Scope of Project (Chapter 2 of the Verification Plan)

This chapter gives a brief description of the planned project and the purpose of the system to be built. It also describes the environment in which the project operates. It identifies the organization structures that encompass all stakeholders. It also gives a brief description of the role to be played by each stakeholder. This includes ad hoc and existing management work groups and multi-disciplinary technical teams that should be formed to support the project.

3 Referenced Documents (Chapter 3 of the Verification Plan)

This is a list of all documents used in the preparation of this Verification Plan. This almost always includes the Project Plan (if one was written), and the applicable Requirements Documents. However, reference of other documents, such as descriptions of external systems, standards, a Concept of Operations, and manuals, may also need to be included.

4 Conducting Verification (Chapter 4 of the Verification Plan)

This chapter provides details on how verification is accomplished. It defines: who does the verification; when and where it is to be done; the responsibilities of each participant before, during and after verification; the deployed hardware and software configuration; and the documents to be prepared as a record of the verification activity. It is also important to define how anomalies are to be handled (that is, what to do when a failure occurs during verification).

In general, the following information should be included in this section:

• A description of the participating organizations and personnel and identification of their roles and responsibilities. This may include for example, a verification conductor, verification recorder, operators, and/or engineering support.
• Identification of the location of the verification effort, that is, the place, or places, where the verification must be observed.
• The deployed hardware and software configuration for all of the verification cases, including hardware and software under verification and any supporting equipment, software, or external systems. Several configurations may be necessary.
• Identification of the documents to be prepared to support the verification, including Verification Procedures, a Verification Report and descriptions of special equipment and software.
• Details on the actual conduct of verification, including:
  • Notification of participants
  • Emphasis on the management role of the verification conductor
  • Procedures for approving last minute changes to the procedures
  • The processes for handling a failure, including recording of critical information, determination of whether to stop the verification, restart, or skip a procedure, resolution of the cause of a failure (e.g. fix the software, reset the system, and/or change the requirements), and determination of the re-verification activities necessary as a result of the failure.

5 Verification Identification (Chapter 5 of the Verification Plan)

This section identifies the specific verification cases to be performed. A verification case is a logical grouping of functions and performance criteria (all from the Requirements Documents) that are to be verified together. For instance, a specific verification case may cover all the control capabilities to be provided for control of the adaptive control system. There may be several individual requirements that define this capability, and they all are verified in one case. The actual grouping of requirements into a case is arbitrary. They should be related and easily combined into a reasonable set of procedure actions.

Each case should contain at least the following information:

• A description name and a reference number.
• A description of the objective of the verification case, usually taken from the wording of the requirements, to aid the reader understanding the scope of the case.
• A complete list of requirements to be verified or traceability to the requirements in the requirements document. Since each requirement has a unique number, they can be accurately and conveniently referenced without repetition.
• Any data to be recorded or noted during verification, such as expected results of a step. Other data, such as a recording of a digital message sent to an external system, may be required to validate the performance of the system.
• A statement of the pass/fail criteria. Often, this is just a statement that the system operates per the need or scenario.
• A description of the verification configuration. That is a list of the hardware and software items needed for verification and how they should be connected (in most cases this is the deployed system configuration). Often, the same configuration is used for several verification cases.
• A list of any other important assumptions and constraints necessary for conduct of the verification case.

previous | next