Model Systems Engineering Documents for Dynamic Message Sign (DMS) Systems
Chapter 5. Validation Plan Guidance
The model Validation Plan can be found in Appendix E. The chapters required for the Validation Plan document are:
This document describes the activity of validation that the system being built meets the user needs and scenarios developed in the concept of operations. The validation documents will generally include three levels of validation documents:
To ensure user needs and scenarios are validated by this activity, trace each need and scenario into a validation case, then into appropriate steps in the validation procedure.
A separate Validation Plan and procedures may be minimal for the simplest projects, especially where the system is commercially available and does not involve any custom software development, and where the agency staff have a very clear understanding of the purpose of the system. Preparation of a validation plan is strongly advised if:
There is also the question of how comprehensive to make the validation effort. It is impossible to validate all possible combinations of actions under all possible operational situations. A good rule of thumb is: if it was important enough to write down as a need or scenario, then it should be validated, at least once. This may not, for example, validate all possible failure mode conditions or all possible incident scenarios. In-process2 validation performed on the needs will help ensure that end-to-end validation of the system will meet the stakeholder needs.
Once the Validation Plan has been prepared, use this checklist to ensure all critical information has been included.
This section identifies the type of validation activity to be performed within this Validation Plan. For instance, this activity may validate the entire system, a sub-system, the deployment at a site, a burn-in, or any other validation activity called for in the Program Plan or in the SEMP.
This section gives a brief description of the planned project and the purpose of the system to be built. Special emphasis is placed on the project’s user needs and issues that must be addressed and validated.
This section also describes the environment in which the project operates. It identifies the organization structures that encompass all stakeholders. It also gives a brief description of the role to be played by each stakeholder. This includes ad hoc and existing management work groups and multi-disciplinary technical teams that should be formed to support the project.
This is a list of all documents used in the preparation of this Validation Plan. This usually includes the Project Plan, (if one was written), and the applicable Requirements Documents. Reference to other documents, such as descriptions of external systems, standards, a Concept of Operations, and manuals may also be included.
This section provides details on how validation is accomplished. It defines: who does the validation; when and where it is to be done; the responsibilities of each participant before, during, and after validation; the deployed hardware and software configuration; and the documents to be prepared as a record of the validation activity.
This section also defines how anomalies are to be handled (that is, what to do when an unexpected situation or a failure occurs during validation).
In general, the following information should be included in this section:
This section identifies the specific validation cases to be performed. A validation case is a logical grouping of functions and performance criteria (all from the Concept of Operations Document) that are to be validated together. For instance, a specific validation case may cover DMS System user permissions by the DMS System Manager. There may be several individual user needs that define this capability, and they all are validated in one case. The actual grouping of user needs into a case is arbitrary; however, the grouping is usually based on the grouping of user needs and the operational scenarios in the Concept of Operations. They should be related and easily combined into a reasonable set of procedure actions. Suggested validation cases that may be used in the Validation Plan document are included in the Validation Plan Sample Cases table (Appendix E).
Each case should contain at least the following information:
Each validation case in Appendix E corresponds to the same name of a section of user needs in Section 4 of the ConOps model document. The applicable operational scenarios defined in section 8 of the ConOps are referenced in each validation case. The details of each validation case will need to be added as the system is further defined.
2 In-process validation is reviewing the needs and requirements during the definition stage by the stakeholders to ensure that all the needs have been identified and traced to the appropriate requirements and have been reviewed for completeness for each of the needs. [ Return to Note 2 ]
United States Department of Transportation - Federal Highway Administration