United States Department of Transportation - Federal Highway Administration FHWA HomeFeedback

Systems Engineering for Intelligent Transportation Systems

TOC      Previous Page      Next Page

4 ITS Technical Processes

This document is intended to help you understand how systems engineering can be used throughout the ITS project life cycle. Chapters 4 and 5 present two different types of processes that support systems engineering:

  1. Technical processes, such as system requirements, high-level design, integration, and verification, are described here in Chapter 4. These processes, depicted in the "V" systems engineering model, are performed to develop an ITS project that meets the user's needs. This chapter leads you step by step through each of the technical processes in the "V". Each process is summarized, key activities are identified, and outputs that should be generated are defined.
  2. Project management processes, such as project planning, risk management, and configuration management, are described in Chapter 5. These cross-cutting activities are just as important to the success of the project, but they do not appear in the "V" diagram because they apply to many different steps in the "V". These processes are used to plan, monitor, and control the ITS project so that it is completed on time and on budget.

Relationship to Traditional Transportation Processes

ITS projects are identified and funded through transportation planning and programming/budgeting processes in each state, planning region (e.g., metropolitan planning area), and agency. The "V" diagram and the systems engineering process begin once a need for an ITS project has been identified. The early steps in the "V" define the project scope and determine the feasibility and acceptability as well as the costs and benefits of the project. These early steps actually support planning and programming/budgeting since they are intended to identify high-level risks, benefits, and costs and to determine if the ITS project is a good investment. The latter steps support project implementation, then transition into operations and maintenance, changes and upgrades, and ultimate retirement or replacement of the system. (The systems engineering "V" is placed in context with the traditional transportation project life cycle in Section 6.1.)

Technical Documentation

Each step of the process that is described in this chapter results in one or more technical outputs. This documentation is used in subsequent steps in the process and provides a critical documentation trail for the project. The documentation that is discussed in this chapter is identified in Table 1, which provides a bird's-eye view of where it fits in the "V". Several resources provide good descriptions and templates for this documentation.7 Note that not every ITS project will require every document listed in the table. (More information on tailoring is provided later in this chapter and in Section 6.2.3.)

About the Examples

This chapter is illustrated with real examples that show how different agencies have used the systems engineering process for their ITS projects. These real examples aren't perfect and shouldn't be taken as the only approach, or even the best approach, for accomplishing a particular step. As time goes by and we gain experience using systems engineering on ITS projects, many more examples will become available.

Table 1: Technical Documentation in the "V" Systems Engineering Process

Documentation Chapter / Process Step 4.1 Using the Regional ITS Architecture 4.2 Feasibility Study / Concept Exploration 4.3 Concept of Operations 4.4 System Requirements 4.5 System Design 4.6 SW/HW Development & Testing 4.7 Integration & Verification 4.8 Initial Deployment 4.9 System Validation 4.10 Operations and Maintenance 4.11 Retirement / Replacement
Relevant portion of Reg ITS Arch C   U            
Feasibility Study   C U                
Concept of Operations     C U              
System Validation Plan     C           U    
System Requirements Document       C U U U
System Verification Plan       C     U        
Traceability Matrix       C U   U     U U
System Acceptance Plan       C       U      
High-Level (Architectural) Design         CU U          
Detailed Design Specifications         C U          
Interface Specifications         C U U        
Subsystem Verification Plans         C   U        
Integration Plan         C   U        
Subsystem Acceptance Plan         C   U        
Unit/Device Test Plan         C U          
SW/HW Development Plans           CU          
Verification Procedures             CU        
Delivery & Installation Plan               CU      
Transition Plan               CU      
O&M Plan and Procedures               C   U  
System Validation Procedures                 CU    
System Retirement Plan                     CU
C: Create documentation U: Primary Use/Update of the documentation

4.1 Regional ArchitectureUsing the Regional ITS Architecture

In this step: The portion of the regional ITS architecture that is related to the project is identified. Other artifacts of the planning and programming processes that are relevant to the project are collected and used as a starting point for project development. This is the first step in defining your ITS project.

  • Define the project scope while considering the regional vision and opportunities for integration
  • Improve consistency between ITS projects and identify more efficient incremental implementation strategies
  • Improve continuity between planning and project development

Sources of Information

  • Relevant regional ITS architecture(s)
  • Regional/national resources supporting architecture use
  • Other planning/programming products relevant to the project

Key Activities

  • Identify regional ITS architecture(s) that are relevant to the project
  • Identify the portion of the regional ITS architecture that applies
  • Verify project consistency with the regional ITS architecture and identify any necessary changes to the regional ITS architecture

Process Results

  • List of project stakeholders and roles and responsibilities
  • List of inventory elements included in or affected by the project
  • List of requirements the proposed system(s) must meet
  • List of interfaces and the information to be exchanged or shared by the system(s)
  • Regional ITS architecture feedback as necessary

Proceed only if you have:

  • Demonstrated consistency with the regional ITS architecture and identified needed changes to the regional ITS architecture, if applicable
  • Extracted the relevant portion of the regional ITS architecture that can be used in subsequent steps
  • Reached consensus on the project/system scope

4.1.1 Overview

The regional ITS architecture provides a good starting point for systems engineering analyses that are performed during ITS project development. It provides region-level information that can be used and expanded in project development.

When an ITS project is initiated, there is a natural tendency to focus on the programmatic and technical details and to lose sight of the broader regional context. Using the regional ITS architecture as a basis for project implementation provides this regional context as shown in Figure 8. It provides each project sponsor with the opportunity to view their project in the context of surrounding systems. It also prompts the sponsor to think about how the project fits within the overall transportation vision for the region. Finally, it identifies the integration opportunities that should be considered and provides a head start for the systems engineering analysis.

A graphic that shows project sponsors focusing on the details of their specific project.  It also shows how the individual projects fit within the overall transportation vision for the region.

Figure 8: Regional ITS Architecture Framework for Integration

Terminology.The regional ITS architecture is a tool that is used in transportation planning, programming, and project implementation for ITS. It is a framework for institutional agreement and technical integration for ITS projects and is the place to start when defining the basic scope of a project.

The regional ITS architecture is the first step in the "V" because the best opportunity for its use is at the beginning of the development process. The architecture is most valuable as a scoping tool that allows a project to be broadly defined and shown in a regional context. The regional ITS architecture step and the concept exploration step that is described in the next section may iterate since different concepts may have different architecture mappings. The initial architecture mapping may continue to be refined and used as the Concept of Operations and system requirements are developed.

Resources.The Regional ITS Architecture Guidance Document provides detailed guidance for regional ITS architecture development, use, and maintenance. (Version 2 of this document provides detailed guidance for using a regional ITS architecture to support project implementation.)

4.1.2 Key Activities

Initial use of the regional ITS architecture requires a few basic activities: locating the right architecture, identifying the portion of the architecture that applies to your project, and notifying the architecture maintainer of any required regional architecture changes. None of these tasks is particularly time consuming – the basic extraction of information can be done in an afternoon, even for a fairly complex project, if you are knowledgeable about the regional ITS architecture. Of course, it can be time consuming to climb the learning curve, and coordinating and building consensus on the scope of the project will require time and effort. Each of the key activities is described in the following paragraphs.

4.1.3 Outputs

The first output of this step is the subset of the regional ITS architecture for the ITS project. While the Rule/Policy requires a subset of the regional ITS architecture to be identified, it does not define the components that should be included. You should consult local guidelines or requirements to help make this determination. In most cases, the following components will precisely define the scope of the project: (1) stakeholders, (2) inventory elements, (3) functional requirements, and (4) information flows.

These four components define the system(s) that will be created or impacted by the project, the affected stakeholders, the functionality that will be implemented, and the interfaces that will be added or updated. Other components may be identified, including market packages, roles and responsibilities, relevant ITS standards, and agreements. For very large ITS projects, this might be several pages of information. For a small ITS project, this might fit on a single page. The information that is extracted will actually be used in the concept exploration, Concept of Operations, requirements, and design steps that follow.

Tools.The Turbo Architecture software tool can be used to quickly and accurately define an ITS project architecture if the regional ITS architecture was developed with Turbo. Turbo Architecture can be used to generate diagrams and reports that fully document the portion of the regional ITS architecture that will be implemented by the project. Turbo Architecture can also be used to develop a project ITS architecture based on the National ITS Architecture if a regional ITS architecture does not exist. The Turbo Architecture software can be obtained from McTrans by visiting their website at http://www-mctrans.ce.ufl.edu/featured/turbo/.

Tip.If you don't find what you need in the regional ITS architecture, then you should add the missing or changed items to your architecture subset and highlight them so it is clear what you changed. For example, if there is a system in your project that is not represented in the regional ITS architecture, add it to your architecture subset and highlight it. The highlighted changes serve two purposes: they allow you to move forward with an augmented architecture subset that you can use in the next steps of the process, and they provide the basis for your feedback for regional ITS architecture maintenance.

The second output of this step – feedback to the regional ITS architecture maintenance team – is just as important as the first output. Submit any recommended changes using the mechanism defined for your region in the regional ITS architecture maintenance plan.

4.1.4 Examples

The subset of the regional ITS architecture that is included in the project can be shown in a series of simple tables and/or a diagram from Turbo Architecture, as shown in Figure 9. This figure identifies the inventory elements and interfaces that will be implemented by a MaineDOT Dynamic Message Sign (DMS) project in which several signs will be installed in Portland, Maine, along with a central control system with interfaces to a number of other centers. Functional requirements that are relevant to the project were also extracted, as shown in Table 2.

The Maine DOT Communications Center controls Maine DOT field devices and portable work zone field devices.  It also coordinates with the Maine Turnpike, Maine Emergency Operations Center, and Local Traffic Operations Centers.

Figure 9: Example: MaineDOT DMS Project Architecture Subset

Table 2: MaineDOT DMS Project Functional Requirements (Partial List)

Element Functional Area ID Requirement
MaineDOT Communications Center TMC Traffic Information Dissemination 1 The Center shall remotely control DMS for dissemination of traffic and other information to drivers.
MaineDOT Communications Center TMC Traffic Information Dissemination 3 The Center shall collect operational status for the driver information systems equipment (DMS, HAR, etc.).
MaineDOT Communications Center TMC Traffic Information Dissemination 4 The Center shall collect fault data for the driver information systems equipment (DMS, HAR, etc.) for repair.

4.2 Feasibility Study/Concept Exploration

Feasibility Study/Concept Exploration.In this step: A business case is made for the project. Technical, economic, and political feasibility is assessed; benefits and costs are estimated; and key risks are identified. Alternative concepts for meeting the project's purpose and need are explored, and the superior concept is selected and justified using trade study techniques.


  • Identify superior, cost-effective concept, and document alternative concepts with rationale for selection
  • Verify project feasibility and identify preliminary risks
  • Garner management buy-in and necessary approvals for the project

Sources of Information

  • Project goals and objectives
  • Project purpose and need
  • Project scope/subset of the regional ITS architecture

Key Activities

  • Define evaluation criteria
  • Perform initial risk analysis
  • Identify alternative concepts
  • Evaluate alternatives
  • Document results

Process Results

  • Feasibility study that identifies alternative concepts and makes the business case for the project and the selected concept

Proceed only if you have:

  • Received approval on the feasibility study from project management, executive management, and controlling authorities, as required
  • Reached consensus on the selected alternative

4.2.1 Overview

In this step, the proposed ITS project is assessed to determine whether it is technically, economically, and operationally viable. Major concept alternatives are considered, and the most viable option is selected and justified. While the concept exploration should be at a fairly high level at this early stage, enough technical detail must be included to show that the proposed concept is workable and realistic. The feasibility study provides a basis for understanding and agreement among project decision makers – project management, executive management, and any external agencies that must support the project, such as a regional planning commission.

Rule/Policy.The Rule/Policy requires the systems engineering analysis to include an analysis of alternative system configurations and technology options. The focus of this Rule/Policy requirement is on design decisions that are made later in the process, but a fundamental analysis of basic systems configurations is performed in this step.

Terminology.It is easy to confuse the concept exploration that is performed in this step with the Concept of Operations that is developed in the next step. Concept exploration is a broad assessment of fundamentally different alternatives – for example, a new electronic toll facility versus additional conventional lanes. The alternatives would have dramatically different concepts of operations, so it is important to select a proposed concept before developing a Concept of Operations. Different alternatives may also have different regional ITS architecture mappings, so this step may iterate with the previous regional ITS architecture step.

The process is driven by the project vision, goals, and objectives, and by the needs for the project that were identified through the transportation planning process. It starts by identifying a broad range of potential concepts that satisfy the project need(s). The concepts are compared relative to measures that assess the relative benefits, costs, and risks of each alternative. Project stakeholders must be involved to establish the evaluation criteria, verify that all viable alternative concepts are considered, and make sure there is consensus on the selected alternative. The recommendations provide a documented rationale for the selected project approach and an assessment of its feasibility. The process is identical to a feasibility study done for large roadway and transit projects.

The alternatives analysis that is performed during a feasibility analysis uses a basic trade study technique, shown in Figure 10, that will be repeated many times during the project life cycle. At this early concept exploration step, the alternatives are fundamental choices, such as to maintain the existing facility ("do nothing"), build a new road, or add ITS technology to the existing facility. During design, the alternatives are design decisions, such as whether signs should be located at location A, B, or C. During construction, alternatives may have to do with optimizing closures while the work is performed. At each step, a set of alternatives is identified and analyzed from technical, economic, and operational perspectives.

A trade study includes the following steps: 1) State the problem, 2) Generate ideas and collect candidate solutions, 3) Select the practical solutions based on predefined constraints and criteria, 4) Evaluate the remaining candidates to arrive at an optimum solution.

Figure 10: Concept Exploration Uses Basic Trade Study Techniques

Tailoring.A feasibility study should be conducted only when a broad analysis is needed before the commitment of development resources. Some states require a feasibility study for certain ITS projects. A feasibility study is typically not required for smaller, incremental ITS projects where there are not fundamentally different approaches for implementation and where feasibility is not in question – for example, a project that adds DMS to an existing system. In other cases, a broad exploration of alternatives is not warranted but a cost-benefit study is needed to make the business case for the project.

4.2.2 Key Activities

Here at the very beginning of project development, the unknowns will certainly outnumber the knowns. Without a Concept of Operations or requirements, many assumptions will have to be made. It is important to educate the group performing this assessment on the concept exploration process and to set a schedule – otherwise, this stage could be an open-ended process since there's always something new over the horizon. The process activities are:

4.2.3 Outputs

The feasibility study establishes the business case for investment in a project by defining the reasons for undertaking the project and analyzing its costs and benefits. Different organizations and different projects will have different requirements, but a feasibility study should contain, at a minimum, the following:

  1. A description of the problem or opportunity that the project is intended to address.
  2. The project objectives that must be achieved for an alternative to be an effective response to the problem or opportunity, and the evaluation criteria that were used.
  3. Economic and risk analyses of each of the alternatives that meet the established objectives, and the reasons for rejecting the alternatives that were not selected.
  4. A summary description of the selected alternative, including the major system features and resources that will be used.
  5. An economic analysis of the funding sources, the life-cycle costs and benefits of the project, and the life-cycle costs and benefits of the current method of operation.

4.2.4 Examples

Identification of Alternatives – Transportation Planning Studies

Feasibility studies that examine alternative concepts are frequently done for large transportation projects as part of corridor studies, major investment studies, and environmental analysis reports. The ITS option(s) in these studies often compete with traditional capital improvement options; hybrid options, which include a mix of technology and traditional capital improvements, are also considered.

For example, a congested corridor in Collin County, Texas, was the subject of a feasibility study report (FSR)9 that was prepared by representatives from the North Central Texas Council of Governments and affected agencies. This FSR examined the following alternatives: (1) do nothing, (2) build a new freeway, (3) build a toll road with electronic collection (two alternatives), and (4) build managed lanes. One summary table that compared the traffic volumes supported by the different alternatives is shown in Table 3. Supported traffic volumes, estimated capital costs, and potential revenue generation were used to compare the alternatives. The analysis favored the electronic toll alternatives.

Broad alternatives analyses like these are included in many planning studies.

Table 3: Comparison of Alternatives – Supported Traffic Volumes for 2025

The table shows the supported traffic volumes are highest for a freeway alternative.  A managed lane alternative supports the next highest volumes, followed by the toll road alternatives.  The do nothing alternative supports the lowest traffic volume.

A simple graph that generally shows monetized benefits of travel time reduction, reduced vehicle operating costs, and improved safety for an example project.  No units are provided, but the alternative is shown to provide substantial cost savings compared to the base condition.Cost-Benefit Analysis

Minnesota DOT developed a guidance document10 for cost-benefit analysis in 2005 that includes several illustrative examples. Generally, higher-level graphics that visually compare the costs and benefits of the alternatives, like the one shown in Figure 11, are used in the body of the cost-benefit analysis. More detailed computation that supports high-level graphics, like the table reproduced in Table 4, is included in appendices.

Figure 11: Example of High-Level Economic Comparison of Alternatives

Table 4: Example of Alternatives Benefit Estimation

A detailed table that shows estimated vehicle hours of travel savings per year from 2011 through 2030 and then converts the vehicle hours of travel saving to constant dollars and present value dollars.  729 million dollars of cumulative benefit are calculated for the example project.

4.3 Concept of Operations

Concept of Operations.In this step: The project stakeholders reach a shared understanding of the system to be developed and how it will be operated and maintained. The Concept of Operations (ConOps) is documented to provide a foundation for more detailed analyses that will follow. It will be the basis for the system requirements that are developed in the next step.

  • High-level identification of user needs and system capabilities in terms that all project stakeholders can understand
  • Stakeholder agreement on interrelationships and roles and responsibilities for the system
  • Shared understanding by system owners, operators, maintainers, and developers on the who, what, why, where, and how of the system
  • Agreement on key performance measures and a basic plan for how the system will be validated at the end of project development

Sources of Information

  • Stakeholder lists, roles and responsibilities, and other components from the regional ITS architecture
  • Recommended concept and feasibility study from the previous step
  • Broad stakeholder input and review

Key Activities

  • Identify the stakeholders associated with the system/project
  • Define the core group responsible for creating the Concept of Operations
  • Develop an initial Concept of Operations, review with broader group of stakeholders, and iterate
  • Define stakeholder needs
  • Create a System Validation Plan

Process Results

  • Concept of Operations describing the who, what, why, where, and how of the project/system, including stakeholder needs and constraints
  • System Validation Plan defining the approach that will be used to validate the project delivery

Proceed only if you have:

  • Received approval on the Concept of Operations from each stakeholder organization
  • Received approval on the System Validation Plan from each stakeholder organization

4.3.1 Overview

The Concept of Operations (ConOps) is a foundation document that frames the overall system and sets the technical course for the project. Its purpose is to clearly convey a high-level view of the system to be developed that each stakeholder can understand. A good ConOps answers who, what, where, when, why, and how questions about the project from the viewpoint of each stakeholder, as shown in Figure 12.

The concept of operations describes a system including the hardware, software, people, procedures, data, and facilities and describes how that system will operate in its operational and support environments.  The concept of operations shoud represent the system from the viewpoint of the operator, the user, the maintainer, and other key stakeholders.

Figure 12: Concept of Operations (Adapted from ANSI/AIAA-G-043-1992)

Terminology.In ITS, we draw a distinction between an Operational Concept, which is the high-level description of roles and responsibilities that is included in the regional ITS architecture, and a Concept of Operations, which is the more detailed, multifaceted document described in this section.

Tailoring.Don't assume that a new ConOps is required for every ITS project. A single system-level ConOps can support many ITS projects that incrementally implement and extend a system. For example, a ConOps may be developed for a large transportation management system. This system may be implemented and expanded with numerous ITS projects over several years. Once the ConOps is developed, it may be reviewed and used with relatively minor updates for each of the projects that incrementally implement the transportation management system.

4.3.2 Key Activities

Although there is no single recipe for developing a ConOps, successful efforts will include a few key activities:

4.3.3 Output

The ConOps should be an approachable document that is relevant to all project stakeholders, including system operators, maintainers, developers, owners/decision makers, and other transportation professionals. The art of creating a good ConOps lies in using natural language and supporting graphics so that it is accessible to all while being technically precise enough to provide a traceable foundation for the requirements document and the System Validation Plan.

Caution.The ConOps is not a requirements document that lists the detailed, testable requirements for the system, nor is it a design document that specifies the technical design or technologies to be used. Resist the temptation to predetermine the solution in the ConOps – you should not unnecessarily preclude viable options at this early step. You also want to "keep it simple" and refrain from using formalized, highly structured English that is more suitable for the requirements and design specifications that follow.

Done right, the ConOps will be a living document that can be revised and amended so that it continues to reflect how the system is really operated. Later in the life cycle, an up-to-date ConOps can be used to define changes and upgrades to the system.

Two different industry standards provide suggested outlines for Concepts of Operations: ANSI/AIAA-G-043-1992 and IEEE Std 1362-1998, as shown in Figure 13. Both outlines include similar content, although the structure of the IEEE outline lends itself more to incremental projects that are upgrading an existing system or capability. The ANSI/AIAA outline is focused on the system to be developed, so it may lend itself more to new system developments where there is no predecessor system. Successful ConOps have been developed using both outlines. Obtain a copy of both, and make your own choice if you need to develop a ConOps.

The ANSI/AIAA-G-043-1992 and IEEE Std 1362-1998 industry standards suggested outlines for Concepts of Operations. ANSI/AIAA-G-043-1992 outline: 1. Scope, 2. Referenced Documents, 3. User-Oriented Operational Description, 4. Operational Needs, 5. System Overview, 6. Operational Environment, 7. Support Environment, 8. Operational Scenarios. IEEE Std 1362-1998 outline: 1. Scope, 2. Referenced Documents, 3. The Current System or Situation, 4. Justification for and Nature of Changes, 5. Concepts for the Proposed System, 6. Operational Scenarios, 7 Summary of Impacts, 8. Analysis of the Prooposed System.

Figure 13: Industry-Standard Outlines for Concept of Operations

Graphics should be used to highlight key points in the ConOps. At a minimum, a system diagram that identifies the key elements and interfaces and clearly defines the scope of the project should be included. Tables and graphics can also be a very effective way to show key goals and objectives, operational scenarios, etc.

Rule/Policy.The Rule/Policy requires identification of participating agency roles and responsibilities as part of the systems engineering analysis for ITS projects. It also requires that the procedures and resources necessary for operations and management of the system be defined. These elements are initially defined and documented for the project as part of the ConOps. In the ANSI/AIAA standard outline, most of these elements fit under Chapter 3 (User-Oriented Operational Description). In the IEEE outline, the current system information is included in Chapter 3 and the proposed system information is in Chapter 5.

The System Validation Plan that is created during this step should describe how the final system will be measured to determine whether or not it meets the original intent of the stakeholders as described in the ConOps. (For further details and examples, see Section 4.9.)

4.3.4 Examples

Many Concepts of Operations have been generated for all types of ITS projects in the last five years. Excerpts from a few examples are included here to show some of the ways that key elements of the ConOps have been documented for ITS projects following the sequence from the ANSI/AAIA outline.

User-Oriented Operational Description (Roles and Responsibilities)

Typically, roles and responsibilities are documented as a list or in tabular form. Table 5 is an excerpt of a table from the California Advanced Transportation Management System (CATMS) ConOps that is structured to show shared responsibilities and to highlight coordination points between the different system stakeholders. This early documentation of "who does what" grabs the stakeholders' attention and supports development of system requirements and operational agreements and procedures in future steps.

Table 5: Roles and Responsibilities (Excerpt from CATMS Concept of Operations)

The lead and support roles are defined for major TMC tasks.  For example, special event traffic monitoring is led by the TMC operator and supported by the TMC manager and maintenance dispatch.  Roles for the TMS engineer, Information Technology personnel, headquarters, and upper management are also defined.

System Overview

The system overview is typically supported by one or more diagrams that show the scope, major elements, and interrelationships of the system. Many types of diagrams can be used, from simple block diagrams to executive-level graphics-rich diagrams. Figure 14 is an example of a high-level graphic that includes basic process flow information, roles and responsibilities, and interfaces, providing an "at a glance" overview of the major facets of the system.

In this example, information is collected from field reports and other sources and provided to the Incident Commander.  The Emergency Operations Center, which includes a Joint Information Center, distributes the information to the traveling public.

Figure 14: Example of System Overview Graphic
(from Communicating with the Public Using ATIS During Disasters Concept of Operations)

Operational Scenarios

In operational scenarios, the ConOps takes the perspective of each of the stakeholders as different scenarios unfold that illustrate major system capabilities and stakeholder interactions under normal and stressed (e.g., failure mode) circumstances. The stakeholders walk through the scenario and document what the agencies and system would do at each step.

Figure 15 shows an example of a scenario that includes some realistic detail that help stakeholders immerse themselves in the scenario and visualize system operation. This is one of five scenarios that were developed for the City of Lincoln StarTRAN AVL system to show the major system capabilities and the interactions between the AVL system and its users and other interfacing systems.

Text Box: Marcel, a StarTran bus operator, usually begins his work shift with administrative activities. After receiving supervisory direction, he boards the bus and prepares the AVL system. He begins by logging into the system.  The system then prompts Marcel for the route to be followed. He enters the planned route number, and the AVL system retrieves the appropriate route and schedule information from the AVL system server. The bus' AVL system then asks Marcel to verify the appropriate route and schedule information were properly retrieved.  Once he provides verification, the bus' head sign is automatically updated to reflect the appropriate route information. The fare payment schedule is automatically adjusted to reflect the verified route, modified as necessary by the system clock to reflect any applicable time-differential rates.  The system then loads the appropriate bus stop announcements for the chosen route. These prerecorded announcements are consistent regardless whether Marcel or another bus operator is driving the route, and have been verified as ADA compliant. These announcements are then broadcast at the appropriate bus stop throughout the route.

Figure 15: Operational Scenario Description11

4.4 System Requirements

System requirements.In this step: The stakeholder needs identified in the Concept of Operations are reviewed, analyzed, and transformed into verifiable requirements that define what the system will do but not how the system will do it. Working closely with stakeholders, the requirements are elicited, analyzed, validated, documented, and baselined.

  • Develop a validated set of system requirements that meet the stakeholders' needs

Sources of Information

  • Concept of Operations (stakeholder needs)
  • Functional requirements, interfaces, and applicable ITS standards from the regional ITS architecture
  • Applicable statutes, regulations, and policies
  • Constraints (required legacy system interfaces, hardware/software platform, etc.)

Key Activities

  • Elicit requirements
  • Analyze requirements
  • Document requirements
  • Validate requirements
  • Manage requirements
  • Create a System Verification Plan
  • Create a System Acceptance Plan

Process Results

  • System Requirements document
  • System Verification Plan
  • Traceability Matrix
  • System Acceptance Plan

Proceed only if you have:

  • Received approval on the System Requirements document from each stakeholder organization, including those that will deploy, test, install, operate, and maintain the new system
  • Received approval on the System Verification Plan from the project sponsor, the test team, and other stakeholder organizations
  • Received approval on the System Acceptance Plan from the project sponsor, the Operations & Maintenance (O&M) team, and other stakeholder organizations

4.4.1 Overview

One of the most important attributes of a successful project is a clear statement of requirements that meet the stakeholders' needs. Unfortunately, creating a clear statement of requirements is often much easier said than done. The initial list of stakeholder needs that are collected will normally be a jumble of requirements, wish lists, technology preferences, and other disconnected thoughts and ideas. A lot of analysis must be performed to develop a good set of requirements from this initial list.

In requirements engineering, you elicit, analyze, document, and validate requirements.  Throughout the process, you manage the requirements and make sure stakeholders participate.

Figure 16: Requirements Engineering Activities

Terminology.EIA-63212 defines requirement as "something that governs what, how well, and under what conditions a product will achieve a given purpose." This is a good definition because it touches on the different types of requirements that must be defined for a project. Functional requirements define "what" the system must do, performance requirements define "how well" the system must perform its functions, and a variety of other requirements define "under what conditions" the system must operate. Requirements engineering covers all of the activities needed to define and manage requirements that are shown in Figure 16.

Caution.Specify What, Not How. Be sure to keep the definition of a requirement in mind as you develop your system requirements. Many requirements documents contain statements that are not requirements. One of the most common pitfalls is to jump to a design solution and then write "requirements" that define how the system will accomplish its functions. Specify what the system will do in the system requirements, and save how the system will do it for the system design step.

It is important to involve stakeholders in requirements development. Stakeholders may not have experience in writing requirements statements, but they are the foremost experts concerning their own requirements. The project requirements ultimately are the primary formal communication from the system stakeholders to the system developer. The project will be successful only if the requirements adequately represent stakeholders' needs and are written so they will be interpreted correctly by the developer.

Caution.In the effort to get stakeholders involved, make sure you don't sour them on the project by making unreasonable demands on their time or putting them in situations where they can't contribute. Many nontechnical users have been subjected to stacks of detailed technical outputs that they can't productively review. Sooner or later, the user will wave the white flag in this situation and become unresponsive. You must (1) pick your stakeholders carefully and (2) make participation as focused and productive as possible.

Tailoring.The Requirements step is an important one that you shouldn't skimp on. Every ITS project should have a documented set of requirements that are approved and baselined. Of course, this doesn't mean that a new requirements specification must be written from scratch for every project. Projects that enhance or extend an existing system should start with the existing system requirements. This doesn't have to be a particularly large document for smaller ITS projects. The system requirements specification for a recent website development project was less than 20 pages.

4.4.2 Key Activities

There isn't one "right" approach for requirements development. Different organizations develop requirements in different ways. Even in the same organization, the requirements development process for a small ITS project can be much less formal than the process for the largest, most complex ITS projects. The differences are primarily in the details and in the level of formality. All requirements development processes should involve elicitation, analysis, documentation, validation, and management activities. Note that each of these activities is highly iterative. In the course of a day, a systems engineer may do a bit of each of the activities as a new requirement is identified, refined, and documented.

4.4.3 Outputs

No matter how you developed your requirements, you must document them in some consistent, accessible, and reviewable way. The requirements development process may result in several different levels of requirements over several steps in the "V" – stakeholder requirements, system requirements, subsystem requirements, etc. – that may be documented in several different outputs. For example, stakeholder requirements might be documented in a series of Use Cases; system requirements, in a System Requirements Specification; and subsystem requirements, in subsystem specifications. All of these requirements should be compiled in a single repository that can be used to manage and publish the requirements specifications at each stage of the project.

Tip.It is much easier to use a standard template for the requirements specifications than it is to come up with your own, and numerous standard templates are available. If your organization does not have a standard requirements template, you can start with a standard template like the one contained in IEEE Standard 830 (for software requirements specifications) or IEEE Standard 1233 (for system requirements specifications). Starting with a template saves time and ensures that the requirements specification is complete. Of course, the template can be modified as necessary to meet the needs of the project.

The system requirements specification should fully specify the system to be developed and should include the following information:

As you read through this list, you may recognize that some of this information has already been collected and documented in previous steps, and there is no need to recreate it here. Refer back to the Concept of Operations that already contains a description of the system boundary, the system itself, and other items in this list.

A System Verification Plan, describing the approach for verifying each and every system requirement, and a System Acceptance Plan, describing the capabilities that must function successfully for customer acceptance, should be created, reviewed, and approved.

4.4.4 Examples

Stakeholder Requirements

The Oregon DOT TripCheck project developed a User Functional Requirements Specification, which lists the user requirements for the redesigned TripCheck website. The excerpt from this document in

Table 8 shows several user requirements for the website autorouting function. As shown, every requirement is prioritized on a scale from 1 ("must have") to 4 ("don't implement") and is related to different types of end users – Commuters (C), Inter-City Travelers (ICT), Tourist Travelers (TT), ADA Travelers (ADA), and Commercial Truckers (CT). These prioritized user requirements were used by the contractor to support Use Case modeling and to define system requirements.

Table 8: ODOT TripCheck User Requirements (Excerpt)

The requirements that are shown are: 1) The system should allow the user to enter a multi-point route using a combination of the criteria specified in MP004c, 2) The system shall allow the user to select destination points by clicking on the map, and 3) The system shall allow the user to specify the following when determining road routes: starting date, starting time, month of travel, quickest route, shortest route, most scenic route, and routes most recommended by others.

Note that stakeholder requirements that are collected through the requirements elicitation process are likely to have a few imperfections. The key is to document the stakeholder requirements, make them as clear and succinct as possible, prioritize them, and then use them to develop more formally stated system requirements.

System Requirements

The Maryland CHART II system is a statewide traffic management system that has been operational since 2001. The CHART program maintains a website that provides all of the CHART documentation at http://www.chart.state.md.us, including a comprehensive system requirements document. A few of the system requirements for the equipment inventory and report generation functions are shown in Table 9.

Table 9: CHART II System Requirements (Excerpt)

3.1.3 Equipment Inventory

The equipment inventory is a list of SHA equipment used in connection with CHART response to incidents. The system provides functions to maintain the inventory, equipment status, and to generate alerts for delinquent equipment. The system shall provide the capability to maintain the equipment inventory. The system shall support the addition of new equipment entries to the inventory. The system shall support the modification of existing equipment inventory entries. The system shall support the deletion of equipment inventory entries. The system shall support the allocation of equipment to events.

3.1.4 Report Generation

This section lists requirements for the generation of reports from the CHART system and archive data. The system shall provide the capability to generate reports from online and archived data. The system shall support the generation of operational reports. The system shall support the generation of a Center Situation report. The system shall support the generation of a Disable Vehicle event report. The system shall support the generation of an Incident event report. The system shall support the generation of traffic volume reports.

Traceability Matrix

Table 10 is a typical traceability matrix that would be maintained and populated throughout the project development process. The matrix may be maintained directly in a database or spreadsheet for small projects or generated and maintained with a requirements management tool for more complex projects. Using either approach, the matrix provides backwards and forwards traceability between stakeholder needs (and other potential requirements sources), system requirements, design, implementation, and verification test cases. As shown, only the unique identifiers (e.g., UN1.1) are actually included in the traceability matrix so you don't have to keep many instances of the actual text up-to-date. Note also that the design and implementation columns would not actually be completed until later in the process.

Table 10: Sample Traceability Matrix

Requirement Source System Requirement High-Level Design Component Code Unit Test Case
UN1.1 R00220 7.2.3 SystemMonitor UT 4.2
R00330 7.3.1 CalcVolume UT 5.5
R00331 7.3.1 CalcCount

4.5 System Design

High-level design and detailed design.In this step: A system design is created based on the system requirements including a high-level design that defines the overall framework for the system. Subsystems of the system are identified and decomposed further into components. Requirements are allocated to the system components, and interfaces are specified in detail. Detailed specifications are created for the hardware and software components to be developed, and final product selections are made for off-the-shelf components.

  • Produce a high-level design that meets the system requirements and defines key interfaces, and that facilitates development, integration, and future maintenance and upgrades
  • Develop detailed design specifications that support hardware and software development and procurement of off-the-shelf equipment

Sources of Information

  • Concept of Operations
  • System Requirements document
  • Off-the-shelf products
  • Existing system design documentation
  • ITS standards
  • Other industry standards

Key Activities

  • Evaluate off-the-shelf components
  • Develop and evaluate alternative high-level designs
  • Analyze and allocate requirements
  • Document interfaces and identify standards
  • Create Integration Plan, Subsystem Verification Plans, and Subsystem Acceptance Plans
  • Develop detailed component-level design specifications

Process Results

  • Off-the-shelf evaluation and alternatives summary reports
  • High-level (architectural) design
  • Detailed design specifications for hardware/software
  • Integration Plans, Subsystem Verification Plans, Subsystem Acceptance Plans, and Unit/Device Test Plans

Proceed only if you have:

  • Approved high-level design for the project
  • Defined all system interfaces
  • Traced the system design specifications to the requirements
  • Approved detailed specifications for all hardware/software components

4.5.1 Overview

In the systems engineering approach, we define the problem before we define the solution. The previous steps in the "V" have all focused primarily on defining the problem to be solved. The system design step is the first step where we focus on the solution. This is an important transitional step that links the system requirements that were defined in the previous step with system implementation that will be performed in the next step, as shown in Figure 18.

High Level design and detailed design are represented as a bridge that allows the systems engineer and software implementer to span the gap between requirements and implementation.

Figure 18: System Design is the Bridge from Requirements to Implementation

There are two levels of design that should be included in your project design activities:

Terminology.High-level design is commonly referred to as architectural design in most systems engineering handbooks and process standards. Architectural design is used because an overall structure for the project is defined in this step. IEEE 61016 defines architectural design as "the process of defining a collection of hardware and software components and their interfaces to establish the framework for the development of a computer system". Of course, ITS projects may include several computer systems, a communications network, distributed devices, facilities, and people. High-level design defines a framework for all of these project components.

Terminology.Detailed design is the complete specification of the software, hardware, and communications components, defining how the components will be developed to meet the system requirements. The software specifications are described in enough detail that the software team can write the individual software modules. The hardware specifications are detailed enough that the hardware components can be fabricated or purchased.

Many consider design to be the most creative part of project development. Two different designs might both meet the system requirements, but one could be far superior in how efficiently it can be developed, integrated, maintained, and upgraded over time. Perhaps the most significant contributor to a successful design is previous design experience with similar systems. The latest car designs all build on 100 years of accumulated automotive design experience. Similarly, the design of a new transportation management system should build on existing successful transportation management system designs. In both cases, the system designer builds on knowledge of what worked before and, perhaps even more importantly, what did not.

Tailoring.It is extremely rare to find an ITS system that is truly "unprecedented", so many if not most system designs should be able to build on existing design information. This is particularly true for projects that are extending an existing system that already includes a well- documented design. In this case, the high-level design will change only to the degree that new functionality or interfaces are added. Similarly, much of the detailed design can be reused for projects that extend the coverage of an existing system.

4.5.2 Key Activities

System design is a cooperative effort that is performed by systems engineers and the implementation experts who will actually build the system. The process works best when there is a close working relationship among the customer, the systems engineers (e.g., a consultant or in-house systems engineering staff), and the implementation team (e.g., a contractor or in-house team).

High-Level Design

High-level design is normally led by systems engineers with participation from the implementation experts to ensure that the design is implementable. Typical activities of high-level design are shown in Figure 19. Each of the activities are actually performed iteratively as high-level design alternatives are defined and evaluated.

High-Level design activities include: 1) develop high-level design alternatives, 2) evaluate alternatives, 3) analyze and allocate requirements, and 4) document the interfaces.

Figure 19: High-Level Design Activities

Detailed Design

Hardware and software specialists create the detailed design for each component identified in the high-level design. Systems engineers play a supporting role, providing technical oversight on an ongoing basis. As you might expect, the detailed design activity will vary for off-the-shelf and custom components, as shown in Figure 21.

If off-the-shelf products are used, then select the off-the-shelf products.  Otherwise, for custom development, prototype user interface and develop detailed hardware and software specifications.

Figure 21: Detailed Design Activities

4.5.3 Outputs

High-Level Design

There isn't a single "best way" to present the high-level design to stakeholders and developers since different users will have different needs and different viewpoints. Over the years, high-level designs have evolved to include several different interconnected "views" of the system. Each view focuses on a single aspect of the system, which makes the system easier to analyze and understand. The specific views that are presented will vary, but they will typically include a physical view that identifies the system components and their relationships; a functional view that describes the system's behavior; a technical view that identifies the interfaces in detail, including the standards to be used; and an informational view that describes the information that will be managed by the system. As shown in Figure 23, these views are just different ways of looking at the same system.

A dynamic message sign system might contain four views: 1) a physical view that shows the system is composed of center equipment, controllers, signs, and a maintenance PC, 2) a functional view that shows functions like compose message and prioritize message, 3) a technical view that identifies the interfaces and NTCIP standards that will be used, and 4) an Information View that shows the data that is stored for every DMS message.

Figure 23: High-Level Design May Include Several Views

Other outputs of the high-level design process include Integration Plans, Subsystem Verification Plans, and Subsystem Acceptance Plans that will be used in the integration and verification of the system. (See Section 4.7 for further details.)

Detailed Design

This activity results in the design of hardware and software for all system components that will support hardware and software development and off-the-shelf product procurement. Other artifacts of the development process include unit/device verification plans. A record of the technical reviews that were conducted should also be included in the project documentation.

4.5.4 Examples

High-Level Design

The CHART II documentation includes a system architecture document that includes many different views of the CHART II system, such as entity relationship diagrams, Use Case diagrams, and network architecture diagrams. Table 11 is an excerpt from the document that shows the subsystems included in the CHART II software.

Table 11: CHART II Software Subsystems (Excerpt)

CI Name Subsystems
Alert Management
Camera Control
Communications Log Management
Data Export Management
Device Management
DMS Control
HAR Control
HAR Notification
Message Library Management
Notification Management
Plan Management
Resource Management
Schedule Management
SHAZAM Management
System Monitor
Traffic Event Management
Traffic Sensor System Management
User Management
Video Monitor Management

In contrast with the CHART II statewide system high-level design, many smaller ITS projects have relatively simple high-level designs, such as the system architecture for the MyBus system depicted in Figure 24. This figure identifies the subsystems and major interfaces in the MyBus system.

The MyBus System includes an AVL system, a schedule change controller, and a schedule database.  The real time schedule data is processed by a predictor and then made available by the MyBus Web server.

Figure 24: Metro Transit MyBus System Architecture

ITS projects that include significant user interface development should prototype the user interface to help users visualize the software that will be developed before significant resources are committed. The objective is to develop a prototype that demonstrates the software look and feel with the least amount of work possible. The simplest prototypes are a series of static images in paper form. For example, when ODOT redesigned its TripCheck website, the implementation team developed a series of "wireframe" diagrams that showed the proposed interface design with enough detail to gather user feedback. One of the 40 wireframe diagrams that was included in the design package is shown in Figure 25.

A wireframe diagram shows how the display will look in general.  It shows there will be a map in the center of the display, menus across the top, alerts and priority announcements on the right side, and quick links at the bottom of the display.

Figure 25: User Interface Prototype Example: ODOT TripCheck Wireframe Diagram

Detailed Design

There are many ways to document software detailed design. Most commonly, it is portrayed using object-oriented techniques and the Unified Modeling Language17, but any technique that the implementation team selects is fine as long as it is detailed enough to support software construction and clear enough to support peer reviews and walkthroughs.

Table 12 is an example of a detailed design for part of the Shadow software that works behind the scenes to keep the traffic information on the ODOT TripCheck website up to date. Note that the interface is defined and that loosely structured program design language (PDL) is used to define the algorithm that is used to process transactions. If much of this appears to be gibberish to you, you are not alone. This is why many agencies use software specialists to provide an independent review of the detailed software development artifacts for higher risk software projects on their behalf.

Table 12: Detailed Software Design Example: ODOT TripCheck Software Class Definition

This detailed design example for the Shadow method defines the algorithm to be used, the method interface, scope, files impacted, related methods, and error handling.

4.6 Software/Hardware Development and Testing

Software/Hardware Development and Testing.In this step: Hardware and software solutions are created for the components identified in the system design. Part of the solution may require custom hardware and/or software development, and part may be implemented with off-the-shelf items, modified as needed to meet the design specifications. The components are tested and delivered ready for integration and installation.

  • Develop and/or purchase hardware and software components that meet the design specifications and requirements with minimum defects
  • Identify any exceptions to the requirements or design specifications that are required

Sources of Information

  • System and subsystem requirements
  • System design
  • Off-the-shelf products
  • Industry standards
  • Unit/Device Test Plans

Key Activities

  • Plan software/hardware development
  • Establish development environment
  • Procure off-the-shelf products
  • Develop software and hardware
  • Perform unit/device testing

Process Results

  • Software/hardware development plans
  • Hardware and software components, tested and ready for integration
  • Supporting documentation (e.g., training materials, user manuals, maintenance manuals, installation and test utilities)

Proceed only if you have:

  • Conducted technical reviews of the hardware/software
  • Performed configuration/quality checks on the hardware and software
  • Received all supporting documentation
  • Verified that unit/device testing has been successfully completed

4.6.1 Overview

Although hardware and software development may be the first task that comes to mind when thinking about an ITS project, the systems engineering approach focuses on the preceding requirements and design steps and on the integration, verification, and validation steps to follow.

This is where the investment in a clear set of requirements and a good system design should begin to pay dividends. The systems engineering process now provides technical oversight as an implementation team of specialists fabricates the hardware and writes the software. This is a highly iterative process, particularly for software, where key features may be incrementally implemented, tested, and incorporated into the baseline over time. Progress is monitored through a planned series of walkthroughs, inspections, and reviews, as shown in Figure 26.

Implementation iterates between implementation and testing and is supported by walkthroughs, inspections, and reviews.

Figure 26: Monitoring Software/Hardware Development

Although the systems engineering approach does not specify the mechanics of hardware and software development (this is left to the implementation team), the development effort is obviously critical to project success. This is the time to build quality into the hardware/software and to minimize defects. A common refrain in the software industry is that you can't test quality into the software – you must build it in from the beginning. The systems engineering activities that are suggested in this chapter are intended to ensure that the implementation team builds quality into their products.

Tailoring.In practice, most of the hardware that is used for ITS projects is purchased off the shelf. Software development is more prevalent, but many ITS projects include little or no software development. ITS projects that do not include custom hardware or software development acquire the necessary off-the-shelf hardware and software components at this step. Detailed specifications created as part of the detailed design step described in Section 4.5 are used to support the acquisition. The system components are acquired, and bench testing is performed to verify that they meet their specifications. In such cases, the detailed hardware/software development and unit testing described in this chapter are not required.

Custom software development for ITS projects has proven to be a relatively risky endeavor. This is why software development receives more attention than hardware development in this chapter. It is beyond the scope of this document to discuss specific software development techniques, but there are several clear factors that contribute to software development success:

4.6.2 Key Activities

The hardware and software specialists implement and test each system component. Systems engineers play a supporting role, providing technical oversight on an ongoing basis to identify minor issues early, before they grow into large problems. The process works best when there is a close working relationship among the customer, the systems engineers (e.g., a consultant or in-house systems engineering staff), and the implementation team (e.g., a contractor or an in-house team). Each of the activity descriptions is followed by a discussion of the technical review and monitoring of that activity.

4.6.3 Outputs

This step results in hardware and software components that are tested and ready for integration and verification. Artifacts of the development process are also delivered, including the Software/Hardware Development Plans, development environment documentation, unit test results, change control records, and supporting products and documentation. A record of the technical reviews that were conducted should also be included in the project documentation.

4.7 Integration and Verification

Subsystem verification, system verification, integration and recomposition.In this step: The software and hardware components are individually verified and then integrated to produce higher-level assemblies or subsystems. These assemblies are also individually verified before being integrated with others to produce yet larger assemblies, until the complete system has been integrated and verified.

  • Integrate and verify the system in accordance with the high-level design, requirements, and verification plans and procedures
  • Confirm that all interfaces have been correctly implemented
  • Confirm that all requirements and constraints have been satisfied

Sources of Information

  • System Requirements document
  • High-level design specifications
  • Detailed design specifications
  • Hardware and software components
  • Integration plan
  • System and Subsystem Verification Plans
  • Subsystem Acceptance Plans

Key Activities

  • Add detail to integration and verification plans
  • Establish integration and verification environment
  • Perform integration
  • Perform verification

Process Results

  • Integration plan (updated)
  • Verification plan (updated)
  • Integration test and analysis results
  • Verification results, including corrective actions taken

Proceed only if you have:

  • Documented evidence that the components, subsystems, and system meet the allocated requirements
  • Documented evidence that the external and internal interfaces are working and consistent with the interface specifications

4.7.1 Overview

In this step, we assemble the system components into a working system and verify that it fulfills all of its requirements. Assembling a puzzle is a nice, simple analogy for this step, but the challenge in an ITS project "puzzle" is that you may find that not all of the pieces are available at the same time, some won't fit together particularly well at first, and there will be pressure to change some of the pieces after you have already assembled them. The systems engineering approach provides a systematic process for integration and verification that addresses the challenges and complexity of assembling an ITS system.

Integration and verification are iterative processes in which the software and hardware components that make up the system are progressively combined into subsystems and verified against the requirements, as shown in Figure 27. This process continues until the entire system is integrated and verified against all of its requirements. This is the opposite of the decomposition that was performed during the Requirements and Design steps, which is reflected in the symmetry between the left and right sides of the "V". Components that are identified and defined on the left side of the "V" are integrated and verified on the right.

Integration creates assemblies from tested components and interfacing systems.  The assemblies are verified and then integrated into the overall system.  The complete system is verified to complete the process.

Figure 27: Iterative Integration and Verification

Terminology.In systems engineering, we draw a distinction between verification and validation. Verification confirms that a product meets its specified requirements. Validation confirms that the product fulfills its intended use. In other words, verification ensures that you "built the product right", whereas validation ensures that you "built the right product". This is an important distinction because there are lots of examples of well-engineered products that met all of their requirements but ultimately failed to serve their intended purpose. For example, a bus rapid transit system might implement a signal priority capability that satisfies all of its requirements. This system might not serve its intended purpose if the traffic network is chronically congested and the buses are never actually granted priority by the signal control system when they need it most. Verification is discussed in this section; system validation is described in Section 4.9.

Integrating and verifying the system are key systems engineering activities. The software and hardware specialists who led the previous step are also involved and provide technical support as their components are integrated into the broader system. Stakeholders should also be materially involved in verification, particularly in the system verification activities. As the verification proceeds from detailed component verification to end-to-end system verification, the implementation team becomes less involved and the stakeholders become more involved. The systems engineering activity provides continuity to the process.

4.7.2 Key Activities

Integrating and verifying the system include basic planning, preparation, and execution steps, described as follows:

4.7.3 Outputs

Integration and verification result in a documentation trail showing the activities that were performed and their results. The outputs include:

4.7.4 Examples

Many verification plans that have been developed for ITS projects are available on the Internet. Although they have many different titles – integration test plans, functional test plans, verification plans – they have similar content. For example, Table 13 is an excerpt from a functional test plan that was used to test the Oregon DOT TripCheck website. The script in the table lists each action that the tester should take and the expected result from the system in a step-by-step procedure that tests links in a website navigation panel.

Table 13: Verification Procedure Example: ODOT TripCheck Functional Test Plan (Excerpt)

1 None Test winter travel links  
1.a   Select Chain Laws Opens: Pages/RCMap.asp?curRegion=ChainLaws
1.b   Select Traction Tires Opens: Pages/RCMap.asp?curRegion=TractionTires
1.c   Select Minimum Chain Requirements Opens: Pages/RCMap.asp?curRegion=MinChainReqs
2 None Test related links Each link opens a browser window with an external URL

Table 14 is a verification procedure from a Maryland Chart II Integration Test Plan that includes a bit more background for each test case in a slightly different format.

Table 14: CHART II Integration Test Plan (Excerpt)

Test ID: General 1
Purpose: To show that a valid username/password is accepted for logging in to CHART II within 15 seconds, and that an invalid combination is rejected. In addition, this test case also demonstrates that the system returns control to the user and the user is not prevented from performing activities in other windows on the desktop. CHART-27, CHART-10, CHART-21, CHART-275, CHART-276, CHART-29, CHART-26 Test Start Date:
Test Pre-Conditions: This test assumes a valid username and password of a user in the CHART2 system is known. Test End Date:
Test Step No. Test Steps Expected Behavior Results As Expected (Y/N) Comments
1 Click on the Login button on the GUI toolbar. An hourglass should display immediately, within 5 seconds, till the login window is displayed. Then, you should be prompted for a UserID and password.    
2 Attempt to login with an invalid username or password. The system should popup an error message indicating that an invalid user ID or password was specified.    
3 Attempt to login with the valid UserID and password. The system should indicate that the user is logged in by showing Operations Center:Username on the GUI toolbar window.    
4 Click on Navigator Navigator window is opened.    
5 Click on DMS node List of DMSs is displayed on the right hand side of the Navigator.    

Reports are generated that document the actual results of the verification tests that were performed. Table 15 is a brief excerpt from a test result report for the desktop application that is used by ODOT to update data on the TripCheck website. Each row in the table summarizes the results for each test case. This excerpt was selected because it includes one of the few test cases in this report in which the actual results did not match the expected results. Note that in Test 2, an error occurred that exposed a software defect that had to be fixed. Identification of defects like this before the system is operational is one of the key benefits of a thorough verification process.

Table 15: ODOT TripCheck 2.0 System Test Results (Excerpt)

1 Enter an incident of type Herbicide Application. An incident of type Herbicide Application. Does not appear in TripCheck. As expected, incident did not go into the transaction table.
2 Enter an incident that is then put on hold. An incident that is on hold. Does not appear in TripCheck. When incident is put on hold a delete transaction is entered in the shadow table. An error occurred with this delete transaction and the incident remained on TripCheck.
3 Put the incident from step 2 back into active status in HTCRS.   Incident is on TripCheck. As expected.

4.8 Initial Deployment

Deployment.In this step: The system is installed in the operational environment and transferred from the project development team to the organization that will own and operate it. The transfer also includes support equipment, documentation, operator training, and other enabling products that support ongoing system operation and maintenance. Acceptance tests are conducted to confirm that the system performs as intended in the operational environment. A transition period and warranty ease the transition to full system operation.


  • Uneventful transition to the new system

Sources of Information

  • Integrated and verified system, ready for installation
  • System Acceptance Plan

Key Activities

  • Plan for system installation and transition
  • Deliver the system
  • Prepare the facility
  • Install the system
  • Perform acceptance tests
  • Transition to operation

Process Results

  • Hardware and software inventory
  • Final documentation and training materials
  • Delivery and installation plan, including shipping notices
  • Transition Plan with checklists
  • Test issues and resolutions
  • Operations and maintenance plan and procedures

Proceed only if you have:

  • Formally accepted the system
  • Documented acceptance test results, anomalies, and recommendations

4.8.1 Overview

Up to this point, the system has been tested primarily in a lab environment. The next step is to ship the system to the actual deployment site(s), install and check it out, and make sure the system and personnel are ready to transition to system operations and maintenance (O&M), as shown in Figure 28.

The development team delivers the system, the O&M team prepares the facility or site, the system is installed and transitioned to operations.  Development team participation declines as the O&M team participation ramps up through the process.

Figure 28: Transition from Development Team to Operations and Maintenance Team

Larger systems may be installed in stages. For example, a closed-circuit television (CCTV) camera network may be built out incrementally over the course of several years and several projects. This may be done to spread the costs across several fiscal years or to synchronize with other construction projects in the region. In other cases, phased deployment may be performed to mitigate risk by deploying the essential core of the system and then adding features over time. If it is necessary to deploy the system in stages, whether due to funding constraints, to mitigate risk, or to synchronize with other projects, it is important to understand the dependencies between successive deployments and to prioritize the projects accordingly.

4.8.2 Key Activities

The following tasks are cooperatively performed to deliver, install, and transition the system to full operational status:

4.8.3 Outputs

The primary output of this step is a fully installed system (in a facility or site modified to meet the system requirements) that has been transitioned to operational status. To support this effort, the following outputs should be generated:

4.8.4 Examples

Deployment plans and installation plans can be complex documents for ITS projects that involve significant center and/or field equipment installation. Planning for deployment and installation must begin early in the project for such systems. For example, the Sunol Smart Carpool Lane Joint Powers Agency (JPA) developed a deployment plan as part of its Systems Engineering Management Plan during initial planning for the I-680 Smart Lane Project. This plan defines deployment activities (see Figure 29), roles and responsibilities, deployment personnel by position, installation equipment and tools, system documentation, and installation considerations such as safety, code and industry standards, planning requirements, weather accommodations, and shop drawing submittals. More detailed installation plans will be prepared by the system integrator based on this deployment plan.

Text Box: Pre-installation Activities as follows: Verify civil and conduit work., Work with the JPA to finalize the Installation Plan, Installation Schedule, and other deployment documents., Ensure that all safety procedures are in place., Secure Caltrans Encroachment Permit. Roadside Equipment Installation as follows: FasTrak Antennas and Readers., Tolling Zone Lane Controllers., Enforcement Beacons., Vehicle Detection System (VDS) Equipment., Closed Circuit TV (CCTV) Equipment., Communications Network Equipment., Other equipment as identified in the RFP. TDC Equipment Installation as follows: Trip Processor Hardware and Software., Customer Service Representative (CSR) Workstations., JPA Smart Lane website., Interface to the Bay Area Toll Authority (BATA) Regional Customer Service Center (RCSC)., Interface to the Caltrans TMC., Interface to the CHP Enforcement Equipment., Other equipment as identified in the RFP. Post-installation Activities as follows: Verify that all of the equipment and software is installed properly, Verify that each internal subsystem communicates properly to each other., Verify that all installed equipment and software operates properly.

Figure 29: I-680 Smart Lane Project Deployment Activities Overview

4.9 System Validation

System Validation.In this step: After the ITS system has passed system verification and is installed in the operational environment, the system owner/operator, whether the state DOT, a regional agency, or another entity, runs its own set of tests to make sure that the deployed system meets the original needs identified in the Concept of Operations.


  • Confirm that the installed system meets the user's needs and is effective in meeting its intended purpose

Sources of Information

  • Concept of Operations
  • Verified, installed, and operational system
  • System Validation Plan

Key Activities

  • Update Validation Plan as necessary and develop procedures
  • Validate system
  • Document validation results, including any recommendations or corrective actions

Process Results

  • System Validation Plan (update) and procedures
  • Validation results

Proceed only if you have:

  • Validated that the system is effectively meeting its intended purpose
  • Documented issues/shortcomings
  • Established ongoing mechanisms for monitoring performance and collecting recommendations for improvement
  • Made modifications to the Concept of Operations to reflect how the system is actually being used

4.9.1 Overview

A few readers may be surprised to see that there is another step in the "V" between initial deployment and operations and maintenance. After all, in the last few chapters we have already verified that the system meets all of its requirements, installed the system and trained the users, and the customer has successfully conducted acceptance tests and formally accepted the system. Aren't we done?

The answer is: yes and no. Yes, the system has been put into operation and is beginning to be used for its intended purpose. No, we aren't done. Now that the system is beginning to be used in the operational environment, we have our first good opportunity to measure just how effective the system is in that environment (i.e., system validation).

The stakeholders validate the system at each step in the development system culminating in a successful system validation at the end.

Figure 30: Validation Occurs Throughout the Systems Engineering Process

Terminology.In systems engineering, we draw a distinction between verification and validation. Verification confirms that a product meets its specified requirements. Validation confirms that the product fulfills its intended use. The majority of system verification can be performed before the system is deployed. Validation really can't be completed until the system is in its operational environment and is being used by the real users. For example, validation of a new signal control system can't really be completed until the new system is in place and we can see how effectively it controls traffic.

Of course, the last thing we want to find is that we've built the wrong system just as it is becoming operational. This is why the systems engineering approach seeks to validate the products that lead up to the final operational system to maximize the chances of a successful system validation at the end of the project. This approach is called in-process validation and is shown in Figure 30. As depicted in the figure, validation was performed on an ongoing basis throughout the process:

Since validation was performed along the way, there should be fewer surprises during the final system validation that is discussed in this step. The system will have already been designed to meet the user's expectations, and the user's expectations will have been set to match the delivered system.

4.9.2 Key Activities

The system validation is the responsibility of the system owner and will typically be performed by the system users.

4.9.3 Outputs

System validation should result in a document trail that includes an up-to-date Validation Plan; validation procedures (if written); and validation results, including disposition of identified deficiencies. There are several industry and government standard outlines for validation plans, including IEEE Standard 101218, which is intended for software verification and validation but is also applicable to broader system verification and validation. Note that this standard covers both verification and validation plans with a single outline.

4.9.4 Examples

There are few good examples of system validations that have been performed for ITS projects. Some of the best examples are evaluations that have been performed for field operational tests (FOTs), and other evaluations that have looked in detail at the benefits of ITS. For example, the evaluation of the ORANGES Electronic Payment Systems FOT initially identified system goals and then related them to quantitative and qualitative performance measures, as shown in Table 16. Each of the performance measures was then evaluated, in many cases using before-and-after study techniques, to determine whether the system goals were achieved. Figure 31 shows results supporting the transponder market penetration goal (Goal 2). This evaluation report is a good example of many validation techniques, including the collection of baseline data, before-and-after studies, statistical analysis, evaluation of other causal factors, and interview and survey activities.

Table 16: ORANGES Evaluation Goals and Performance Measures

FOT Evaluation Goal Measure
1. Increase parking revenue
  • Revenue received
2. Increase transponder market penetration
  • Number of smart card users that newly acquire a transponder
3. Reduce transaction times
  • Average transaction times
4. Increase prepaid revenue share
  • % revenue prepaid
5. Reduce monthly pass distribution costs
  • Procurement, inventory, delivery, commissions for any conventional passes made available on smart cards
6. Increase automated payment equipment uptime
  • % equipment availability
7. Cardholders use the joint account
  • Card use profiles
  • Average prepaid balance
  • Modal use profile
8. Understand customer perceptions
  • General benefits
  • Ease of use
  • Convenience of revaluing
  • Customer feedback
9. Understand operations/maintenance staff perceptions, including:
  • General benefits
  • Reduced payment disputes
  • Reduced transfer abuse
  • Ease of customer use
  • Maintenance
  • Operations/maintenance staff feedback
10. Understand planning/management staff perceptions, including:
  • General benefits
  • More comprehensive data collection
  • Planning/management staff feedback
11. Understand interagency perceptions, including:
  • General institutional issues
  • Interagency collaboration
  • Partnership feedback

A graph that summarizes the cumulative number of cards issued by the agencies over the course of the analysis period.

Figure 31: ORANGES Evaluation – Cumulative Transponders Issued

4.10 Operations and Maintenance

Operations, Maintenance, Changes and Upgrades.In this step: Once the customer has accepted the ITS system, the system operates in its typical steady state. System maintenance is routinely performed and performance measures are monitored. As issues, suggested improvements, and technology upgrades are identified, they are documented, considered for addition to the system baseline, and incorporated as funds become available. An abbreviated version of the systems engineering process is used to evaluate and implement each change. This occurs for each change or upgrade until the ITS system reaches the end of its operational life.


  • Use and maintain the system over the course of its operational life

Sources of Information

  • System requirements (operations/maintenance requirements)
  • Operations and Maintenance Plan and procedures
  • Training materials
  • Performance data
  • Evolving stakeholder needs

Key Activities

  • Conduct Operations and Maintenance Plan reviews
  • Establish and maintain all operations and maintenance procedures
  • Provide user support
  • Collect system operational data
  • Change or upgrade the system
  • Maintain configuration control of the system
  • Provide maintenance activity support

Process Results

  • System performance reports
  • Operations logs
  • Maintenance records
  • Updated operations and maintenance procedures
  • Identified defects and recommended enhancements
  • Record of changes and upgrades
  • Budget projections and requests

Proceed only if you have:

  • Demonstrated that the system has reached the end of its useful life

4.10.1 Overview

Now that the ITS system is up and running, it enters a "steady state" period that lasts until the system is retired or replaced. During this period, operators, maintainers, and users of the system may identify issues, suggest enhancements, or identify potential efficiencies. New releases of hardware and software will be installed and routine maintenance will be performed. Approved changes and upgrades are incorporated into the system baseline using the systems engineering process, as shown in Figure 32. O&M personnel might also identify process changes that may streamline O&M activities. All changes to the processes should be documented.

The V systems engineering process is used to update the system baseline as changes and upgrades are approved and implemented.

Figure 32: Changes/Upgrades Performed Using Systems Engineering

Successful operations and maintenance of the system will lead to customer and user satisfaction; for example, the CCTVs will be online and fully functional at all times; rush-hour drivers will be able to obtain accurate, up-to-the-minute speed, accident, and construction reports before they head out the door; and transit vehicles will arrive on time. This is when the system benefits are realized.

4.10.2 Key Activities

In most systems, operations and maintenance is where the lion's share of life-cycle costs are incurred. The key activities are performed periodically unless a change is considered severe and affects system performance dramatically.

4.10.3 Outputs

The current system configuration, including hardware, software, and operational information, must be documented and maintained. A complete record of all system changes should also be documented and readily available. This is especially helpful when trying to duplicate an anomaly identified by a user or operator.

System performance reports should be generated, both from any installed automated performance monitors and from user-support calls received. Trend analysis reports can be generated and reviewed to identify system deficiencies.

Text Box: Document Maintenance and Operations Activities, Develop and Maintain a Cost Database for Maintenance and Operations, Analyze Maintenance and Operations Requirements, Analyze Staffing Requirements for Maintenance and Operations, Develop a Training Program for Maintenance and Operations Personnel, Prioritize Maintenance Needs, Develop and Maintain a Spare Parts Inventory, Develop a Maintenance Plan, Develop an Operations Manual

Figure 33: Kentucky ITS M&O Plan Best Practices

4.10.4 Examples

Operations and Maintenance Plans

The Kentucky Transportation Center developed a Maintenance and Operations Plan for ITS in Kentucky that provides recommendations for supporting and coordinating ITS maintenance and operations activities throughout the Kentucky Transportation Cabinet. It inventories ITS equipment and systems, identifies national best practices for operations and maintenance (see Figure 33), assesses current maintenance and operations practices in Kentucky, and makes recommendations. Many of the recommendations and best practices identified in the report will be relevant to other agencies. This broad agency-wide plan complements the detailed procedures that are used to operate and maintain individual systems.

Operations and Maintenance Procedures

Operations and maintenance procedures are detailed and don't make particularly good reading unless you actually operate and maintain one of these systems, in which case they are indispensable. These manuals will be subject to relatively frequent changes as personnel will find errors and new and better ways to operate and maintain the system. A short excerpt from the CHART II O&M Procedures is shown in Figure 34.

Text Box: A.1.5 Dialogic board installation/config: A.1.5.1 Setting the board identification number. - If more then one board is to be used, each board must have a unique identification number. Turn the rotary switch (SW100) to select one of the 16 board ID settings. A.1.5.2 Attaching the PEB Terminator (XTERM) (PEB MODE ONLY) - Attaching the PEB terminator to the XTERM socket. To terminate the voice resource board, use the resource module position.  Insert the terminator in the resource module position, make sure that Pin 1 as indicated by black dot is positioned in the upper right corner in the XTERM socket. A.1.5.3 Installing Dialogic Adapter - Insert the Dialogic adapter in system ISA slot

Figure 34: CHART II O&M Procedures

Change and Upgrade Plans

Metro Transit in Seattle, Washington, upgraded its existing Transit AVL system to support transit traveler information systems as part of the Smart Trek program. To support this upgrade, detailed cost estimates were made based on systems engineering analysis of the AVL enhancements that would be required to support the traveler information objectives of the Smart Trek project. The estimate is shown in Table 17.

Table 17: Metro Transit AVL System Upgrade

The AVL system upgrade includes new CPU boards, modem boards, termination boards, AVL software, and labor resulting in estimated upgrade costs of more than 1 million dollars.  Recurring costs are also estimated at 137,700 thousand dollars.

4.11 Retirement/Replacement

Retirement/replacement.In this step: Operation of the ITS system is periodically assessed to determine its efficiency. If the cost to operate and maintain the system exceeds the cost to develop a new ITS system, the existing system becomes a candidate for replacement. A system retirement plan will be generated to retire the existing system gracefully.


  • Remove the system from operation, gracefully terminating or transitioning its service
  • Dispose of the retired system properly

Sources of Information

  • System requirements (retirement/disposal requirements)
  • Service life of the system and components
  • System performance measures and maintenance records

Key Activities

  • Plan system retirement
  • Deactivate system
  • Remove system
  • Dispose of system

Process Results

  • System retirement plan
  • Archival documentation

Proceed only if you have:

  • Planned the system retirement
  • Documented lessons learned
  • Disposed of the retired system properly

4.11.1 Overview

Systems are retired from service for a variety of reasons. Perhaps the system is being replaced by a newer system, or maybe the Concept of Operations has changed such that stakeholder needs are going to be met in an alternative manner that will no longer require use of the system. For example, the emergency call boxes that currently dot many of the nation's highways are beginning to be retired because their usage has decreased dramatically due to widespread use of cell phones. Many of the first-generation ITS systems are twenty years old and approaching the end of their useful life. Regardless of the reason for the retirement of the system, you should make sure that everything is wrapped up (e.g., hardware and software inventory identified for disposal is audited, final software images are captured, and documentation is archived), the contract is closed properly, and the disposal of the system is planned and executed.

4.11.2 Key Activities

This step represents the end of the system life cycle – the retirement and disposal of the ITS system. An important characteristic of the systems engineering process is the planning of all events; the retirement of the system should be planned as well.

The retirement plan should include a complete inventory of all software and hardware, final system and documentation configurations, and other information that captures the final operational status of the system. This should include identification of ownership so that owners can be given the option to keep their equipment and use it elsewhere. It should also include how the system and documentation will be disposed of, including an assessment and plan if special security measures should be in place or if there are environmental concerns that might dictate the site of disposal. You should also plan to erase the content of all storage devices to protect any personal data that might pose privacy concerns. The retirement plan should be reviewed and approved by all parties, including the agency or contractor providing O&M, the owner of the system (if different), and other key personnel.

If the system to be retired is not documented as well as it should be, steps are taken to capture all necessary data and reverse engineer interfaces and any system configuration information that is needed to support a replacement system. Existing databases may need to be exported and translated into a format suitable for the replacement system.

The next activity is to execute the retirement plan and record the results. It's also a good idea to hold a "lessons learned" meeting that includes suggested system improvements. All recommendations should be archived for reference in future system disposals. The O&M contract should be officially closed out if one exists.

4.11.3 Outputs

A system retirement plan will be generated that describes the strategy for removing the system from operation and disposing of it. Its execution will result in the retirement of the ITS system. The final system configuration, including hardware, software, and operational information, will be documented and archived, together with a list of "lessons learned".

7The California Systems Engineering Guidebook and the Florida Systems Engineering Management Plan both include good documentation descriptions and templates. See Chapter 7 for more resources.
8Visit http://www.its.dot.gov/arch/ for more information on the National ITS Architecture.
9See http://www.nctcog.org/trans/corridor/SH121/.
10See http://www.oim.dot.state.mn.us/EASS/
11From "StarTran Automated Vehicle Location System Concept of Operations", StarTran and City of Lincoln, NE.
12EIA-632 is the Electronics Industry Association Standard "Processes for Engineering a System".
13The Florida Systems Engineering Management Plan is available at http://www.floridaits.com/ SEMP/Index.htm.
14See http://www.incose.org/ProductsPubs/products/toolsdatabase.aspx.
15INCOSE (www.incose.org) has a comprehensive list of requirements management tools in its tools database.
16IEEE 610 is the IEEE Standard Computer Dictionary.
17There are many resources available if you want to learn more about UML; www.uml.org is a good place to start.
18IEEE 1012-2004: IEEE Standard for Software Verification and Validation.

Download the free adobe acrobat reader to view PDFs You will need the Adobe Acrobat Reader to view the PDFs on this page.


TOC      Previous Page      Next Page FHWA