Model Systems Engineering Documents for Adaptive Signal Control Technology (ASCT) Systems
|Functional requirements||What the system is to do|
|Performance requirements||How well it is to perform|
|Non-functional requirements||Under what conditions it will perform|
|Enabling requirements||What other actions must be taken in order for the system to become fully operational|
|Constraints||Limitations imposed on the design by agency's policies and practices, such as type of software, type of equipment and external standards|
|Interface requirements||Definitions of the interfaces between sub-systems or with external systems|
|Data requirements||Definitions of data flows between sub-systems or with external systems|
In all cases, it is essential to describe the functional, performance, non-functional and enabling requirements and the constraints. If the proposed system will interface with another system or requirements are also required at the sub-system level (which may be the case if you are requiring some customization or new modules specifically for your application), then interface requirements and data requirements may also be required.
This document does not define how the system is to be built. This document sets the technical scope of the system to be built. It is the basis for verifying (via the Verification Plan) the system and sub-systems when delivered.
The target audience for this document is the same as for the Requirements document. This document describes how the system will be tested to ensure that it meets the requirements. Detailed guidance to help you with each chapter is included in Chapter F.
These documents plan, describe and record the activities which verify that the system being built meets the user needs and scenarios developed in the concept of operations, by fulfilling the requirements described in the requirements documents. Usually, for even moderately complex systems, the following three levels of verification documents are prepared:
A critical issue is assuring that all requirements are verified by this activity. This is best done by tracing each requirement into a verification case and then into appropriate steps in the verification procedures.
It will not describe detailed procedures for data collection and analysis, although existing procedures defined elsewhere may be referenced. In general, the procedures that will map to the verification plan will be developed by the system developer, supplier or vendor, and approved by the agency, once the system (or appropriate sub-system) has been installed.
Conducting the verification is generally the responsibility of the system developer, supplier or vendor, although the verification tests may be undertaken by the agency. The verification tests should be witnessed by the agency and the results should be audited by the agency to ensure their veracity.
The target audience for this document is the same as for the Concept of Operation. This document (or set of documents) describes how the performance of the system will be measured to determine whether or not the needs expressed in the Concept of Operations have been met. It will describe the measures of performance that will be used, the data that will be needed and the type of analysis that will be appropriate. This will include a description of the data and analysis that will need to be available from the proposed system, as well as other data collection and analysis that will be undertaken external to the system. Detailed guidance to help you with each chapter is included in Chapter G.
These documents plan, describe, and record the activities which validate that the system being built meets the user needs and scenarios developed in the Concept of Operations. Usually, for even moderately complex systems, the following three levels of validation documents are prepared:
The method of procurement of an ASCT system has a major impact on the ability of an agency to select the most appropriate system for its situation. 23 CFR 635.411 requires procurement of ITS systems and components to be competitive, unless certain conditions are met. However, competitive procurement does not mandate "low-bid" procurement. In fact, in procurement of complex IT and ITS system, "low-bid" is generally inappropriate. The requirement for competitive procurement is satisfied through an RFP or similar process that allows for careful evaluation of the extent to which each requirement is met and the suitability of the method used to satisfy the requirement. The evaluation process may also include consideration of costs to determine best value for money.
The requirements for a complex system may be classified as mandatory, desirable and optional. All should be considered in the evaluation. Very often, "optional" can be used to define requirements that will be important at a future date, but need not be included in the package purchased at this time. This ensures that future system expansion and migration paths are not precluded by the initial system design and capabilities.
Steps in the procurement process that may be applicable to your situation are described below.
Once your requirements have been drafted, there may be some that you are not sure can be met by any, or a sufficient number, of the available systems. This is a point at which you need to consider whether it will be appropriate to purchase a commercially available system (with or without some customization) or develop a unique system. When this occurs, it is appropriate, and perhaps desirable, to release to vendors a formal request for information (RFI) focused on the specific requirements about which you are uncertain. Response to an RFI should not be a condition of future participation by a vendor in the procurement process.
Use of the RFI will allow you to decide whether or not your requirements can be satisfied by a commercially available system. If not, you can then decide whether to modify your requirements or develop a much more detailed set of requirements and a specification that would be appropriate for development of a unique system.
More comprehensive feedback from vendors can be obtained by distributing a draft version of your requirements to vendors. This is appropriate when you have developed requirements that will involve some customization or may include assumptions about appropriate technology. Provided you have included a Concept of Operations and clear statements of need, this provides the opportunity for vendors to contribute in two effective ways.
Vendors will recognize requirements that assume a design that is different from theirs and this gives them the opportunity to redefine the requirement in a manner that does not preclude their system simply because of the wording or structure of the requirement. Vendors will also recognize high-cost customization or new development that would be necessary to satisfy a requirement, and this also gives them the opportunity to suggest alternatives that would substantially reduce the procurement cost or streamline the development.
Responses from vendors should be treated as confidential and not shared with other vendors. If a vendor expects a response to be shared, he is less likely to offer advice that he considers proprietary and part of his competitive advantage.
A request for qualifications (RFQ) is appropriate when there are mandatory requirements placed on the capabilities and experience of the vendor. This is a means of reducing the agency's risk. It provides the opportunity to assess a vendor's financial stability, their track record in providing support and training, and proof that an advertised system is fully operational and successful. It should NOT be used simply to reduce the workload of the agency in evaluating responses to an RFP by limiting the number of vendors permitted to respond to a detailed specification or RFP. It should be used after requirements have been prepared and the agency is certain that vendors who show satisfactory qualifications will be able to also satisfy the technical requirements.
A request for proposal (RFP) is the key vehicle for assessing the extent to which an ASCT system will satisfy the agency's requirements. Using a "best value" approach supported by systems engineering provides the most effective use of the system requirements in the procurement process. As illustrated in Figure 5, the requirements are referenced at every step in the process to help guide the selection. While it may contain mandatory, desirable and optional requirements, the vendor should be required to explain how their system satisfies each requirement, and not simply be permitted to provide a Yes/No or Pass/Fail response. Because of the complex nature of ASCT systems, few requirements can simply be submitted to a Pass/Fail test. There are often different ways in which systems may claim to satisfy a requirement, and not all methods will be equally suitable (or acceptable to an agency) in all situations. Each answer should be evaluated to determine firstly whether or not the Yes/No answer is accurate, secondly the extent to which it satisfies the requirement, and thirdly as to whether the method of satisfaction is acceptable to the agency
In addition, the method by which a vendor satisfies a requirement may have other implications, such as the effect on work practices, staff efficiency, or requiring additional equipment or software to be efficiently implemented. The method used to compare responses to an RFP should include a means of accommodating the assessment of compliance. The agency must also select an appropriate means of accommodating variation in cost between vendors. Two methods are commonly used:
Figure 5. Best Value Supported by SE Analysis
The use of "low-bid" contracting is rarely a satisfactory method of procuring a complex software system such as ASCT as a complete design-bid-build package. However, it is often possible and appropriate to separate the intelligent portion of a system (such as central software and servers, and local controller software) from the physical components that can be clearly and concisely specified, and construction work that can be undertaken by a qualified contractor.
If this approach is adopted, it is generally appropriate to first use the RFP process to select the ASCT system and vendor. Then the specifications can be completed for the equipment and services to be procured under a low-bid contract with confidence that there will not be compatibility issues between field equipment and the ASCT system.
When purchasing a product, if it is tightly specified, then there is limited risk that product will fail to meet the purchaser's expectations. However, when procuring an ASCT system, much of what is being purchased are services, not simply a product. If an agency is restricted to a traditional "low-bid" process, a strategy that builds sound systems engineering into the rigid procurement process (illustrated in Figure 6) would include the following:
It is critical that the compliance with the requirements is demonstrated as early as possible, preferably before any of the project budget is spent.
Figure 6. Low-Bid Supported by SE Analysis
A very important component of the procurement plan is the verification plan. It must be prepared before a formal RFP is issued and, regardless of the procurement approach you have selected, before a contract is signed. The verification plan should set out the method by which each requirement will be verified as satisfied, who will undertake verification tests and when within the process each verification test will be conducted. The plan will also include a test and verification matrix that will identify which requirement is verified by each test.
It may be appropriate for compliance with some requirements to be demonstrated prior to selection of a vendor or system, particularly for mandatory requirements that cannot be demonstrated by reference to existing operation with other agencies. For example, if you require an ASCT system to operate with flashing yellow arrow for permissive left turns, and a candidate system has not previously demonstrated such operation successfully, it is appropriate to require this demonstration before finalizing your system selection.
The verification plan should be provided to potential vendors with the RFP. In general, the test procedures are not part of the verification plan at this stage. The procedures can only be written after the ASCT system has been selected and any customization designed. Note that it is usually most appropriate (and cost effective) to schedule tests to occur at various stages during the project and not leave all testing until after installation is complete.
A commonly used but often less successful approach is to begin with market research and employ a low-bid process, illustrated in Figure 7. A detailed set of specifications is prepared based on a list of features and capabilities identified during the market research. It is extremely difficult to avoid specifying unique technology using this approach and the only control over brand choice comes at the submittal stage. The unfortunate experience of many agencies has been that many requirements are only discovered during the acceptance phase at the end of the construction period. This approach has been used for many of the adaptive installations whose final operation has not been satisfactory to the implementing agency.
Figure 7. Market Research / Low Bid Approach
For further guidance on procurement practices for ASCT and ITS projects, consult the following references.
Special Experimental Project 14 (SEP 14): https://www.fhwa.dot.gov/programadmin/contracts/sep_a.cfm
The Road to Successful ITS Software Acquisition: https://www.fhwa.dot.gov/publications/research/operations/its/98036/rdsuccessvol2.pdf
For further guidance on preparation of verification plans, see:
Systems Engineering for Intelligent Transportation Systems, A Guide for Transportation Professionals http://ops.fhwa.dot.gov/publications/seitsguide/section4.htm#s4.7
Systems Engineering Guidebook for ITS https://www.fhwa.dot.gov/cadiv/segb/
In the event that only one system fulfills the requirements, and when Federal funds are used, the acquisition process will be governed by 23 CFR 635.411.2 This regulation governs the acquisition of proprietary materials. In this case, certification from the FHWA is required that one of the following conditions is true:
A Public Interest Finding (PIF) for proprietary purchases is not required in these cases. A PIF is required when the proprietary product is not the only product that fulfills the requirements. It is recommended that if more than one product fulfills all requirements, they should be considered competitively.
Proprietary acquisition is also possible for limited experimental application. This might apply to an adaptive system being installed as a pilot project to determine suitability for broader implementation. In this case, you should develop an experimental plan showing what you hope to learn from the experiment that you do not know before the experiment is conducted. Experimental application cannot be used to circumvent the requirement for system engineering.3 If you have used proprietary acquisition to install a small pilot installation of adaptive operation and you wish to now expand adaptive operation to become a much larger system, a PIF will not be issued simply because you already have a small proprietary system, if other systems may fulfill your requirements.
2 Federal Highway Administration, "Construction Program Guide" web page, available at: https://www.fhwa.dot.gov/construction/cqit/propriet.cfm [ Return to note 2. ]
3 Federal Highway Administration, "Construction Projects Incorporating Experimental Features" web page, available at: https://www.fhwa.dot.gov/programadmin/contracts/expermnt.cfm [ Return to note 3. ]
United States Department of Transportation - Federal Highway Administration