The systems engineering approach discussed in previous chapters may be viewed as an extension to the traditional project development process that is already established in transportation agencies. As transportation organizations gain experience with ITS projects and the systems engineering approach, they typically find that they can weave the systems engineering processes and best practices into their overall project development process.
Transportation projects are identified and funded through transportation planning and programming/budgeting phases. Funded projects are then implemented using a process similar to the traditional capital project development process shown in Figure 36, but the exact process used for ITS projects will vary with the type of project. For example, ITS projects that install only field equipment (e.g., variable message signs) would use a process that is very close to the traditional process shown in Figure 36. ITS projects that involve hardware and software development and integration would require additional systems engineering analyses that would be significant extensions to the traditional process.
Figure 36: The Traditional Transportation Project Development Process
While project development processes vary from state to state and from organization to organization in each state, the transportation project development process tends to have the same major steps.
As it has evolved through a century of building roads and public transit systems, the transportation project delivery process used by most agencies today already includes many important features of the systems engineering process. In both processes, the system is specified in increasing detail, beginning with needs, moving to requirements, and then into design. Multidisciplinary project teams and systematic stakeholder outreach and communications are hallmarks of a good transportation project development process and sound systems engineering practice. By taking advantage of this similarity in concepts and processes, the systems engineering process can be integrated as an extension to the agency's existing project development process. This alignment of the traditional transportation project development process with the systems engineering process is shown in Figure 37.
Making these types of linkages and mainstreaming ITS development into the agencies' project development process makes it easier to incorporate systems engineering into each agency's process.
Although there are similarities, there are also key differences between the traditional process and the systems engineering approach that should be considered when planning your next ITS project. For example, in the traditional transportation project development process, there is clear contractual separation between the consultant that prepares the PS&E and the contractor that builds the project. This is a risky approach for many ITS projects, in which it is important to have more continuity across the project development life cycle so that the contractor who ultimately implements the ITS system clearly understands the underlying user needs and requirements and has the latitude to implement the most cost-effective solution. For example, the contractor that implements custom software for an ITS project should also participate in the detailed software design. You would not want to impose a contractual barrier between the software designer and the software implementer. Differences like these have led to use of innovative procurement practices for ITS projects that are discussed in the next section.
Figure 37: Systems Engineering as an Extension of the Traditional Project Life Cycle
The project development process is strongly influenced by the selected procurement strategy. ITS projects have been procured through traditional low-id, systems manager, design-build, and other innovative procurement approaches. The procurement strategy will influence who (i.e. agency, consultant, or contractor) takes the lead for each process step, but the fundamental systems engineering process steps (Concept of Operations, Requirements, Design) should still be accomplished for all types of ITS system development projects. An important thing to remember is that the agency is never completely off the hook, regardless of the procurement approach. With any approach, there is always a need for the agency to be involved in the process.
The systems engineering analysis requirements identified in FHWA Rule 940.11/FTA Policy Section VI include a requirement for analysis of procurement options.
A poorly chosen procurement strategy can adversely impact a project just as much as the lack of a sound systems engineering approach. It is important to tailor the procurement strategy based on the type of ITS project and not to assume that it always has to be done the same way. The traditional approach of putting a specification on the street and awarding the implementation to the low-bid contractor may work well for projects with extremely well-understood requirements. However, the complications of inexperienced personnel, new requirements and technologies, changing environments, and shifting priorities lead to the need for newer approaches for many ITS system acquisitions. This is particularly true when a project includes custom software development.
Table 20 identifies some of the procurement approaches that have been used for ITS projects and highlights a few of the considerations for their use in your next project.
Table 20: Procurement Approaches for ITS Projects
|Commodity Supplier||Low-bid selection of prequalified packages; fixed-price contract||Applicable only for unmodified off-the-shelf software or hardware|
|Low-Bid Contractor with Consultant Design||
Consultant performs 100% of design; may provide additional services during implementation
Low-bid selection of contractor; fixed-price contract
No collaboration on design between agency and contractor
Conventional way to procure construction projects
Only applicable to extremely well-defined ITS projects
Not applicable to ITS projects with significant software development
Single contractor responsible for planning, design, implementation, and testing
Negotiated (qualifications-based) award precludes contractor from providing construction/equipment
Separate low-bid procurements managed by the agency for construction and equipment
The selected systems manager has overall responsibility for delivering an operational system
Additional contracting burden on agency
Accommodates iterative development and collaboration between agency and contractor
Preferred for projects with significant software development
|Design-Build Contractor with Design Consultant||
Requirements/high-level design may be prepared by a consultant prior to contractor selection
Best-value contractor selected based on price and qualifications
Single contractor responsible for design, implementation, and testing
Contractor can provide construction services/equipment
Some agencies do not allow design-build contracting
Loss of continuity/flexibility if contractor does not participate in requirements/initial design
Do not use a low-bid process to select a design-build contractor
Preferred for major projects with significant construction
Negotiated (qualifications-based) award limits consultant work to personal services
Consultant services used for system requirements, design, and agency support during system implementation (define test plans, etc.)
Used to support initial consultant/system manager selection in previous approaches
Can also be used to supplement in-house capabilities, support systems engineering process improvement, etc.
Used to support acquisition of a capability/function rather than a specific system
Best-value (qualifications-based) or low-bid (cost/rates-based) selection
Outsourced functions may include traveler information, toll collection; may involve a public-private partnership
A hybrid approach may be preferable for some ITS projects. For example, systems that include significant software development and significant construction may best be implemented with a hybrid approach that includes separate contracts for each of these activities.
Obviously, selecting the "best" procurement approach for your ITS project and your agency is not a trivial matter. Fortunately, there is help. The Guide to Contracting ITS Projects and its companion web-based tool, available at http://www.citeconsortium.org/Model /index.htm, have been developed through the National Cooperative Highway Research Program (NCHRP). The documentation provides insights into the procurement processes for ITS and advises on how to make the right choices based on the particular circumstances of a project and the procuring agency. The web-based tool walks you through an eight-step decision model, beginning with initial decisions about the type of project and finishing with recommendations on how to build the contractual terms and conditions.
The contracting model includes decisions about work allocation, method of award, and contract form and type. Each step in the model consists of a series of questions to elicit the many possible dimensions, including project size and complexity, staff experience, and agency policies.
In a recent Talking Technology and Transportation (T3) webinar, Successful ITS Procurement, Mac Lister of the FHWA Resource Center and Phil Tarnoff of the University of Maryland concluded that "The right procurement approach may not guarantee success, but the wrong approach will guarantee failure." The key is to keep asking "What are we buying?" The answer to that question, along with any uncertainty you may have in answering it, should guide you in selecting the right approach for you and your project.
There are several different ways that a system can be developed and delivered using the "V" systems engineering model. The best development strategy depends on how much you know about the system that you want to implement, whether you have all the funds that you need to implement the system in one fell swoop, your agency and contractor capabilities, and your assessment of the project risks. Three basic development strategies can be used:
There are many variations and hybrid combinations of these three basic strategies. Table 21 summarizes the distinctions among them.
Table 21: Development Strategy Comparison19
|Development Strategy||Define All Requirements First?||Multiple Development Cycles?||Deliver Interim Releases?|
Table 22 identifies some of the reasons to pick one strategy over another. In general, the once-through strategy is well suited for low-risk projects and the evolutionary strategy is more adaptable and well suited for higher-risk projects where there are significant unknowns.
Table 22: Development Strategy Opportunities and Risks20
(Reasons to Use)
(Reasons to Avoid)
Chapters 4 and 5 described the systems engineering process steps in some detail. On some projects, a given step may be performed very informally (e.g., on the back of an envelope, or in an engineer's notebook); on other projects, the activity may be performed very formally, with interim products under formal configuration control. This document is not intended to advocate any level of formality as necessary or appropriate in all situations.
The systems engineering analysis requirements identified in FHWA Rule 940.11/FTA Policy Section VI allow the systems engineering analysis effort to be tailored so that it is on a scale commensurate with project scope.
INCOSE also stresses variation in the systems engineering process:
Like all processes, the Systems Engineering process at any company should be documented, measurable, stable, of low variability, used the same way by all, adaptive, and tailorable! This may seem like a contradiction. And perhaps it is. But one size does not fit all.21
This message is particularly important for ITS projects because so many of our projects are smaller, less complex, less risky projects like signal system upgrades. Even for small projects, you still should have documented requirements, design, and verification procedures. Tailoring isn't an invitation to skip steps. Tailoring allows you to adjust the amount of formal documentation and reviews and to focus the process on those steps that are most critical to your project's success.
In order to decide on the process that is appropriate for your project, you should perform a risk assessment to understand the complexities involved and how many unknowns there are. Some ITS projects are much larger and more complex than others, which makes them a greater risk and thus candidates for more rigorous processes, as shown in Figure 38. Ultimately, you want to define a process that will address the project's risks, no more and no less, so a preliminary risk analysis is a good way to determine how much process is appropriate.
Figure 38: Tailoring Process Based on Risk
To get you started, Table 23 lists some of the project characteristics that affect project risk. The table shows, for example, that the number of stakeholders involved in a project is an indicator of project complexity and risk. A project that involves only a single agency/department will tend to be lower risk and require less effort to develop a Concept of Operations and user requirements than a project that involves multiple agencies. There are a couple of disclaimers that should be made about the table. It includes rules of thumb that may not apply to your project. For example, user requirements can still be a challenge even for a single agency/department project if there is no consensus within the department. Also, your project will likely fall between the two "lower risk" and "higher risk" extremes in many areas.
As an example of how the systems engineering approach might be tailored for different types of projects, the California Systems Engineering Guidebook for ITS22 uses three typical projects:
Table 24 summarizes the tailoring information that is included in the California Guidebook for the three ITS projects. (Refer to the California Guidebook for a more comprehensive analysis of the three example projects.)
Of course, your project won't exactly match any of the three examples, and even if it did, you would want to take into account your own environment, process requirements, and staff experience when you tailor the process for your own project. The best approach is to think about the risks of your particular project and determine how best to mitigate them with a tailored systems engineering process. Think about the process ahead of time and write down what you are going to do so that everyone on your team and your stakeholders understand and agree on the right steps to follow and the level of detail that is needed. Whether you call it a project plan, a systems engineering management plan, or something else, it's critical to put your process and your plan in writing.
Table 23: Tailoring the Process Based on Project Risk
|Risk Factor||Lower Risk||Higher Risk|
|Project is/includes:||Process Tailoring||Project is/includes:||Process Tailoring|
|Project Type||Expansion/ enhancement||Review/use/modify existing SE documentation.||New system development||Create new SE documentation.|
|Project Size||Small (e.g., <$300K)||Use less formal project controls. You still need to plan the work, manage changes, and monitor and control risk, but the number of formal plans can be consolidated or significantly reduced to fit the budget.||Large (e.g., >$5M)||Use more formal project controls (planning, monitoring and control, risk management, configuration management) with documented formal plans for each aspect.|
||One agency/department||Less formal reviews/walkthroughs, but buy-in is still important. Basic ConOps considers operations and maintenance, roles and responsibilities.||Multiple agencies||Emphasize reviews and walkthroughs. More focus on ConOps, including specific roles and responsibilities, operational scenarios, etc. More time specifying/ reconciling/prioritizing user requirements.|
||Isolated system||Less formal high-level design – just make sure you understand the parts of your system and internal interfaces.||Regional integration||Focus on architectural design and interagency interface definitions and agreements.|
||Proven technology||Adapt available, proven, performance-based design specifications.||State-of-the-art technology||Increase focus on feasibility studies, prototyping, trade studies/alternatives analysis. Perform studies prior to development contract.|
||Off-the-shelf procurement||Use higher-level performance-based design specifications.||Custom hardware/ software development||More focus on requirements analysis and modeling, architectural design, detailed design, and implementation process.|
||Downtime acceptable; neither safety nor security critical||Little/no specialty engineering required.||High availability required; safety and/or security critical||Add specialists to the team. Additional focus on process and quality assurance throughout development cycle.|
|Project Understanding||Good understanding of needs/requirements||Single iteration through the process to deliver complete system may be most efficient.||Needs/requirements unknown||Evolutionary, incremental approaches work best. Deploy a little, learn a little, and repeat.|
|Staff Experience||Staff experienced with similar projects||Use established processes. Use in-house systems engineering if appropriate.||Staff has no experience with similar projects||Use systems manager/consulting support. Additional oversight, staff development.|
Table 24: Systems Engineering Application Examples23
|Process Step||Example 1: Add DMS to Existing System||Example 2: Add Shared DMS Control||Example 3: Upgrade Existing Signal System|
|Regional Architecture||None||Existing architecture mapping applies||Low||Add new interface to existing mapping||Low||New mapping|
|Feasibility Study||None||Original feasibility study applies||Medium||Assess feasibility of adding new interface to existing system||Medium||Survey existing system and alternatives|
|Concept of Operations||None||Existing ConOps should apply||Medium||Focus on O&M responsibilities shared between agencies||Low||Define how system will operate and performance measures|
|System Requirements||None||Existing SRS applies – verify capacity||Medium||Define requirements for shared control, secure remote access||Low-medium||Short requirements document and verification plan|
|High-Level Design||None||OTS products; existing specs apply||Medium||Define architecture for shared control and define interfaces||Low||Identify legacy components, new components, and interfaces|
|Detailed Design||None||OTS products; existing specs apply||Medium||Specify SW design||Low-medium||Performance-based controller specs, central computer specs to support OTS procurement|
|Software/Hardware Development||None||Verify no update to SW necessary||Defined by SEMP/Plan||Custom SW; purchase COTS SW, servers, and comm HW||None||All components OTS|
|Unit/Device Testing||None||Verify factory tests performed||Defined by SEMP/Plan||Unit test custom SW; check out purchased HW and COTS SW||Defined by SEMP/Plan||Central computer tested; bench tests on controllers/firmware|
|Subsystem Verification||Low||Verify controller, signs, and comm working||Defined by SEMP/Plan||Host/integrate custom SW on HW||Defined by SEMP/Plan||Integrate/verify central computer to controller interface and verify functionality|
|System Verification||Medium||Reuse original procedures||Defined by SEMP/Plan||New procedures for new capabilities; all documentation ready||Defined by SEMP/Plan||Perform system-level acceptance tests; all documentation ready|
|Initial Deployment||Medium||Normal construction issues||Defined by SEMP/Plan||Staff trained; system installed and configured||Defined by SEMP/Plan||Deployment already occurred; support cutover/fallback|
|System Validation||None||System validation already performed||Defined by SEMP/Plan||Assess effectiveness; before-and-after study||Defined by SEMP/Plan||Validate performance, user, and maintainer satisfaction|
The three-legged stool shown in Figure 39 is a well- known metaphor for the three key leverage points that an organization has to improve its capability and efficiency in any area. The three legs that are shown are the major determinants of cost, schedule, and quality: the people in the organization, the technologies/tools that they use, and the processes and practices that are established. This metaphor is useful here as we consider the ways that an organization can improve its systems engineering capability.
Figure 39: Quality Leverage Points
The stool metaphor is a good one since there has to be balance across each of these leverage points in the same way that all three legs of the stool have to be the same length. For example, if you add technology and begin to use a sophisticated systems engineering tool but have not trained the people that will use the tool or established some best practices for its use, then it is likely that the organizational capability and efficiency will actually decrease as users struggle with the tool and spend less time performing productive tasks. The following three sections briefly discuss each of the three leverage points for improving the systems engineering capability in your organization; keep in mind that there must be a systematic plan for increasing the organization's systems engineering capability over time that balances all three of the leverage points.
People represent a collection of skills, and a wide variety of skills are needed to successfully employ the systems engineering process in your organization. Every ITS project should have access to systems engineers who are familiar with the agency's processes and can tailor them to reflect the project size and level of complexity. The systems engineer will work with stakeholders to define the needs of the system, Concept of Operations, and system acceptance criteria. They will establish the requirements and specify and test the system. Whether this set of skills is in-house or contracted out, you should have a reasonable understanding of the systems engineering process and what can and should be expected during the project life cycle.
One good way to build competency into systems engineering is to identify one or more systems engineering specialists within your organization. These specialists would be a shared resource that could begin to make the application of systems engineering more consistent across projects. Depending on the size of your organization and the number of ITS projects, this could be a part-time job, a full-time job, or the responsibility of a small group. Many agencies have used their Information Technology group as a source for systems engineering and information technology skills.
In addition to identifying a few specialists, the organization should make a more general commitment to improve skills in project management and systems engineering for ITS projects. Many organizations have encountered the pitfall of using a project manager who is skilled at traditional transportation projects but is ill prepared to manage a high-technology ITS project. Even if much of the systems engineering work is contracted out, all members of the project team should be trained so they can be effective participants in the systems engineering process. One of the recurring themes in this document is that broad stakeholder participation in the systems engineering process is essential.
Short courses in systems engineering are available through the National Highway Institute (NHI) and the Consortium for ITS Training and Education (CITE). (See Section 7.4 for more information.) A few larger agencies, including Caltrans and Florida DOT, have begun to implement systems engineering training that is more tailored to their specific processes. One benefit of implementing a common systems engineering process within your organization, as described in Section 6.3.2, is that it allows you to use a common training program that will be applicable to all agency ITS projects.
Note that the latest transportation legislation allows 100% use of federal funds for training and education. Specifically, SAFETEA-LU Section 5204(e), "Surface Transportation Workforce Development, Training, and Education", allows states to use 100% federal funding from the five core program areas for professional development activities. For example, this would allow you to use part of the funds from an ITS project to send agency project managers to systems engineering training or to host a commercially available systems engineering course at your facilities. For more detail on this opportunity, contact your FHWA Division Office Specialist.
One key to systems engineering is the use of a repeatable process or set of processes. As your organization employs systems engineering practices more and more, you should sit down with your colleagues at the conclusion of each project and determine what worked well and what should be done differently the next time. Start to document the processes you used during the project life cycle, and modify them based on your lessons learned. You'll want to pay extra attention to any project reviews you held to measure their quality. After you've developed a good set of processes that have been used on several projects, obtain agreement from your organization to establish processes for your organization. This will require buy-in from senior management, who should establish a policy that will support the processes and guidelines for their use.
Be sure to look at existing processes that are already defined in your organization. You may find that the Information Technology group has already implemented a good set of processes, many of which can be applied to ITS projects. The processes that are used for traditional capital development projects may also be helpful. For example, many agencies already have good, established project management processes such as risk management, establishment of integrated project teams, and value engineering processes, any of which may be equally applicable to ITS projects. When processes apply to many types of projects, it makes sense to define a single organizational process that can be applied to ITS projects and other types of projects.
There are many excellent resources that can help you to build systems engineering processes into your organization's standard business practices. The best-known resource is the Software Engineering Institute's (SEI's) Capability Maturity Model Integrated (CMMI). CMMI is a good place to start since it reflects process improvement experience and lessons learned from broader industry and includes a wealth of processes and best practices for systems engineering, software engineering, and acquisition, all in a single unified framework. The staged representation of CMMI, shown in Figure 40, is instructive because it shows a proven approach to process improvement. You start with the right people trying to do the right things at Level 1, move to documenting the process on a project-by-project basis at Level 2, and then implement these tested processes for the organization at Level 3. This is a natural progression that results in good processes that support the organization's business needs and are supported by upper management and organizational policy. Each CMMI maturity level provides a foundation for the level above it.
Figure 40: CMMI Maturity Levels
Once you have defined processes for the organization, the process that is used on a particular project will be based on the organizational process and tailored in a systems engineering management plan (SEMP) or similar document. Once all projects use tailored versions of the common process, the organization can begin to compare results, share lessons learned, and transfer people more easily between projects. Consistent historical data can be collected and used to improve estimates and consistency. A commitment on the part of the whole organization to use these processes, eliminate perceived shortcuts, and continuously improve the process will result in better-quality systems.
Documentation standards go hand in hand with organizational process standards. Document templates and formatting standards will save time and result in more complete and consistent documentation that can be shared more easily between projects. Documentation templates define the scope, content, and structure of each deliverable document. Like organizational process standards, standardized document templates can be tailored for each project. If your organization does not already have a library of such standards, they can be created based on industry standards and existing guidance. For example, the California Systems Engineering Guide and the Florida Systems Engineering Management Plan both include useful systems engineering documentation templates. (See Chapter 7 for potential resources.)
Like the systems engineering process itself, the use of systems engineering tools should be scaled to meet the needs of the project. The right suite of tools for a Transportation Management Center project would be overkill for a website upgrade. Systems engineering for small ITS projects is routinely done with standard desktop applications – a word processor for documents, a small database or spreadsheet for requirements management, and a drawing tool such as Microsoft Visio to generate the graphics and diagrams. For larger projects, more specialized systems and software engineering tools will allow you to do a better job of managing the larger set of requirements and more complex design artifacts that may be worked on in parallel by several engineers. Similarly, project planning, monitoring, and control can be handled with a tool like Microsoft Project for small-to-medium-sized projects. More sophisticated project tracking and control tools, document management, and configuration management tools will help you to better manage larger ITS projects.
Defining a standard suite of tools for your organization can facilitate sharing of information and people between projects. Of course, when the systems engineering effort is contracted out, the consultant/contractor may have their own preferred tool suite. Larger agencies have started to standardize on tools such as requirements management tools and to require that contractors use them so that the agency staff does not have to be fluent in the range of tools that contractors might otherwise select. If such a standard suite is defined for your agency, there should be access to training for the tools and guidance for applying them to an ITS project.
Although many good tools are readily available, you should weigh the introduction of a new tool and its resultant learning curve against its potential efficiencies. One agency recently experienced more than a year of delay as a sophisticated project management tool was procured, installed, and then found to require extensive customization to support the agency's business practices. Any major tool acquisition should itself be supported by a systems engineering analysis. Make sure you understand the needs, validate the requirements for the tool with prospective users, and then do a trade study of the alternatives. After acquisition, the new tool should be used on a pilot project or two before it is used more broadly within the agency.
TOC Previous Page Next Page