3.4.4       Risk Management

Introduction - Risk Management

Although ITS projects come in many shapes and sizes, they all use technology (computers, communications, sensors, etc.) and frequently include the exchange of information between systems. The technology and integration that sets ITS projects apart also creates challenges for the ITS project manager. Every ITS project manager wants a successful result at the end of the project, with “success” measured by:

• how well the implementation satisfies the needs of the people who use it, and
• how closely the project stayed within the budgeted cost and schedule.

The effects that can compromise success are collectively referred to as “risk”. A key value of systems engineering is to manage the risk on a project. The following is a brief discussion of risk.

ISO 17666:2016 defines risk as an “undesirable situation or circumstance that has both a likelihood of occurring and a potentially negative consequence on a project”. Other definitions equate risk to variability or to the chance that desired outcomes will not be achieved. The New Zealand transportation agency is an international leader in risk and asset management and it defines risk as “the chance of something happening that will have an impact on objectives. it is measured in terms of a combination of the likelihood of an event and its consequence.”^[1]

There are many areas of risk that should be considered when deploying an ITS project. These areas, along with some questions relevant to understanding how significant the risk might be, include:

• Technical (e.g., Is the project using any technologies that have not been widely deployed or that the project team is unfamiliar with? Are the requirements well defined?  Are the development or test facilities inadequate?  Is all technical documentation receiving adequate review?)
• Schedule (e.g., Is the schedule aggressive?  Are there particular tasks for which small schedule slips will have a major impact on the final deliverable?  Is the schedule dependent on timely delivery of equipment, information, or review from parties that may not be bound by the project schedule?)
• Cost or Funding (e.g., Is the budgeted cost realistic for the planned systems?  Is funding for the project secure, or is only part of it in place?  Are there pending agency budget cuts that might impact development or operations?)
• Institutional (e.g., Does the project require agreements related to agency data sharing that haven’t yet been created?  Are there regulations or agency hurdles that must be overcome for the project to succeed?)
• Personnel (e.g., What will happen if there is a loss of key agency or contractor personnel?  What will be the impact if key personnel do not have adequate experience?)
• Environmental (e.g., Does the deployment schedule take into account typical seasonal conditions- call for installations at a typically rainy time of year, or during winter months?  Are there environmental restrictions that might impact system deployment?)

 

Description

Risk management seeks to help a project avoid impediments to successful completion in keeping with the project environment. Similar to the management of the project, the management of risks needs to be planned and executed. Unlike project management, the different tasks associated with risk management are expected to be performed concurrently and repeatedly. A representative risk management process is shown conceptually in Figure 27. Risk management is a developed area with existing standards and reference material. (Technical Committee ISO/TC 262, 2020)

Figure 27: Representative Risk Management Process

(Source:ISO/TC262)

To be valuable to a project, risk management must identify the relevant risks, assess the threat that the risk poses to the project, and mitigate risks that are determined to be of significant risk to the project. The value of mitigating a risk must be assessed against the impact, likelihood, and urgency of the risk occurring compared against the available resources of the project to mitigate the risk.

Risk management requires the support of the project team, including both management and staff. A successful risk management effort receives the support of project management to assign risk management activities with sufficient staff resources, to make risk management an ongoing effort, and to foster an environment where project staff communicate accurately with risk management staff. The risks to the project are most directly known by the line employees who will design, implement, or operate the systems or services implemented by the project. The staff assigned to risk management are most effective in risk identification when the line employees are empowered and encouraged to directly communicate with risk staff.

Risk management inherently has an adversarial component with the project team. The risk analysis performs a critique of the ability of the project team. Any risk that is identified can be interpreted as a criticism of the resources of the team. The assessment of a risk as highly likely to occur can be interpreted as questioning the skills or decisions of team members. Risk mitigation can also be viewed as an additional assignment that comes on top the assignments of an already-overworked staff member.

 The value of mitigating a risk should be assessed against the resources to be applied to the mitigation effort in determining if the mitigation effort should be undertaken. Any time that management determines that a risk must be mitigated, team resources are diverted from their ongoing assignment to the mitigation assignment, which can impact the cost or schedule of the project. Mitigation efforts can represent an opportunity cost related to the portions of the project delivered late or foregone.

In a full effort for risk management for a project with sufficient resources, dedicated staff should be assigned to perform activities related to risk identification, risk analysis, and risk mitigation, along with maintaining a record of each risk throughout the project. Such a level of effort should be implemented for projects with high-risk characteristics both in likelihood of risk occurrence and impact of occurrence. Risk management in this scenario requires ongoing commitment from project management to regularly review the evolution of the risk register and apply project resources to resolve issues determined as posing a threat to the success of the project.

The efforts in performing risk management should be tailored in response to a lower-risk project environment. For a project that initially is determined to be low risk, the risk identification may be performed by project management staff speaking with designers, implementers, and future operators. Risk management should continue throughout the project, with the possibility of risk mitigation continuing until risks posing significant threats to the project are no longer relevant. Limiting the risk management effort for a project with limited resources is a risk in itself.

Are there any other recommendations that can help?

All useful systems incur some risk. The goal is a balance between system performance and risk. That is why the focus is on only the most critical risks. Lesser risks will and should be accepted.

From a management viewpoint, there are four ways to handle risk.

1] Mitigate the risk by allocating contingency funds to its resolution if it becomes necessary

2] Accept a risk that cannot realistically be mitigated, such as an earthquake

3] Avoid the risk by changing the requirements or design

4] Transfer the risk [e.g., to an insurance company or to a developer under a fixed price contract].

Even if a dedicated risk management team is in-place, everyone on the team must be encouraged to identify potential risks. A “shoot the messenger” atmosphere will only allow hidden risks to grow out of control.

Uncertainty is what makes risk management both difficult and essential. There are statistical techniques such as probabilistic decision theory for reasoning under uncertainty. The most basic technique is expected value. Risk is computed as the probability of occurrence multiplied by the consequence of the outcome. Probability is between 0 [minimal] and 1 [certain]. Consequence is expressed in terms of dollars, features, or schedule. Multiplying probability of occurrence and consequence [impact analysis] together gives a risk assessment value between 0 [no risk] and 1 [definite and catastrophic].

When exact data is not available for expected costs and probabilities. One can get reasonably good results simply by rating risks qualitatively relative to three to four categories in each of impact (rows) and likelihood (columns). Below is an example of the matrix used for such an evaluation. The numbers are the order in which the risks are to be considered. The lower the number the higher the priority of the risk.  This table is just for illustration purposes, and in the real world its unlikely any project would proceed with risks in the categories 1-3 in the table. 

 

Impact	Likely 0.7-1.0	Probable 0.4 to 0.7	Improbable 0.0 to 0.4
Catastrophic 0.9 to 1.0	1	2	6
Critical 0.7 to 0.9	3	4	8
Marginal 0.4 to 0.7	5	7	10
Negligible 0 to 0.4	9	11	12

 

A closer look at definitions and examples of impacts and probability ratings

Here are definitions to firm up the impact levels used in the matrix [from INCOSE Systems Engineering Handbook]. Here the “mission” is the purpose of the system such as traffic management.

·        Catastrophic: Failure would result in project failure meaning a significant degradation/non-achievement of technical performance.

·        Critical: Failure would degrade system performance to a point where project success is questionable, for example: a reduction in technical performance.

·        Marginal: Failure would result in degradation of secondary system functions, a minimal to small reduction in technical performance.

·        Negligible: Failure would create inconvenience or non-operational impact. No reduction in technical performance.

Here are examples of some of the characteristics that would impact the probability of failure [adapted from INCOSE Systems Engineering Handbook].

Maturity:

·        Existing system, probability is 0.1

·        Minor redesign, 0.3

·        Major change [feasible], 0.5

·        Complex design [technology available], 0.7

·        State of the art [some research done] 0.9

Complexity:

·        Simple design, 0.1

·        Minor increase in complexity, 0.3

·        Moderate increase in complexity, 0.5

·        Significant increase in complexity, 0.7

·        Extremely complex, 0.9

Note that if there are multiple risks. The overall probability will be at least as high as the highest of them. Often it will be even higher.

Managing Risk with Systems Engineering

At its core, systems engineering is a set of activities that seek to reduce risk, particularly the risk that the resulting system will not support the intentions of the agency, but also including the risk of schedule and cost overruns. Systems engineering, like all good engineering processes, increases the likelihood that the implementation will fulfill the user’s requirements. Systems engineering processes do impose initial costs to a project, and the level of systems engineering required by any given project should be proportional to the risk imposed by the project.

The discussion below provides examples of risk mitigation strategies for four key categories of projects, as well as providing a guide to the type and amount of systems engineering that needs to be done. A further discussion of how to tailor systems engineering activities to the risk level of a project is given in Section 4.3.  The four categories of projects are:

• Projects that will develop new software or hardware technology, develop new agency operational practices, or develop new interagency interfaces.
• Projects similar to previous successful projects for which systems engineering was performed.
• Projects that fit into a category of projects for which the same systems engineering documents can be developed and applied to the whole category
• Projects that do not require development work that will select from existing technology products only, including those covered by Model Systems Engineering Documents (see discussion below).

 

New Development Projects

These represent the greatest risk, particularly when an agency is undertaking a new operational activity as well as developing a new technology. For these projects a comprehensive approach for performing and documenting systems engineering from scratch is important for the management of risk for projects in this group. While traditional ITS technology is now common and available as finished products, systems software is still subject to considerable integration, and newer operational approaches are emerging, such as technologies related to connected and automated vehicles. Many of these technologies are not developed to the stage of commonly available products and software, and the detailed custom systems engineering described in Chapter 3 will be applicable.

 

Projects That Re-Use Systems Engineering Documents

Agencies often implement technology projects that extend or duplicate work done in prior projects. They support the same activities that are performed by the same stakeholder agencies. In these cases, those documents provide a complete and correct statement of needs and, particularly, requirements for the new projects. In these cases, agencies should simply reuse the previous documents.

 

Agencies may take the documents for a prior project and make the necessary minor changes, and then use them specifically for the project at hand. The extent to which the needs and requirements contained in those documents avoid being specific about technology governs their reusability to later projects.

 

Projects that fit into a category of projects

 

Another way to spread the applicability of one set of documentation is through the use of categorical systems engineering. For example, if an agency plans to build a number of systems in different locations as a series of disconnected projects, but all the projects respond to the same needs and fulfill the same requirements, then one set of systems engineering documents can be applied to all the projects. In one example, the Ohio Department of Transportation defined categories of projects, such as small-scale traffic signal systems, all of which are owned and operated by one agency whose arterial operational approach using those systems is the same for all projects. The needs and requirements for the category fully apply to each project. Minnesota DOT has adopted a similar approach, where lower-risk projects in defined categories can use systems engineering documents that are already developed, while more complex and risky projects can use templates to guide the development of project-specific systems engineering documents.

 

Product Selection Projects

Many projects do not involve development of product technologies, either hardware or software, and the principal decision during deployment is selecting the products to be used. These sorts of projects deploy the most common basic technologies used in operations, including traffic signals, dynamic message signs, closed-circuit traffic monitoring cameras, and detection and traffic sensor systems. For these, FHWA has developed a library of Model Systems Engineering Documents. These documents provide a model concept of operations, from which an agency user will simply select the needs that are appropriate to the project at hand, with minimal required tailoring. Once the needs are selected, appropriately traced and worded requirements are associated with those needs that can be used for procurement and acceptance of the products in a project setting. As of this writing, FHWA has published the following Model Systems Engineering Documents:

 

·        Model Systems Engineering Documents for Traffic Signal Systems (working draft), available for download here: https://ops.fhwa.dot.gov/publications/fhwahop19019/index.htm

·        Model Systems Engineering Documents for Dynamic Message Signs (final), available for download here: https://ops.fhwa.dot.gov/publications/fhwahop18080/index.htm

·        Model Systems Engineering Documents for Closed-Circuit Television (final), available for download here: https://ops.fhwa.dot.gov/publications/fhwahop18060/index.htm

·        Model Systems Engineering Documents for Transportation Sensor and Detection Systems (in development as of this writing)

 

These approaches help minimize the development of unneeded documents, but still provide meaningful benefits for mitigating the risk of technology deployment projects.

 

Checklist

The following checklist can help answer the question “Are all the bases covered?” once the activities above have been planned or performed.  The checklist is not sufficient by itself.

R  Is the risk management plan included in the Project Plan/SEMP?

R  Have all sources of risks been identified?

R  Technical [e.g., new detectors do not perform as expected]

·        Institutional [e.g., agency data sharing, new regulations, public opposition]

·        Funding [delays or cuts]

·        Environmental [e.g., temperature levels for outdoor field equipment, restrictions on building]

·        Personnel [e.g., loss of key personnel, substandard performance]

·        Commercial [e.g., vendor does not deliver the commercial product]

R  Were experts and stakeholders queried in all the areas of risk to develop a broad list of credible risks?

R  Are the risks prioritized and the most critical ones identified?

R  For each high priority risk, are there ways to eliminate the risk? Or, reduce its likelihood and/or impact?

R  For each high priority risk, have the symptoms of the problem and a means for monitoring them been identified?

R  Are the high priority risks regularly monitored throughout the project?

R  For each high priority risk, is there a risk resolution plan?