<<Previous Contents Next >>

Getting More by Working Together — Opportunities for Linking Planning and Operations


2. Opportunities for Linking Planning and Operations


2.3 Performance Measures


BACKGROUND

"What gets measured gets managed." This often-repeated maxim recognizes that performance measurement can focus the attention of decisionmakers, practitioners, and the public. By focusing attention on the operating performance of the transportation system, performance measures serve as an important mechanism for increasing awareness of management and operations approaches within the planning process. Performance measures provide a means to link a transportation agency's perspective with the experience of those who use the transportation system. The act of defining performance measures and tracking performance often requires communication and coordination between those who manage operations for the transportation system and those involved with planning and policy development. Those who manage operations often have data and expertise on real-time system performance. Planners and policymakers can use this information in order to set goals, track progress, and make investment decisions.

What Is Performance Measurement?

Performance measurement is a process of assessing progress toward achieving predetermined goals, including information on the efficiency with which resources are transformed into goods and services, the quality of those outputs (how well they are delivered to clients and the extent to which clients are satisfied) and outcomes (the results of a program activity compared to its intended purpose), and the effectiveness of government operations in terms of their specific contributions to program objectives.10

Performance measures have many functions. They can be used to:

  • Frame what attributes of the transportation system are most important;
  • Provide information on current conditions and trends;
  • Evaluate the success of implemented and ongoing projects;
  • Provide a metric for communicating with decisionmakers and the public about past, current, and expected future conditions; and
  • Serve as criteria for investment decisions in the transportation planning process.

Performance measures can be grouped into three categories (See Exhibit 5 for examples):

  • Input measures – which generally address the supply of resources;
  • Output measures – which address the delivery of transportation programs, projects, and services; and
  • Outcome measures – which address the degree to which the transportation system meets policy goals and objectives.

Exhibit 5: Classes of Performance Measures

Exhibit 5

While input and output measures are the easiest to implement, outcome measures focus on the effects that the traveling public most cares about – issues such as travel time and delay, safety, and reliability. Increasingly, MPOs, transit operators, and DOTs are becoming customer-focused, which increases attention to the development of outcome measures. Still, outcome-oriented performance measurement is minimally practiced in many regions. A survey of MPOs asked, "Does your planning process reflect measurements of actual system performance, like travel time, reliability, and incidence of non-recurring congestion?" Of those that responded, 45 percent answered no.11

How Can Performance Measurement Create Stronger Linkages Between Planning and Operations?

Performance measures can help link planning and operations by focusing attention on customer-oriented outcomes and elevating attention to M&O strategies within the transportation planning process. Increased coordination and collaboration among operations and planning staff can also help instill operations thinking into policy and planning decisions.

Outcomes are beneficial for both planners and operators, as shown in Exhibit 6.

Exhibit 6: Benefits of Collaboration for Performance Measures

Exhibit 6

Key ways in which performance measures strengthen such collaboration are described below.

Performance Measures Bring Focus to Customer-Oriented Outcomes

Performance measurement has traditionally been in the realm of planners and policy analysts as part of the planning and investment prioritization process. Metrics tended to be those that can be modeled and used for long-term investment decisionmaking, such as average travel times and miles of congested roadways.

Case 10: Focusing on the Customer: Washington State DOT

As part of Washington State DOT's efforts to define performance measures for traffic congestion, the agency moved beyond traditional measures of average travel speeds to define measures focused on travel reliability (e.g., though use of a "buffer index"* to account for non-recurring delay). These measures were developed through coordination between planners and operators, and involve ongoing coordination in tracking performance. Prior to this effort, non-recurring delay did not receive this systematic consideration.

WSDOT publishes a quarterly report on the State's transportation system titled Measures, Markers, and Mileposts. Also referred to as the Gray Notebook, the report highlights the status of current projects, details where transportation funds are being used, and updates progress on management and operations measures such as incident clearance time and travel information provision. Washington State DOT has found the customer-oriented performance measures to be very effective in drawing attention to the benefits associated with its transportation investments and in building credibility for the agency.

According to a Washington State DOT staff person:
"The Secretary felt that by building the State DOT's accountability, the agency could attract more funding. The Secretary focused on making the case that WSDOT is on top of things. The best way to do that was through operations data because it gets at aspects of the system that the public cares about."

Contact Toby Rickman: Rickman@wsdot.wa.gov

* The Buffer Index expresses the amount of extra time a traveler must allot for each trip in order to be on time 95 percent of the time. As an index, this measure is useful for comparisons regardless of travel time and trip distance. The measure can also be presented in actual minutes of extra time required in cases where one wishes to evaluate reliability for a particular trip. Typically, the index is calculated for each road segment, and a weighted average is calculated using vehicle-miles of travel as the weighting factor.

Exhibit 7: Sample Tools for Measuring Performance of M&O Strategies

IDAS (ITS Deployment Analysis System) - a sketch planning approach focused on helping practitioners with specific ITS investment choices. IDAS draws on a database of a diverse set of ITS project evaluations in order to provide comparative costs and benefits over a range of possible ITS alternatives. Comparisons may be less precise than other methods because they do not account for specific local conditions, but the tool is relatively simple to operate and has low data requirements. More information on IDAS can be found at: http://www.camsys.com/idas03.htm

PRUEVIIN (Process for Regional Understanding and Evaluation of Integrated ITS Networks) - an analysis methodology containing techniques, programs, and data sources designed to assess the benefits of several integrated ITS services at the corridor level. Once set up, PRUEVIIN can be used repeatedly by both planners and operations personnel to explore optimal system arrangements, accounting for daily variability in travel demand, weather, and incidents. PRUEVIIN has modest data requirements but requires significant effort to operate. An application of PRUEVIIN can be found at: http://www.itsdocs.fhwa.dot.gov//JPODOCS/ REPTS_TE//13605.html

DYNASMART-P - a software tool developed for traffic operations planning applications under Federal Highway Administration's Dynamic Traffic Assignment (DTA) research program. DYNASMART-P combines (1) dynamic network assignment models, used primarily in conjunction with demand forecasting procedures for planning applications, and (2) traffic simulation models, used primarily for traffic operations studies. DYNASMART-P overcomes the limitations of traditional static assignment and simulation models by using advanced traffic modeling techniques to capture the dynamics of congestion formation and dissipation associated with time-varying demands and network conditions. Potential applications include:

  • Providing dynamic traffic assignment methods for traditional transportation planning analyses.
  • Assessing impacts of ITS and non-ITS technologies on the transportation network in the planning process.
  • Assessing impacts of different traffic operation and control strategies.
  • Supporting decisionmaking for work zone planning and traffic management.
  • Evaluating incident management strategies.
  • Evaluating congestion pricing schemes that vary with location, time, and prevailing roadway conditions.

DYNASMART-P is available from McTrans Software Center: http://mctrans.ce.ufl.edu/. For further information contact Henry Lieu at FHWA, Henry.Lieu@fhwa.dot.gov.

Case 11: Elevating M&O Strategies: North Central Texas Council of Governments (NCTCOG)

At NCTCOG, the MPO for the Dallas-Fort Worth region, data on system performance were used in developing an annual performance report to the MPO board (e.g., region-wide lane-miles of congested roadway, transit on-time performance). The performance report presented a forthright statement to local officials about the significant transportation, air quality, and funding constraints facing the region. The performance report helped local officials appreciate the important place of M&O strategies in the regional transportation vision.

Measurement of performance in terms of incident-based delay also yielded positive impacts in the planning process in the Dallas-Fort Worth region. When estimating vehicle emissions many regions that struggle with air quality issues do not consider the delay (and associated pollution) caused by incidents. As a result, incident response programs are undervalued. In response to this situation, NCTCOG estimates the contribution of incident delay to regional emissions. As a result, the MPO is able to take credit in its air quality conformity analysis for emissions reductions resulting from a successful incident response program.

Contact Natalie Bettger: nbettger@nctcog.org

Case 12: Incorporation into Strategic Plans: California Department of Transportation (Caltrans)

Caltrans developed performance measures into the agency's strategic plan. The agency links transportation system performance measurement to informed decisionmaking by focusing on measures that reflect the role that the transportation system plays in achieving broader State objectives. The measures are focused on outcomes and address system-level characteristics rather than specific projects. Some of the measures oriented toward systems management and operations include:

  • Number of corridors with reasonable alternatives during closures, and
  • Hours of both recurring and non-recurring delay by mode.

Caltrans seeks to use the measures to improve partnerships with stakeholders and to improve linkages with non-transportation issues (such as economic development and shifting demographics). The agency has developed an operations-oriented strategy to provide a framework for coordinating institutional linkages and partnerships that are necessary for successful systems management.

Contact Joan Sollenberger: joan_sollenberger@dot.ca.gov

Case 13: Annual Performance Reporting in the Albuquerque Region

The Mid-Region Council of Governments (MRCOG), the MPO for the Albuquerque metropolitan area, demonstrates the region's transportation system performance through an annual publication called Local Motion. This performance information is available to the public on MRCOG's Web site and is intended to educate the public, the staff of local governments, and elected officials. Local Motion summarizes continuously collected traffic count data on freeways, arterials, and collector streets. Every three years, Local Motion includes a report card for the area's transportation system to assist in developing the long-range Metropolitan Transportation Plan. The report card rates the system based on criteria that relate to management and operations such as emergency vehicle response time, congestion levels, and miles of roadway with ITS coverage. As a result of these performance reports, transportation officials and the public are able to evaluate the success of existing programs and target future projects accordingly.

Contact Sheila ter Bruggen: sterbruggen@mrcog-nm.gov

Increasingly, transportation agencies are moving toward a customer-oriented focus and want to develop performance measures that reflect what is most important to transportation system users (see Case 10). Examples of performance measures that focus on management and operations include:

  • Total or average hours of incident-related delay
  • Consistency of peak and off-peak travel times
  • Extent of real-time information provision (e.g., lane-miles or intersections for which information is available; number of ways to access such information)
  • Transit on-time performance

By addressing issues that are important to transportation system users, performance measures can focus help attention on management and operations where the experience of transportation system users is considered in a more immediate sense.

Thus, the process of developing and implementing effective performance measures can motivate collaboration between transportation operations and planning staffs. Moreover, advances in ITS and real-time monitoring of traffic mean that operators have access to an incredible depth of traffic data that can be used to measure more accurately the real traffic conditions experienced by users. This information brings to light a range of issues, such as incident-related delay and reliability, which are important customer issues but have not traditionally been included as performance measures.

Performance Measures Elevate the Status of M&O Approaches

Efforts to focus on system performance often result in better recognition of the value associated with M&O approaches. Data on system performance can highlight the value of investments in programs that minimize incident-related delays, provide information on real-time travel conditions, and improve emergency response times by showing how they can improve transportation system reliability and reduce travel times for customers. The limited number of tools to quantify the benefits of operational strategies is often noted as a constraint in bringing greater attention to M&O strategies. However, there is a growing number of tools on the market today to predict the effects of operational strategies on system performance. (See Exhibit 7 for examples of several tools.) Some agencies also have found success in measuring performance before and after implementation of operations-oriented projects.

Given increased traffic congestion, limited road space, and funding constraints, public decisionmakers in many areas recognize the limitations associated with constructing new transportation infrastructure to meet regional transportation goals. Use of performance measures and measurement of the benefits of M&O strategies, can help these decisionmakers to appreciate the value of M&O approaches toward meeting both short-term and long-term goals (see Case 11).

Performance Measures Help Inform Policy Decisions

By focusing attention on system characteristics that are important to the traveling public, performance measures can help planners focus on the day-to-day experience of transportation system users. This provides important balance in settings where planners have been exclusively focused on very long-term development of the network. With greater focus on the day-to-day characteristics of the system, the issues faced by operators such as incident response, work zone management, and provision of traveler information take on greater importance. As a consequence, mid- and long-term planning will reflect greater consideration of management and operations planning and investment needs. Greater understanding of operations issues by planning staff can also help in setting transportation policies.

TAKING ADVANTAGE OF LINKAGE OPPORTUNITIES

There are a number of opportunities to use performance measurement to build stronger linkages between planning and operations. Following are some ways to take advantage of performance measures in this way.

Involve Operations Managers in the Process of Developing Performance Measures

Agencies responsible for major investment decisions often take the lead in developing performance measures. However, it is critical that this process involve practitioners who are concerned primarily with day-to-day operation of the transportation system. The operator's perspective relates closely to near-term concerns of the traveling public. Though operations practitioners have clear expertise to contribute the performance measure development, a focused effort may be required to ensure that they understand the importance of their involvement in the process and the importance of performance measures being supportive of regional goals.

Incorporate Operational Performance Measures Into Strategic and Long-Range Transportation Plan (LRTP)

Performance measures in the LRTP should be driven by the goals and objectives of the plan, which in turn, should reflect the region's vision for its transportation system. Customers (including the general public, freight shippers, and others) are increasingly concerned about operational performance of the transportation system, including the reliability of the system and the availability of information about travel conditions that can inform the best travel time, mode, and route (see Case 12).

Incorporating operational performance measures into the LRTP provides an avenue for operators and customers (through public involvement) to get involved in the planning process. It can provide better information to customers and stakeholders on the progress being made toward desired goals and objectives, and can, therefore, serve to make long-range plans more real to the public. Moreover, incorporating performance measures helps to ensure that regional transportation system management and operations programs receive adequate attention in prioritization of projects for funding.

Use Operations Data for Tracking Performance in Annual or Quarterly Reports

Periodic performance reports provide an excellent mechanism to make planning more relevant to everyday experience. A number of MPOs, transit operators, and State DOTs use performance reports to inform decisionmakers about the trends in system performance.

Such reports work as a linkage in a number of ways:

  • First, they provide a realistic view of system performance improvements achievable through management and operations investments.
  • Second, they provide operations managers with guideposts and goals that provide some measure of how operations programs are contributing to the long-term goals of the system.
  • Third, they support policy that is realistic about system constraints and that supports the role of management and operations in maintaining acceptable transportation performance.

Agencies that report performance measures in a quarterly or annual performance report encourage a sustained communications link between planning and operations staffs. Even a very simple report providing one or two performance measures can have a positive effect in broadening the discussion over investment priorities. There are many cases where a particular activity or project requires temporary coordination or exchange between planners and operators, but sustaining such communication is critical for changing the everyday perspective of these departments to routinely consider operations tools within the planning process. Routine, sustained, performance reporting is, therefore, particularly valuable (see Case 13).

Use Performance Measures to Motivate Data and Tool Development

Given the rapid evolution of automated travel data collection technology, it is helpful to discuss performance measures beyond those that are supported by current capabilities. As one element of a performance measurement effort, transportation agencies within a region may jointly wish to define the most appropriate measures and associated data needs, without allowing current resources to limit the discussion. This can be used to establish goals for data collection and measurement and to provide some focus in reviewing the stream of evolving transportation information technologies.

LESSONS LEARNED

The very characteristics that make performance measurement a useful linkage between planning and operations can also make performance measurement a complex challenge. The process often demands new data and tools and may require new levels of coordination between agencies or departments. The following lessons have emerged from agencies that have faced such challenges.

Begin With Simple Output Measures

Some regions focus on output measures, which are often simpler than outcome measures. For example, the Maricopa Association of Governments performance measures include the number of signals coordinated and the percentage of cross-border signals coordinated between cities. While not specifically related to the customer, such measures can still succeed in increasing policy and investment focus on the region's M&O progress. Output measures are particularly effective where there is already some appreciation among local decisionmakers for the value of M&O solutions. Using simpler output performance reporting can inspire the attention and collaboration necessary to design measures that address the most important aspects of the system performance.

Do Not Expect a Clean and Simple Process

Defining performance measures takes time and may not yield immediate, refined outputs. This is particularly the case when the process is working to attract a broader audience. When new stakeholders and perspectives are drawn to participate, the process becomes more complex; deciding on concrete characteristics to measure raises difficult questions about the fundamental objectives served by the transportation network. Agencies should approach performance measurement with awareness that the effort will be a challenge. Initial performance measures may not be perfect, but initial steps are apt to attract greater interest and advance the effort.


10 Performance Measures of Operational Effectiveness for Highway Segments and Systems - A Synthesis of Highway Practice, NCHRP Synthesis 311, Transportation Research Board; Washington, DC, 2003.

11 Survey of MPOs on Linking Planning and Operations, Association of Metropolitan Planning Organizations, January 2004.


<<Previous Contents Next >>