Improving Transportation Systems Management and Operations – Capability Maturity Model Workshop White Paper – Performance Measurement
5. Implementation Plan Capability Improvement Actions
All agencies included some aspect of Performance Measurement in their implementation plans to improve agency capability, including the need to match performance measures with TSM&O goals and objectives. Those agencies that had developed some form of Performance Measurement did so on an ad hoc basis with no linkage to the overall vision of their TSM&O program. Specifically, the mechanism for incorporating performance measures was their inclusion in the TSM&O Strategic Plan. Because the strategic plan is a high level document, additional types of supporting documents also were cited as necessary for implementation of a Performance Measurement program; these include a TSM&O data business plan and TSM&O Performance Measurement plan (or similar name). Topics that would be covered in these documents include the following:
- Linking performance measures to TSM&O goals and objectives (performance measures should be used to track progress toward meeting goals, objectives, and targets).
- Establishing common performance measure definitions for internal partners (planning and construction) and external partners (emergency responders).
- Defining performance measures for all aspects of operations, including TIM, weather, work zones, and signal operations.
- Linking output measures to outcome measures for system performance, that is, understanding how changes in operations-related output measures affect system performance.
- Promoting operations in traditional planning and programming processes, especially integrating them into formal documents (e.g., long-range transportation plan, transportation improvement program).
- Applying a common set of performance measures for all phases of the project development process, from initial planning to final evaluation.
- Reporting formats, including dashboards.
- Strategy for integrating and using TSM&O performance measures in agency decisionmaking processes
- Data collection, management, and system development, including identification of data sources and the filling of data gaps in the data; as software is required for a Performance Measurement system, additional documents would be required to identify architecture, system design, and functional requirements. Data management also includes validation, quality control, aggregation, and accessibility to a wide range of users.
- Developing a modeling plan and tools for supporting TSM&O analysis
- Specifying evaluation procedures for completed TSM&O projects so that analyses are done consistently and results are transferrable
- Communication strategy for describing the benefits of TSM&O to upper management and the public
- Specifying target setting procedures.
5.1 Integration of Performance Measures into Decisionmaking
A major area with which agencies seem to struggle was the integration of TSM&O performance measures into investment decisionmaking at all levels. Agencies know they need to do this but are unsure how to proceed. Several agencies noted that the MAP‑21 requirements are spurring interest in Performance Measurement in general, and that this interest should be leveraged for implementing TSM&O Performance Measurement. As a result of the MAP‑21 requirements, many are investigating the purchase and use of private vendor probe data for the development of outcome measures. The question of how to use outcome and output measures in decisionmaking is problematic to agencies, however. There is little precedent, and the best approach depends on the institutional structure of the agency and the degree to which other units support TSM&O. As a starting point, performance measures were identified for use in project selection, prioritization, and resource allocation. TSM&O performance measures also should used as a basis for identifying investments in several related functions:
- Long-range planning in coordination with MPOs
- TSM&O deployment planning
- Annual program plans.
5.2 Develop TSM&O Staffing and Succession Plans
A commonly expressed motivation for undertaking Performance Measurement in the first place was to justify TSM&O programs. Agencies felt that their programs have positive benefits but it was difficult to explain them because TSM&O projects “don’t have ribbon-cutting” and the public doesn’t associate largely invisible TSM&O actions with congestion improvements. In the absence of the visibility and higher impact of capacity expansion projects, performance measures are seen as a way to get the TSM&O “story” across. Even in cases where agencies had published performance measures showing the positive effect of TSM&O, the message was difficult to convey. Improved communication procedures would be a big help. In the few cases where customer surveys exist, they are not focused on TSM&O activities but rather agencywide performance; questions about the details of TSM&O activities are rare in these surveys. It was recognized that customer surveys can be a significant way to communicate the effectiveness of TSM&O; however, little guidance exists on how to design and conduct customer surveys specifically targeted to the topic of TSM&O.
Coordination of Performance Measurement activities with internal (and especially external) partners was another common theme in the Implementation Plans. It was recognized that TSM&O performance programs were developed largely in a vacuum and that many other departments and agencies are involved in tracking various aspects of transportation system performance. It also was recognized that data from outside groups could be leveraged for TSM&O needs, highlighting the need to disseminate performance data and measures to partners.
Internally, examples of units requiring coordination for Performance Measurement are planning and programming, those responsible for work zones (construction and maintenance), and signal operations (if a different group does this). Externally, emergency responders (police and fire) and MPOs have a stake in coordinating Performance Measurement activities. For example, police CAD systems generate data usable for TSM&O purposes. Many MPOs routinely develop congestion reports (outcome related); many have purchased their own data.
A strong desire existed to observe and learn what TSM&O departments in other agencies were doing. Clearly, the profession is at the beginning of the Performance Measurement era and there is little precedent for how to structure a program. Also, some degree of experimentation is needed. As a result, many implementation plans suggested that peer-to-peer exchanges be undertaken.
Likewise, because comprehensive performance management has not been undertaken (the highest scores of “3” were achieved by only three agencies), several agencies identified the need “to start small.” Several suggested developing a pilot program in a region or selected priority corridors. Others wanted to do a staged approach to full implementation, building that approach into their Performance Measurement planning documents.
