Approaches to Target Setting for PM3 Measures

Chapter 4. Target Setting Approaches

In discussing target setting approaches, it is important to be mindful of the Federal requirements for target setting. For that reason, throughout this portion of the report information from the Federal Requirements for Target Setting subsection will be reiterated where appropriate. In this manner, the report will demonstrate how various target setting approaches can be deployed to meet the needs of an agency’s constituency while also facilitating compliance with Federal regulations.

Overview of Target Setting Approaches

Approaches for setting targets vary, though most agencies use a hybrid approach in which they not only use different approaches for different measures, but also multiple approaches for a single measure—reinforcing the concept that target setting is a multidimensional process. The Washington State Department of Transportation (WSDOT) example presented later in this document exemplifies the hybrid approach. This section of the report provides an overview and discussion of different approaches for target setting and how they apply to the PM3 performance measures.

Target Setting Approaches

Neither the legislative provisions nor the Federal regulations mandate that agencies use a certain approach to setting targets. There are many relevant factors to consider in target setting, customer and stakeholder perspective, agency experience, commitment to regular communicating and reporting, sphere of agency control, financial resources, and timeframe, among others. Agencies balance these oftentimes competing factors.

The FHWA PBPP Guidebook and the National Cooperative Highway Research Program (NCHRP) Report 666: Target Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies identify several approaches for setting targets. In setting System Performance targets, State DOTs and MPOs may find that it is effective to deploy aspects of the multiple approaches identified below:

• Policy driven—Targets are set in a “top-down” manner, such as by senior executive management or a political body. As observed in NCHRP Report 666, this approach is often taken in response to public outcry over a particular transportation issue with direct action from elected officials being called for. Under this approach, senior management or a political body defines targets in the context of larger transportation goals or policies; staff is then tasked with developing an investment plan to meet the target and conducting analysis needed to demonstrate attainment of the target under a future funding scenario.
  This is a very direct approach and has the benefit of providing agencies with a well communicated target that conveys a strong message. However, it also can create a challenge for agencies if practitioners were not involved in the target setting process and/or if the process was not informed by real-world planning, funding, and technical constraints that may impact attainment of the target.
• Modeling/analysis driven—Modeling may be used to assess progress towards targets or to help define a target level. Project and policy scenarios can be tested using travel demand models, microsimulation models, and other post-processing tools to include metrics to evaluate their relative contribution to progress toward targets. Models may be used to set targets in either a top-down or bottom-up manner. Top-down modeling determines the strategies or funding needed to achieve a target that meets requirements either set by the agency. In bottom-up modeling, target levels are limited to what is realistically achievable given resource constraints.
• Consensus based—With this approach, transportation staff set targets through an internal, consensus-based process drawing on the expertise and experience of staff across different units within the agency while taking into account funding and other resource constraints. Transportation agencies may collaborate with stakeholders and staff from other agencies in this process. This approach can lead to very meaningful and effective targets as it recognizes factors specific to a region or State, as well as limitations impacting target attainment.
• Customer feedback based—Under this approach, direct feedback on system performance and objectives are gathered from system users through surveys and other outreach methods. This feedback is then used by transportation agency staff to develop targets that are closely aligned with the needs of the “customer”—the system users.
• Benchmark based—Targets are established based on a review of similar investment approaches and results from other transportation agencies. With this approach, criteria should be set for peer group selection and analysis. Once an agency defines its peer group, the analysis will measure each peer’s performance relative to the others with the results of the comparative analysis used to determine the target level for the agency.

Target Setting for PM3 Measures

A target by itself will not change the dialogue about resources, budget priorities, and funding allocation. What the targets can do, however, is provide additional support to ongoing discussions about funding adequacy, the operating practices, and the policies and processes that are used to develop and implement projects.

There are a few issues to consider along the path of setting and using performance measure targets. The following questions are important for agencies to consider:

• Is the responding agency a State DOT or an MPO?
• Will the targets and measures be a key part of local or State decision-making?
• Is the goal to meet the targets or use the targets as aspirational levels that might spur alternative practices, executive action, or legislation?
• Will the target setting begin with local/regional priorities or at the State level? What road systems will be included in the target setting of each agency? How will the targets be integrated in the transportation planning and project development process?
• How much experience does the agency have in using the data for system evaluation or reporting progress? What data sets does the agency have on-hand that would be useful for system evaluation and reporting? What other datasets might be useful to explain year-to-year changes?
• What is the plan for adjusting the targets—in response to experience, funding changes, policy actions, or other events?
• How will the results be communicated to the various audiences? How will we explain differences in datasets from year to year? If an agency uses other performance measures for their investment decisions, how do these measures relate to the PM3 measures? How will factors outside the agency control be handled?

The target setting process will evolve over time, as there is more clarity and experience with the data, measures, type, and scale of applications. Operationalizing these targets will be a challenge for many States and MPOs; a few key steps and considerations listed below are helpful, but there is no “cookbook” for target setting or for the follow-up communication program. Experience, agency policy, interest from the public, decision-maker input, and other factors also will shape the application and range of performance targets.

Target values, generally, represent the desired performance of a measure. In practice and in recognition of the complexity of the application, however, the target values will likely be a combination of the following:

• What is achievable in a physical and technical sense?
• What is supportable through discussion with the public and policy-makers?
• How do transportation performance measure targets relate to broader societal or governmental outcomes?
• What are the implications – in terms of financing, public perception, and procedural requirements for meeting or not meeting targets?

Whether the goals are of the “stretch” variety—performance levels that are difficult to achieve—or those that have achievable levels are a different topic (discussed below). Conceptually, all performance measures, goals, thresholds, and targets are a product of a fully involved planning process. A program of public engagement during the planning process that generates a picture of what the public wants their State/region to be when it “matures” uses targets as a numerical indicator of progress, not an end result. Getting to such a process will require a few iterations of planning and measurement, but it might be useful to think about the kind of information that this process would yield when setting goals in the near term. Such a process, and the set of outcomes it would describe, allow transportation concerns to be appreciated in the broader discussion of economic and social issues, and provide a forum for a more explicit discussion about the transportation service and system performance levels that can be delivered for (1) the amount of funding that will be available; and (2) the amount of funding, policy, travel option, and planning changes that are needed to achieve the goals.

The Federal rules for system performance require State DOTs and MPOs to work together on target setting. The regulation deadlines show the State DOTs setting Statewide targets first, and then MPOs developing targets for their region.¹⁷ This type of “State-in-the-lead” process makes sense in the infrastructure quality measures, such as pavement and bridge quality, where the State DOT has most of the data and a significant portion of the funding, as well as priority-setting responsibility for maintenance and rehabilitation. In the system performance realm, however, there are many States where one or more metropolitan regions will likely have congestion targets that are not in the rural style of “close to free-flow conditions.” States with multiple urban areas may see differences between mobility targets in those urban areas. Most large and many medium-sized, metropolitan regions already have targets that function as “how many hours of stop-and-go traffic are acceptable over how much of the system?” In this situation, there appear to be two general approaches to the target setting process. Both approaches involve coordination between the State DOT , the MPO, and other local transportation agencies (e.g., municipalities, transit agencies). Options include the following:

• Begin with a Statewide target. This approach is procedurally consistent with the initial schedule in the MAP-21 legislation. State targets for the system performance and reliability measures are set before the metropolitan area targets. In this approach, the targets might be set for the roads on the Interstate Highway or NHS. This approach is grounded in the State responsibility for operating the transportation system.
• Begin with local targets. This approach builds from the premise that the metropolitan planning process and the funding procedures in many States involve the development of fiscally constrained transportation improvement programs based on the regional long-range transportation plans. The local agencies and the local DOT office would begin the discussion and value setting using the local plans. For States where more than one metropolitan region is setting targets, the statewide target might involve calculating the average of the metro area targets (likely some weighted average involving population or person-miles traveled).
• Examine targets across multiple goal areas simultaneously (e.g., travel time/congestion, infrastructure condition, safety). Such an approach requires the ability to analyze the cross-asset allocation of investments in each goal area. This form of tradeoff analysis provides a means for assessing the impacts of different approaches to investing. In performance management, tradeoff analysis is useful in establishing performance targets because it provides insight into the cost and performance implications of different strategies:
  • What is the effect of concentrating investments into a single performance category (i.e., pavement, safety, or congestion/mobility) rather than selecting projects purely on a benefit/cost basis? For example, if we choose to invest more heavily in pavement improvements, what are the performance impacts on safety and congestion?
  • How do we decide which deficiencies to deal with first?
  • Should minimum spending levels be established for each program area and how this would be accomplished?

The consideration of which approach to use might be determined by the long-term goals of a State’s metro regions. If the transportation goals and land use patterns will be similar, the process should be less involved than if there are different goals. For example, a region that encourages dense development near the traditional town center and invests in public transportation, walking, and biking will likely have different congestion goals than a region emphasizing capacity expansion, system operations, and suburban growth patterns. Constructing the Statewide target, where every region is pursing the same long-term goals, might have different processes than a State with regions pursuing different goals.

The differences between the perceptions of congestion between large metro areas, small metros, or rural areas also might play a role in deciding the approach. The “start local” approach might accommodate the differing perceptions of acceptable performance among urban, suburban, and rural areas. No matter where the target setting process begins, coordination between Government levels and agencies will allow local area input, including getting citizen feedback, local policy decisions, available funding, and other important criteria.

Just as with area-to-area differences within a State, there also are differences in expected performance and mobility within a metropolitan region or on freight-intensive routes. Stop-and-go traffic in downtowns is typically more acceptable than in rural areas. Downtowns usually have shorter trips and more travel mode and route options than trips from rural or suburban regions. Downtown solution strategies usually do not include significant roadway widening projects as often as in rural corridors. The target setting process then might ultimately include subarea analyses in some future year. This will be more important if targets are used to decide which projects will be selected. When subarea or corridor conditions are better than the targets, projects, operations investment, and policy changes would be lower priority than in areas where conditions do not meet their targets. Center-city corridors that are congested, but above their target value, may not require additional study or investment, even though their congestion may be worse than some suburban region where acceptable performance is defined differently. This sort of process will take several cycles to optimize.

Target Setting Process Summary

The target setting process used by States and MPOs should satisfy the Federal regulations (23 CFR 490.105)¹⁸ and incorporate local information and data analysis to address agency and constituent needs. Broadly speaking, the National performance measures also were designed to provide paths for agencies to use data, measures, and targets to identify system performance concerns; prioritize projects, programs, and policies to address them; and to allocate funding and promote accountability and transparency for their spending.

Once targets are set and used, the Federal regulation allows for adjustments.¹⁹ This will be important early in the measure deployment process as experience is gained. Travel demand will change, transportation funding programs are periodically adjusted, and technology advances will alter the approach to transportation service. Target adjustment also is a part of the regular planning process and even if the Federal measures are not a component of decision-making, it is appropriate to spend time ensuring the targets are appropriate. There also will likely be a need for longer-term targets, as well as the near-term targets specified in the regulation.

Given the schedule for larger funding and development programs—the typical transportation improvement programs have 4- to 6-year periods and larger projects can take 10 to 15 years from the initial planning stage to opening—any move toward an aspirational target will take time. It will be important to specify the actions being taken to address needs and meet targets as a part of the evaluation discussion. It is likely that many States and regions can meet the requirement for “significant progress,” even while system performance is getting worse if they show both the performance measure history and the change in trend while adjusting for changes in the external contributing factors.²⁰

It is suggested that agencies should move toward a process where the system performance measure targets are decided in the same way that most funding decisions are made—coordinated between the State and metropolitan area levels in a process that involves the agencies, decision-makers, and the public. The project selection process decides the best use of metropolitan area funds and seeks to achieve community goals and vision. The goals and vision can be translated into congestion targets that all stakeholders can understand.

The process might use the following steps or questions to help the States and MPOs in the initial round of target setting, though this process and the steps are not required except where aspects are based on the statutory or regulatory provisions cited:

• Define the purpose. Why are you setting the target? One purpose could be to use the measures for agency-level project evaluation and selection. This has a broad set of implications and a potential for adopting measures that are (still) not well understood based on a dataset that is very new. Another purpose could be to ensure that the Federal requirements are met. This approach has less pressure and fewer ramifications for overall agency procedures. This does not mean that lessons cannot be learned and incorporated, but the timeline for incorporating the targets into the regular project development and selection processes can be slower and the adaptations consistent with agency capabilities.
• Set target parameters. The timeframe for target achievement and the scope (areas, roads, modes included) are defined in the Federal regulations.²¹ Conservative values should be used based on trends until the measures and affecting factor are understood. Agencies also should identify considerations, such as economic or population growth projections that might change future targets and expectations.
• Assemble the baseline data and analyze the trends. Measures should be calculated using the data in the NPMRDS second version (v2 and future versions) dataset for as many years as feasible. Agencies might use areas, regions, or corridors where operating conditions are well known to serve as benchmarks for the analysis.
• Identify and assess influencing factors. Possible influencing trends should be collected from agency datasets and experiences, as well as other agencies or jurisdictions. This should include consideration of the following factors:
  • The role of the economic recession of 2007/2009 in past trends, as well as the likelihood of future economic challenges at the region or National level.
  • Historic performance trends—What are the key elements of past trends and any policy, practice, or operating strategy changes that explain the measure values?
  • Fiscal limitations and tradeoffs—Many agencies have dealt with funding limits, relating past performance to those funding levels should inform future target discussions.
  • Effects of autonomous and connected vehicles—For example, how will connected vehicles alter crash and mobility trends? Will vehicles with no drivers have a beneficial effect on traffic or will driverless vehicles cause more congestion.
  • The amount of funding and policy flexibility in future decision-making is an important element in future target setting.
  • How will influencing factors will change over the time span of a target?
  • What risks are associated with each of the factors, including magnitude and likelihood of change in trend?
• Establish a target and an annual update plan. This is especially for the early years and it will be important to check in with the data to understand the trends and possible data irregularities.²²

Target Setting Example—Washington State Department of Transportation

This section of the report describes an example of how one agency, the WSDOT, responded to the Federal requirement to adopt performance targets and the ongoing encouragement to use targets and performance measures to shift to more data driven, goal oriented, decision-making.

Washington State Department of Transportation Outcomes

WSDOT created Statewide (NHS) performance targets for the PM3 measures:

• National Highway Performance Program (the percentage of person-miles traveled on the Interstate System that were reliable and the percentage of person-miles traveled on the NHS, but off the Interstate System that were reliable).
• Freight Movement on the Interstate (the Truck Travel Time Reliability Index (TTRI)).
• The Congestion Mitigation and Air Quality Improvement Program (non-single occupancy vehicle travel in the Seattle urbanized area [NHS], peak hours of excessive delay per capita in the Seattle urbanized are [NHS]).

Table 1 shows the roadway performance targets adopted by WSDOT.

(Source: Washington Department of Transportation.)

A summary of the resulting WSDOT targets can be found at the WSDOT MAP-21 Accountability website. This website provides access to all of the MAP-21 performance measure information developed by WSDOT. A folio was specifically developed to present the target setting process and outcomes for the above categories of PM3 measures.

The initial targets for the State were adopted with an expectation that current performance trends would continue for the next 4 years. Therefore, to compute the expected outcomes, trend lines were computed and applied to the best available measurements of current conditions.

The decision was made to initially adopt a more conservative target, given the agency’s ability to revise its targets at the midpoint of the four-year performance period. This allows the agency to gain a better understanding of the new data sources being used and how they data sources reflect roadway performance. Important factors will be the transportation system improvements being implemented within the State’s major urban areas, where much of the congestion occurs, as well as how well the impacts of major disruptions in rural areas that are outside of the control of the agency (e.g., major snowstorms and major forest fires) are reflected in the performance metrics.

Role of Targets in Transportation Performance Management

Several years ago, WSDOT made the strategic decision to adopt a more PBPP process. As part of that decision, WSDOT adopted Practical Design principals within the agency, which it refers to as Practical Solutions.

Practical Design emphasizes a focus on identifying the core purpose and need for each project selected for agency funding, and then working to scope those projects so that the resulting design stays focused on achieving those needs and core purposes.²³ The intent is to create better informed decisions by emphasizing a performance-based practical design (PBPD) approach to project selection and design. Under Practical Design, the project development process attempts to find creative, lower cost solutions that resolve the core purpose and needs, while also considering short- and long-term project and system goals. The intended result is that the agency can eliminate nonessential project design elements, resulting in lower costs and improved value. Improving traffic operations is heavily weighted when congestion relief is an identified project need.

Because the Practical Design process is very different than how projects have traditionally been developed, scoped, and delivered, the WSDOT is very interested in being able to measure and describe the benefits of using the new process. This involves both using data to describe how needs are identified and tracking system performance over time to demonstrate the outcomes from the process.

When operational improvements are made instead of conventional capacity improvements, many resulting performance improvements are found in more reliable travel conditions, rather than higher volumes or faster average speeds. Being able to use the PM3 statistics—and the NPMRDS data—to measure these performance improvements is very important for communicating with the legislature, as WSDOT strives to continue delivering performance improvements despite limited resources.

Challenges to Target Setting

WSDOT’s biggest issue with target setting has been the availability of data and in particular its lack of data on roadway performance for non-NHS road segments. Because of FHWA’s purchase of NPMRDS data, data are available for measuring and reporting roadway performance on the NHS. Unfortunately, WSDOT does not have data similar to the NPMRDS for roads under its jurisdiction (State Routes) that are not part of the NHS. Similarly, MPOs in the State do not have data for many roads within their jurisdictions that are not on the NHS but that are important to the transportation system’s performance within their urban areas.

While this lack of data does not need to directly affect the computation of the National performance metrics or the setting of National PM3 targets, the lack of data on non-NHS roads under their jurisdiction creates difficulties for WSDOT and the MPOs in conducting their larger needs identification and performance monitoring efforts. And those difficulties create problems with the adoption of the PM3 measures and targets within the agencies’ larger PBPP.

The issue is that, without data on non-NHS segments, if NPMRDS data are used for identifying needs and for monitoring roadway performance trends for NHS road segments, a different process will have to be used for non-NHS road segments than for NHS segments. This will create a lack of consistency within the agencies’ monitoring and project prioritization procedures and will make it difficult for them to track outcomes—and thus achieve performance targets—across all roads under their jurisdiction.

To resolve this issue, WSDOT considered the following options for overall performance reporting and target setting:

1. Perform PM3 reporting and target setting only for NHS routes and only for Federal reporting purposes.
2. Buy more data, allowing expansion of the system to all roads in the State system and allowing expansion of the adoption and use of PM3 reporting to those other roads.
3. Use a different process than the NPMRDS for computing PM3 measures, but one for which the necessary data are available statewide.

The first of these options would meet FHWA requirements and would provide performance and reliability reporting on the largest and most important roads in the State. This would be very useful for examining the performance of those roads, but as noted above, it means that the State would not have consistent reporting on all of its roads. The lack of consistent reporting means that the Department would either need to: 1) use different procedures for evaluating performance, identifying needs, and judging the effectiveness of implemented projects for NHS and non-NHS roads; or 2) use different procedures for identifying needs for agency decision-making than the procedures used for reporting PM3 measures to FHWA.

The second option would involve purchasing the additional data needed to apply consistent procedures throughout the State. Multiple States have adopted this approach.

Using a different technique for computing PM3 from existing State resources was the remaining option. The positive aspect of this solution is that a consistent set of measures would be used for all State routes. The primary negative aspect of this approach is that the data sources that cover the entire State highway network are not as robust as the NPMRDS at monitoring roadway performance and, in particular, reporting on the reliability of roadway performance.

Ultimately, WSDOT chose the first option. It used the NPMRDS to produce the PM3 statistics required by MAP-21 and then set targets—but only for the NHS—based on those statistics. The steps used to perform this task, and the setting of targets, are described in the remainder of this chapter.

The downside of this decision is that the agency had two options:

• Using a different process as part of its Statewide needs identification process in order to use the same process for all roads in the State.
• Using two different procedures for NHS and non-NHS roads in the needs identification process.

WSDOT chose to use its historical needs identification process, rather than use two different procedures. This allows all roads in the State to be treated the same. For example, the use of NPMRDS provides much better temporal information about road performance, which yields a better understanding of the reliability of roads sections. Instead, the agency must rely on other data sources and procedures within its historical project and needs identification process.

While the needs identification process is still based on the agency’s historical process, WSDOT is working to incorporate FHWA’s PM3 measures and the NPMRDS into that process as one more data source for identifying and quantifying needs.

Technical Challenges to Setting Targets

Once WSDOT decided to use the NPMRDS data both for reporting federally required PM3 statistics and for developing targets for the NHS, the next task was to determine how to set those targets.

Several decisions were made after analysis of the alternatives. The first decision was to use American Community Survey (ACS) data to estimate the percentage of single-occupant vehicles (SOV). The ACS was selected because its use also allowed straightforward computation of urban areawide trends in SOV use. The ACS is readily available and can account for the changing mode split pattern occurring in the Seattle metropolitan region, where travel patterns are expected to continue to change over the four years covered by the PM3 targets.

The next technical decision WSDOT had to make was to use the average vehicle occupancy (AVO) statistics published by FHWA.²⁴ AVO is needed to estimate person-miles of travel from the vehicle volume estimates that currently are available.

For many years, the WSDOT collected large numbers of vehicle occupancy data on the Puget Sound (Seattle area) freeway system with the specific intent of monitoring use of the regional high-occupancy vehicle (HOV) lane system. These data could have been used to meet both the SOV and AVO data requirements. However, this data collection system was curtailed several years ago because of its cost. Therefore, for the PM3 reporting, WSDOT determined that direct collection of the required vehicle occupancy data would be too expensive.

The next technical challenge was to develop or select software tools that could effectively analyze the NPMRDS data. WSDOT had performed earlier research into the attributes and use of the version 1 NPMRDS database. On the basis of that research, it was clear that the task of creating the software needed to effectively use the NPMRDS was significant. To lower the cost of that task, WSDOT joined many other States in purchasing access to the Regional Integrated Transportation Information System (RITIS) MAP-21 tool created with American Association of State Highway and Transportation Officials (AASHTO) support. Use of this tool greatly simplified the analysis of NPMRDS data.

Determination and Extrapolation of Trends: Using the RITIS tool, WSDOT then examined how to best approach the development of targets. To inform its target adoption, WSDOT first wanted to gain an understanding of the PM3 measures and how those measures were changing over time, as this would allow the Department to better understand what targets were reasonable.

The biggest issue with understanding both current system performance using the PM3 metrics and the current trends of those PM3 metrics resulting from the change-over from the NPMRDS version 1 database to the NPMRDS version 2 database. These two versions of NPMRDS used different approaches to the analysis of vehicle probe data. Version 1 of NPMRDS did not allow the computation and use of space mean speed estimates calculated from collected vehicle probe data. Instead, it relied entirely on the geolocations and instantaneous speeds from reporting vehicle probes. Version 2 of the NPMRDS allows the computation of space mean speed from these data (i.e., the computed distance traveled between the time reported for consecutive geolocation points allows for the computation of the average speed at which that vehicle is traveling).

The result of this change is twofold: (1) the data in the version 2 database are expected to provide a more accurate estimate of roadway conditions; and (2) there is a discontinuity in estimated congestion levels on roads between the statistics produced with the version 1 and version 2 data sets.

An additional consideration of WSDOT was the fact that because the changeover from version 1 to version 2 occurred recently, there were not enough years of version 2 data for computing trends, but there was enough version 1 data for trend computation. If both version 1 and version 2 data were used in the trend computation, the discontinuity in the estimates would have a large effect on the estimated trend, and this was not a good outcome.

WSDOT resolved this technical challenge by using the version 1 data to compute ongoing trends in congestion, but using only the version 2 data for the current estimate of congestion conditions. This means that the forecast of future conditions is based on an initial starting point from the version 2 NPMRDS data set, and the trend line is computed from the version 1 NPMRDS data set. The process is similar to that used by the NCDOT discussed in the following section.

All version 1 data were collected and reported in a similar manner, and therefore, the trends revealed in those data should present an accurate estimate of changing congestion patterns and not represent changes in the measurement process itself.

WSDOT’s analysis of the congestion trends using version 1 data showed that the best fit trend line was computed by using a compound growth rate.

The formulas from the trend models developed with version 1 data were then applied to version 2 data to predict expected performance two and four years into the future. This approach took advantage of the two data sets’ strengths. Version 1 has the data needed to make the trend computations; version 2 has a better estimate of current conditions.

Because this entire process was new, WSDOT also conducted due diligence to provide the agency with confidence in the trend estimates’ reliability. To perform this due diligence, WSDOT asked the outside consultant that staffed the urban mobility pooled fund study (Texas Transportation Institute) and reviewed the trends computed by that effort to confirm that the trends from that study produced outcomes similar to those computed with the NPMRDS. The trends from the pooled fund study project were similar to those developed with the version 1 NPMRDS work, so this approach was adopted.

WSDOT was not able to use the AASHTO MAP-21 RITIS tools to compute the peak hours of excessive delay in the Seattle urbanized area measure, as this tool was not yet functional. This PM3 measure was computed by using vehicle speed, delay, and volume data from WSDOT’s Corridor Capacity Report.²⁵ WSDOT has performed detailed analysis of freeway use and performance annually in the Seattle metropolitan region since 1997. That allowed a very accurate computation of the average percent change in delay from year to year. The growth rate for freeway congestion from this analysis was assumed to continue for future conditions and was used for the peak hours of excessive delay statistic. The limitation with this historical analysis is that it relies on data collected by WSDOT’s freeway surveillance and control system. This excellent data source is not available for roads outside of the State’s urban freeway system.

Identifying Vulnerable Portions of the Network: While the NPMRDS data can be very useful in identifying vulnerable portions of the highway system, because NPMRDS data are not available for the entire State highway network and the agency does not have the money to buy such data for the entire State, WSDOT has not formally used the NPMRDS for this purpose. Instead, the agency has relied on its historical mechanisms for identifying vulnerable road segments. For example, before the PM3 requirement, WSDOT already had performed a specific project identifying sites problematic for freight movements, so the results of that effort were used to identify road segments that should be examined more closely.

Agency staff who analyze potential sites identified through the existing needs identification process do use NPMRDS and other vehicle probe data sets to examine the performance of those road segments, but that process currently is not a formal part of the Department’s needs assessment process.

Performance Management: As part of the agency restructuring undertaken to better implement the Practical Solutions-based PBPP, WSDOT continues to refine its project identification, selection, and prioritization process. As part of that effort, the agency has created an Integrated Scoping Group, which is determining how to incorporate the wide range of needs, issues, and plans from across the Department that need to be included in the PBPP. The Integrated Scoping Group must include not just roadway performance but also pavement repair and rehabilitation, environmental improvements, and other important State needs.

The integrated scoping process that is part of the new Practical Solutions approach continues to evolve as the agency reviews the outcomes of the last round of this process, discusses those outcomes with the State legislature, and examines how to best expend the funds it has across the needs for data collection, analysis, and project delivery. The PM3 and other Federal targets play a role in this process, but Federal requirements are still only one factor among many that the agency must consider.

Other Challenges

In adopting the PM3 targets and in implementing PBPP principals in general, WSDOT chose to move toward a more performance-based planning approach for project selection and implementation. The fact that the Department adopted a very public approach to measuring and reporting changes in urban freeway congestion more than 20 years ago, helped considerably in encouraging adoption of the PBPP principals. However, WSDOT still faced three major challenges:

• A lack of financial resources to perform the PBPP tasks in ways that met both FHWA’s desired outcomes and State-specific needs and constraints.
• Conflicting interests between different stakeholders, both within and outside of the Department.
• Concerns that many of the factors associated with performance outcomes were outside of the control of the agency and about what that meant for setting targets against which WSDOT would be judged.

The impacts of the first of these challenges have been discussed above. The heart of this challenge is that the Department lacks the budget to fully fund the data and analytical tasks needed to completely shift their needs assessment, project selection, and prioritization process to new data sources and analytical techniques. This challenge is compounded by the fact that many of the newer data sources are far from perfect and often have biases and errors that are not yet known, which leads to resistance to using them among some stakeholders.

While older procedures and historically used data sets also contain errors and biases, those systems have become “accepted practice.” Changes from those accepted practices to new practices and procedures face internal resistance, as staff—and outside stakeholders—are rightly concerned about how the new biases and errors will affect outcomes, and therefore, how they should be identified and addressed in the new procedures.

Because many of the new data sources require the purchase of fairly expensive privately collected data sets, and the purchase cost of those data must come from already constrained financial resources, it can be difficult to generate internal support for those funds.

This outcome has set up the next challenge, stakeholder concerns that the use of two different performance-based systems (one on the NHS and one for roads off the NHS) would result in unequal consideration of the needs and priorities of those two different sets of projects. The result has been delays in shifting to using the Federal PM3 metrics within the Department’s internal decision-making process. While the results of Federal reporting are considered in the agency’s identification of needs, they currently are not at the core of the decision-making process because they are not universally available.

Next, there have been challenges in selecting the targets that are adopted and submitted to FHWA. WSDOT is concerned with understanding how effectively the new performance monitoring systems will report changes in performance over time, especially given the difficulties that probe data sets have traditionally had in identifying the operational performance of HOV and high-occupancy toll (HOT) lanes that are physically contiguous to general purpose lanes. That is, if dynamic tolling and HOT lanes are a key part of the agency’s congestion relief program, it is imperative that the metrics used to examine roadway performance account for the success of those efforts. Vehicle probe data sets have historically had difficulty separating the performance of physically contiguous roadway lanes. Consequently, the agency wishes to observe how the NPMRDS-based PM3 measures report those changes before setting performance targets.

In addition, growth in the Seattle metropolitan region has been among the highest in the Nation in recent years. While a large amount of regional growth has been absorbed by growth in multimodal travel, congestion growth on the region’s freeway system remains an issue, and it is difficult for the agency to estimate the size and location of future population and employment growth within the context of the PM3 target setting process.

These issues have led the agency to be conservative in its approach to target setting, even as the Department has pushed for increased legislative support for funding for operational improvements and other congestion relief projects.

Washington State Department of Transportation Approach to Setting Targets

The WSDOT did consider adopting targets other than its adopted approach of “current trends will continue, and therefore our expected performance is current conditions plus the current trend.”

One major reason for adopting this conservative approach to targets is that the Department has little experience with forecasting the impacts of major corridor specific changes on the new FHWA PM3 measures; this makes it unclear how those measures will affect future PM3 outcomes. For example, WSDOT has implemented new HOT lanes on I-405. These value pricing initiatives provide congestion relief options, but NPMRDS version 1 data are not able to accurately track changes in performance on these contiguously located facilities. It is unclear how effectively NPMRDS version 2 will track these changes. Adopting PM3 targets would make it difficult for WSDOT to account for these types of highly beneficial projects.

The FHWA regulations will allow the WSDOT to revise those targets to be more aggressive in a few years, if the newer adopted programs have their intended effects and the version 2 NPMRDS data capture those forecast improvements. Therefore, an initial, conservative position of “trends will stay the way they are,” with the ability to adopt more stringent targets in two years, is a good approach to the initial PM3 targets.

The last target setting activity that had to take place was that MPOs and Rural Transportation Planning Organizations (RTPO) needed to select targets. Importantly, WSDOT spent a considerable amount of time and effort working with regional MPOs and RTPOs as the PM3 procedures and targets were developed. This means that MPOs and RTPO staff were well informed about their options, the requirements of the MAP-21 submittals, and the data and analytical packages available to them. After careful consideration, all the MPOs and RTPOs in the State chose to adopt the statewide targets. These agencies saw two major advantages of adopting this approach: (1) it limits the resources that an MPO would need to allocate to the effort to both compute and then justify the adoption of different target outcomes; and (2) for areas with smaller congestion growth, this also removes political pressure to achieve more congestion relief than could be achieved.

Target Setting Example—North Carolina Department of Transportation

Starting in fall 2017, the NCDOT began the target setting process for the PM3 measures. The effort included an array of technical and institutional activities: trend analysis, performance forecasting, subject matter expert interviews and workshops, leadership coordination, workshop facilitation, and documentation and rationale supporting final target submissions. Figure 5 outlines the process that NCDOT followed for target setting.

Figure 5. Diagram. Overview of North Carolina Department of Transportation target setting process.
(Source: North Carolina Department of Transportation.)

Target Definition

The process initiated through a compilation of key definitions, terms, and requirements associated with each measure, and also included a detailed schedule of activities and meetings required to meet the target submission deadlines.

Development of PM3 Reliability Measures

Through the use of the NPMRDS and other NCDOT data sources, a detailed trend analysis was conducted for each measure at the statewide, system, and regional level. These data were aligned with work zones, truck weigh stations, and other critical network impacts to determine the level of impact on each performance measure. An evaluation of a more than 5-year trend at the Statewide, regional, and corridor scale for the System Performance and Freight measures was made.

An issue emerged when using the NPMRDS in that vendor supplying the data were changed starting in 2017. Discontinuities in the resulting trends were noticed when this change occurred for non-Interstate NHS roadways and the freight reliability measure. To compensate, it was assumed that 2017 was the baseline value for the measures and that the trend rate up to 2016 would be applied to extrapolate the baseline. Figure 6 through figure 8 present 5-year annual trends (2013 through 2017) for the three measures, as well as trend lines and associated potential target ranges.

Figure 6. Graph. Percent of person-miles traveled on the Interstate that are reliable—trend and target ranges.
(Source: North Carolina Department of Transportation.)

Figure 7. Graph. Percent of person-miles traveled on the non-Interstate National Highway System that are reliable—trend and target ranges.
(Source: North Carolina Department of Transportation.)

Figure 8. Graph. Interstate truck travel time reliability—trend and target ranges.
(Source: North Carolina Department of Transportation.)

Each measure was assessed at the regional scale and also aggregated by urban and non-urban facilities and individual corridors. It was found that the measure values for the Charlotte and Raleigh Durham urban areas heavily influenced the statewide values. The Statewide TTRI in 2017 is 1.40. Similar to the LOTTR metrics, statewide reliability is impacted significantly by the Charlotte and Raleigh/Durham regions which average a 1.78 truck travel time reliability and total 17.5 percent of Interstate truck VMT. Because this measure is weighted by facility miles, the non-urban 1.20 truck travel time reliability has a large impact on the Statewide average. Figure 9 presents a map of truck travel time reliability with weigh station locations identified. Weigh stations and work zones impact reliability—for example I-85 in Cabarrus/Rowan and I-85 in Warren County, are examples where work zones have led to degraded truck travel time reliability.

Figure 9. Map. 2017 freight reliability measure map.
(Source: North Carolina Department of Transportation.)

Consideration of External Factors

Several external factors that likely impacted the five-year performance trend and could impact future performance were examined. These factors were considered in conjunction with the trend extrapolations in setting the targets.

• VMT Growth—As presented in figure 10, since 2012, urban VMT has averaged a 2.8 percent increase per year while rural VMT has remained overall steady. VMT growth is a primary driver of decreasing reliability.
• Truck Travel Demand—The Statewide Multimodal Freight Plan projected future commodity flows, showing a 43 percent increase in total tonnage over the next 30 years, especially in bulk commodity groups. It is expected that many of the truck trips associated with the highway commodity flows will travel on non-interstate roadways for some portion of the trip. This growth will have significant impacts on both LOTTR and truck travel time reliability performance.
• Severe Truck Crashes—Crashes involving trucks can particularly impact truck travel time reliability. Per analysis in the Statewide Multimodal Freight Plan, between 2011 and 2015, over 49,000 truck-involved crashes occurred in the State, with the total number of crashes increasing in each year. Ten counties accounted for nearly half of all truck crashes in the State, all of which are located within one of the State’s major metropolitan regions: Charlotte, Triangle, Triad, Asheville, and Fayetteville regions.

Figure 10. Graph. Urban and rural vehicle-miles traveled by system.
(Source: North Carolina Department of Transportation.)

• 2018 to 2027 Planned Statewide Transportation Improvement Program Investments—Interstate and non-Interstate NHS expansion projects planned to be open to traffic by 2021 in the Charlotte and Raleigh/Durham regions were considered. There were 14 committed projects ($712 million) that could positively impact reliability over the next 4 years. Work zones associated with current and planned projects can have a significant impact on reliability; however, project management and phasing strategies can minimize this impact. Incident management strategies and traffic management can also significantly benefit reliability.

Development of PM3 Measures for Peak-Hour Excessive Delay Per Capita and Non-single Occupancy Vehicle Travel Measures

In North Carolina, the Peak-Hour Excessive Delay (PHED) per capita Non-Single Occupancy Vehicle (non-SOV) Travel measures currently apply only to the Charlotte, NC-SC Urbanized Area. A coordination group—the Metrolina Coordination Group—comprised of the multiple affected MPOs within the urbanized area (as well as NCDOT) was formed to conduct the target setting. The group reviewed PHED data for recent years to identify a range of possible targets based on a trendline forecast for the first 4-year reporting period (figure 11). PHED increased year over year, from 14.69 hours per capita in 2014 to 19.30 hours in 2017.

Figure 11. Graph. Recent trends—Annual hours of peak-hour excessive delay per capita for the Charlotte, North Carolina-South Carolina urbanized area.
(Source: North Carolina Department of Transportation.)

The group also reviewed ACS data for recent years to identify a range of possible non-SOV Travel targets. The percent of non-SOV travel has remained fairly steady between 2012 and 2016, ranging from 21.5 percent to 21.8 percent. The trendline shows a slight downward trend, which represents decreasing performance over the year period.

Final Targets North Carolina PM3 Targets

Table 2 presents the final PM3 targets for North Carolina that emerged from the target setting process described above.

(Source: North Carolina Department of Transportation.)