Approaches to Forecasting the Third Performance Management Rulemaking (PM3) Measures for Target Setting
Chapter 3. Emprical Analysis: Assessing At-Risk Highway Segments
Often, there are reporting segments on a State DOT’s or MPO’s transportation network that are “vulnerable” to failing to meet performance thresholds; this applies to the system reliability (Level of Travel Time Reliability (LOTTR)) and PHED measures which are threshold based. For example, an Interstate reporting segment with an LOTTR of 1.49 meets the 1.50 threshold for being reliable, but just barely. Similarly, the reporting segments with an LOTTR of 1.51 do not meet the 1.50 threshold. Because the definition of reliability depends on LOTTR metrics for 4 time periods, it also is useful to know how many of the time periods are vulnerable.
These reporting segments that are “on the cusp” may be viewed as vulnerable links for the next performance period. Different scenarios can be tested such as “what would happen if all the metrics’ values close to the threshold, surpass the threshold in four years?” Identifying these vulnerable portions of the network and accounting for them in target setting is a challenge for DOTs as it involves analysis techniques not commonly found in commercial tools. As with trend extrapolation, VMT trends on the vulnerable segments should be considered, as well as any capacity or operational improvement projects that are being planned. Conversely, the decline in performance due to major planned work zones—such as bridge and interchange replacements—should also be considered in setting targets.
Data Requirements
The data requirements for calculating the performance metrics and measures are outlined in FHWA publication FHWA-HIF-18-024. In general, the data required are continuously collected travel time data for relatively short one-way highway segments.
Analysis Tools
FHWA provides detailed step-by-step guidelines on how to calculate the performance metrics and measures in publication FHWA-HIF-18-024.
For forecasting the performance measure/metrics and setting targets, this “Assessing At-Risk Highway Segments” approach can be accomplished with a spreadsheet-based or similar analysis.
Forecasting Measures and Target Setting Process
The following detailed steps can be used for forecasting measures/metrics and identifying a target using the “Assessing At-Risk Highway Segments” approach.
Step 1. Using the data sources outlined in FHWA publication FHWA-HIF-18-024, gather all the relevant information to calculate the PM3 performance metrics and measures for system reliability, truck reliability, and peak hour excessive delay.
Step 2. Using the detailed methodology outlined in FHWA publications FHWA-HIF-18-040 and FHWA-HIF-18-024, respectively, calculate the PM3 performance metrics and measures for system reliability, truck travel time reliability and peak hour excessive delay.
Step 3. Identify segments (TMC) of the roadway network which can be considered as vulnerable. For the LOTTR metrics, time periods with values from 1.40 to 1.49 are a reasonable starting point for determining vulnerability. For the PHED measure, the threshold is determined based on speed limit. Vulnerable segments with 5 miles per hour (mph) over the TMC threshold can be considered as vulnerable; if the speed limit-based threshold is 27 mph, then the vulnerability threshold is 32 mph. Vulnerability can be used in multiple ways to provide more insight into system condition. For example, segments with an LOTTR value between both 1.40–1.49 and 1.45–1.49 can be identified.
Step 4. Account for external factors which are typically outside of the control of State DOTs as well as internal factors that are under the control of State DOTs. Expected VMT growth over the target time horizon is the main factor to consider.
Step 5. Reclassify the LOTTR and PHED metrics for each of the original segments by relaxing the original thresholds to the new vulnerability thresholds. In the above examples, these would be 1.40 and 1.45 for LOTTR. Also, keep track of number of time periods for each segment than fail the new thresholds. While not strictly relevant for the measure calculation, these numbers could be of value for other purposes, e.g., identifying off-peak times that have reliability problems.
Step 6. Considering the impact of external and internal factors (step 4), as well as the forecasted performance (from step 3), State DOT’s will select a performance target could be selected based on their level of comfort.
Example
The above step-by-step process for forecasting performance metrics and measures and setting target are implemented using NPMRDS data. In this section, the following measures were analyzed—Percent of Person-Miles Traveled on the Interstate that are Reliable and Percent of Person-Miles Traveled on the Non-Interstate NHS that are Reliable.
Step 1: Gather Data
Required data was gathered as indicated above.
Step 2: Calculate Performance Metrics
Percent of Person-Miles Traveled on the Interstate that are Reliable
Using the above-described methodology, the Interstate reliability measure was calculated for 2014, 2015 and 2016 (using NPMRDS version 1 data) as well as for 2017 (using NPMRDS version 2 data). Table 2 depicts the 2017 metric results, breaking down the LOTTR values in six bins ranging from reliable to unreliable. The bin of interest for short-term forecasting in the 1.40–1.49 range.
Table 2. Percent of person-miles traveled on the Interstate that are reliable.
| LOTTR Range |
Percent of Reliable Person-Miles |
| 1–1.24 |
65.0% |
| 1.25–1.39 |
6.2% |
| 1.40–1.49 |
5.5% |
| >= 1.50 |
23.3% |
(Source: FHWA, using NPMRDS data.)
Step 3: Conduct Analysis
The two measures addressing the percent of person-miles traveled that are reliable are peculiar in their treatment of what is considered reliable versus not. These measures assume a threshold of 1.50 LOTTR as the defining metric in which a segment is reliable. This presents a situation whereby small changes in roadway or travel conditions may cause segments that are barely less than 1.50 LOTTR to flip and become “unreliable.” A sensitivity analysis was conducted to determine what would be the resulting measures under different scenarios.
The base scenario are the current conditions measured for 2017; scenario 1 assumes that TMC segments that currently have an LOTTR of 1.45 or more will reach an LOTTR of 1.50; scenario 2 assumes that TMC segments that currently have an LOTTR of 1.40 or more will reach an LOTTR of 1.50; and so on. The following scenarios continue progressively assuming lower thresholds for TMC segments, reaching the unlikely scenario 7, which assumes that TMC segments that currently have an LOTTR of 1.15 will reach an LOTTR of 1.50. This is shown in table 3 and figure 3 show how each of the performance measures changes under each progressive scenario.
Table 3. Sensitivity analysis for Interstate and non-Interstate National Highway System reliability.
| Risk Scenario |
Traffic Message Channel Level of Travel Time Reliability Threshold |
% Interstate Reliable |
% Non-Interstate National Highway System Reliable |
| Base |
1.50 |
82% |
86% |
| 1 |
1.45 |
80% |
82% |
| 2 |
1.40 |
78% |
76% |
| 3 |
1.35 |
75% |
66% |
| 4 |
1.30 |
73% |
56% |
| 5 |
1.25 |
70% |
46% |
| 6 |
1.20 |
69% |
37% |
| 7 |
1.15 |
66% |
29% |
(Source: NPMRDS and FHWA Occupancy Factor.)
Figure 3. Chart. Sensitivity analysis for Interstate and non-Interstate National Highway System reliability.
(Source: FHWA.)
Step 4: Account for Additional Factors
External Factors
The following external factors (also called exogenous factors) that are typically outside the control (at least operationally) of transportation agencies are considered—Travel demand; Truck demand; Economic trends; Tourism; and Population. The sensitivity analysis conducted above provides the range of possible outcomes. Examining external factors will help to narrow the range of possibilities.
- Travel Demand: Over the last few years, assume that the VMT has been increasing at a rate of 1.7 percent annually on rural facilities and 3.9 percent on urban facilities.
- Economic Trends: Assume that economic trends have been trending positive over the last few years. Nonfarm employment grew 2.5 percent annually, while the new housing permits issued grew 12.4 percent annually.
- Population: Assume that population has been steadily growing at an annual rate of 1.0 percent in rural and 1.3 percent in urban areas over the last few years.
Step 5: Set Targets
Increasing travel demand, improving economic conditions, and increasing population trends indicate that the system performance could potentially worsen in the future. Due to the lack of availability of extended historical data, targets for the performance measures could be selected using a conservative approach. For 2-year targets, 1.35 was selected as the LOTTR threshold; and for 4-year targets, 1.25 was selected as the LOTTR threshold.
Percent of Person-Miles Traveled on the Interstate that are Reliable
The estimated two-year and four-year targets using the analytic procedure for this measure are the following:
- Two-year Target—75.0 percent
- Four-year Target—70.0 percent
It is important to note that the process of target setting is consensus based. The results of technical analyses can inform the process, but ultimately analysts use their own judgment in establishing targets for both State DOTs and MPOs.
