Appendix – Example Ten-Step Process for Work Zone Mobility Performance Measurement using the National Performance Management Research Data Set
An example of how State DOTs can use NPMRDS data to systematically measure work zone mobility performance using a 10-step process is shown in figure 19. The process, which is not required under Federal law, draws upon existing FHWA resources on transportation and work zone performance measurement but is specifically customized to using the NPMRDS as the data source.12 NPMRDS travel time and speed data are the foundational mobility metrics for this process; using those data, State DOTs can calculate a variety of performance metrics (e.g., work zone travel time, speed, delay, travel time variability, and road-user costs). This 10-step process provides for a systematic, repeatable, inexpensive, and easy-to-adopt process for work zone performance measurement. The richness of the NPMRDS data (e.g., 24/7 coverage, intra-hour data granularity, repeatable temporal and diurnal analyses) allows State DOTs to subset and/or combine the data in many ways to suit their specific analysis needs.
Figure 19. Chart. Example process for estimating work zone mobility impacts using the National Performance Management
Research Data Set
Source: FHWA
NOTE: This process is not intended to provide step-by-step assistance on work zone performance management. Rather, it provides a practical approach for using NPMRDS data for work zone performance measurement. For additional detail and explanation on work zone performance measurement concepts, metrics, and methods, please refer to resources available at https://ops.fhwa.dot.gov/wz/decision_support/performance-development.htm.
Step 1 – Gather Construction Information
Work zone mobility impact estimation begins with gathering comprehensive work zone activity data (WZAD)13 on when, where, and how the project is being performed. This information includes work zone attributes such as location, work type, time, impact, duration, lane closures, changes in lane geometry, operational changes, and signage. It also includes contextual information such as alternative routes, immediate transportation network, incident data, and weather data. Having real-time, accurate, comprehensive, and standardized WZAD allows agencies to: 1) provide work zone information to internal and external stakeholders; 2) analyze and manage potential work zone impacts across all stages of project development; and 3) systematically evaluate the performance of past work zone projects.
It is also beneficial to start locating and collecting historical data including traffic volume, safety, mobility, incident, and weather data for the work zone area of interest.
Step 2 – Determine Analysis Type
Analysis type refers to the extent, complexity, scope, and intended use of the performance measurement exercise. The analysis can range from a simple descriptive summary of work zone performance to diagnostic analyses explaining the reason for the performance to more complex predictive and prescriptive analyses. Agencies can start with simple metrics such as average work zone speed and travel time and then graduate to more sophisticated analyses such as queuing, travel time reliability, and road-user cost estimation, coupled with temporal and spatial trend analyses. The analysis scope may range from project-level to corridor-level to district/region-level (e.g., State DOT District-level, metropolitan area level) to statewide-level (i.e., the NPMRDS provides the ability to both drill down and aggregate up).
Agencies may choose different analytical tools based on the analysis type, scope, and complexity. For incremental analysis (e.g., monthly analysis of work zone performance as new data gets uploaded to the NPMRDS), Microsoft® Excel is sufficient and preferable due to easy data manipulation and analysis. For longer analysis periods that lead to larger datasets, as well as for complex analyses, agencies may use more sophisticated tools such as SAS®, R, and Tableau® that provide advanced statistical analysis, data manipulation, and data visualization capabilities.
Step 3 – Select Performance Metrics
The next step in the process is to choose the specific measures to calculate, tied to the analysis type from the previous step. Many transportation agencies use average speed, delay, travel time, and queue length as the main work zone mobility performance indicators. These metrics are easy to understand and good for communicating impacts to transportation professionals, contractors, and the traveling public. Metrics range from simple to complex, based on the effort to gather the necessary data to compute the metrics and the complexity of the calculations:
- Simple Metrics – Speed, Travel Time, and Delay. Work zone speed, travel time, and delay are the most straightforward metrics that the NPMRDS enables. Agencies can calculate average speed and travel time, deduce delay, and extend the analysis to include temporal and spatial performance variations and compare current performance against historical averages or agency-defined speed thresholds. These basic mobility performance measures are easy to calculate and to incorporate into existing work zone mobility performance measurement programs.
- May Require Additional Data and Modeling – Queue Length. Although the NPRMDS provides 24/7 speed and travel time data, it provides only AADTs and does not provide intraday traffic volume data. Further, NPMRDS roadway section lengths (i.e., TMC lengths) are sometimes too long (e.g., many miles long in rural areas) to accurately represent the effect of traffic queue buildup and dissipation. Therefore, agencies may need to supplement NPMRDS data with more granular traffic volume and/or additional speed and travel time data so that they can perform accurate queuing and throughput analysis. One option to supplement NPMRDS data is to purchase more-granular probe-vehicle travel time and speed data from third party data providers.
- Advanced Metrics – Travel Time Reliability, Road User Costs. The basic NPMRDS travel time and speed data can be used to calculate derived measures such as travel time reliability and road-user costs. These may involve advanced statistical modeling and data processing as well as additional data points such as intraday traffic volume (e.g., hourly, peak period, peak hour volumes), standard values for road user costs (e.g., value of time), and assumptions around travel time reliability expectations.
The following case studies, available at https://ops.fhwa.dot.gov/wz, showcase the use of probe-vehicle data for work zone performance management:
- A Policy-Driven Approach for Work Zone Mobility Performance Management – Ohio DOT
- Utilizing Probe-Vehicle Data for Work Zone Mobility Performance – Virginia DOT
Now is also a good time to identify visualizations, charts, and graphics to produce. Visualizations may range from simple line and bar charts to more sophisticated visualizations such as heat maps, box plots, and contour plots, depending on the audience. For example, line and bar charts may be best suited for non-technical audiences, while more sophisticated visualizations such as heat maps, box plots, and contour maps are better directed at technical audiences and for deep analysis purposes.
Examples of visualizations used by different State DOTs are available at https://ops.fhwa.dot.gov/wz/decision_support/perf_meas_examples.htm.
The NPMRDS travel time and speed data provide a good foundation to build a variety of work zone mobility performance metrics and charts. For the sake of simplicity in this document, the main measures of work zone mobility performance are speed, travel time, and delay. Because travel time and delay are well correlated with queue lengths, the case study does not focus on estimating queue length. However, agencies may want to estimate queue lengths for various purposes, including examining the impacts of work zones on off-ramp, on-ramp, and adjoining arterial street operations. Agencies can estimate queue lengths using Highway Capacity Manual computational methods with travel time and speed data from the NPMRDS coupled with agency traffic volume data.
Step 4 – Download NPMRDS Data
The NPMRDS Massive Data Downloader allows users to download data from the NPMRDS. A key step in the process is to map the “work zone area of interest”14 to specific NPMRDS roadway segments (i.e., TMCs). The work zone area of interest generally includes:
- Advance area and transition area
- Work zone activity area
- Post-activity / termination area
Depending on the scope of the analysis, agencies also can include alternate routes, nearby intersections, and the immediate transportation network in their analysis.
The following are the key activities to be performed in this step:
- Select TMC segments
- Select the analysis timeframe, including time period before (baseline), during, and after the work zone
- Choose the units for speed and travel time
- Choose the aggregation (averaging) level: 30-, 15-, 10-, or 5-minute
- Download and import the data into the analysis tool of choice (e.g., Excel, SAS®)
Baseline period is a reference period that represents normal traffic operations without the work zone. The baseline may be a few months to a few years, depending on data availability and road network changes over time.
The data aggregation level describes the granularity of the performance measurement. In most cases, hourly data provide a good understanding of congestion effects of work zones. Further, the data file sizes may be more manageable with hourly data. However, agencies may choose to use more granular aggregation levels if they want to better understand the intra-hour congestion build up and dissipation effects of work zone traffic.
Note on missing data and data cleaning: There may be situations where NPMRDS data are missing (null value) and/or the data may not line up with the rest of the data in the dataset. Missing data are most likely during low volume periods (nighttime) or segments with lower AADT (e.g., rural areas). As a general rule, agencies may choose to ignore data rows with incorrect or missing data but should be cautious in drawing conclusions from periods or segments with sparse data. The NPMRDS help link shown previously provides information on how to handle missing data.
Step 5 – Establish Performance Thresholds
Performance thresholds indicate the acceptable level of performance degradation that the transportation system can handle, or road users can tolerate. While agencies strive to maintain performance under all conditions, it may not be practically feasible to do so, given the capacity constraints that work zones may impose. Performance thresholds allow agencies to set acceptable limits for work-zone-induced delay, speed reduction, or travel time reliability impacts; and consequently, enable agencies to manage the performance of their work zones to those established thresholds. Thresholds may vary from agency to agency, and from project to project, depending on the roadway classification, work type, project duration, work zone length, lane closure hours, etc. Some agencies divide delay thresholds into categories such as minor, moderate, and severe congestion. Agencies may also choose to adjust delay thresholds based on road user expectations—for example, delay tolerance may be lower for work zones in less congested areas and higher for projects on already congested corridors. Agencies may also implement performance thresholds for alternate routes associated with work zone projects.
Examples of performance thresholds include:
- Delay and travel time thresholds
- Speed thresholds
- Queue thresholds
- Travel time reliability thresholds (e.g., acceptable planning index, buffer index)
- Traffic volume and throughput thresholds (though the NPMRDS provides AADT, it does not provide actual traffic volume and throughput data; agencies may need to get these data from other sources)
Figure 20 presents an example of performance thresholds and associated NPMRDS data showing actual performance against established thresholds.
Figure 20. Graph. Example of Performance Threshold
Source: FHWA
The Ohio Department of Transportation (ODOT) uses a policy-driven approach to manage work zone mobility. During project planning, lane closures are allowed on highways only if the volume and queue requirements of the Permitted Lane Closure System are met. The maximum volume threshold is 1,000 to 1,490 vehicles per hour per open lane (depending on truck percentages and terrain), and the allowable queue threshold is 0.75 miles. If the project team estimates queues to be greater than 0.75 miles, the associated lane closure will not be allowed, and the project team must submit an exception request along with appropriate queue and delay mitigation strategies. Similarly, ODOT strives to meet a minimum work zone speed threshold of 35 miles per hour. ODOT staff monitor queues through freeway cameras (where coverage is available) and speeds using probe-vehicle data. The monitoring results are appropriately communicated to the respective ODOT District and project engineers. According to ODOT’s processes, the project team takes appropriate action to improve and address performance issues. More information on ODOT’s work zone performance management approach is available at https://ops.fhwa.dot.gov/wz.
Step 6 – Perform Calculations
This step involves converting raw data from the NPMRDS into metrics and information for decision-making. Travel time and speed data from the NPRMDS form the basis of all the performance measures to be calculated, using which agencies can calculate average speed and travel time, analyze travel time reliability, and develop speed profiles and travel time trends.
The discussion presents an example of how to aggregate travel times at the corridor-level using NPMRDS TMC data (i.e., multiple TMCs make up a corridor). Refer to the NPMRDS help documents for additional information on calculating other metrics as well as recommended best practice “dos and don’ts”. The simplest way to aggregate travel times across multiple, sequential TMCs is to sum the individual TMC travel times at a given time of day to calculate the average travel time for the corridor. This method of travel time calculation is called Estimated Travel Time or instantaneous travel time (figure 21). This method is most commonly used for larger data aggregations (e.g., for NPMRDS data averaged over an hour or more).
Figure 21. Equation. Estimated Travel Time
Source: FHWA
Average Corridor Speed at a given time of day is computed by dividing the length of the corridor by the travel time. Figure 22 shows the formula for calculating this metric.
Figure 22. Equation. Average Corridor Speed
Source: FHWA
In the case of more granular analyses, agencies may choose to use the Actual Travel Time method for aggregating travel times across TMC segments. This approach works especially well when the travel times of individual TMC segments are higher than the aggregation interval itself. Actual travel time at a given time of day is calculated by traversing the corridor through the individual TMC segments while incrementing the date/time stamp for the next TMC segment by the travel time of the previous TMC segment (figure 23). The Average (Actual) Corridor Speed can then be calculated by dividing the Actual Travel Time over the length of the corridor.
Figure 23. Equation. Actual Travel Time
Source: FHWA
Using the basic travel time and speed data, additional metrics such as travel time reliability and road user costs may be calculated. Additional data will be needed to calculate these measures, including free flow speed and travel time data, hourly traffic volumes, vehicle occupancy, and dollar value of time for road users.
For a quick representation and illustration of the calculated measures, it is best if they are presented using graphical visualizations to demonstrate and understand trends and patterns. Corridor travel times and reliability can be visualized using trend lines, while speed profiles can be visualized using heat maps.
Additional resources on calculating performance measures using NPMRDS data are available at https://npmrds.ritis.org/analytics/help/#npmrds. Resources on work zone performance measures are available at https://ops.fhwa.dot.gov/wz/resources/publications/fhwahop13011/index.htm. Resources on calculating travel time reliability are available at https://ops.fhwa.dot.gov/publications/tt_reliability/TTR_Report.htm#Whatsteps.
Step 7 – Interpret Results
This step brings together of all the data analysis and calculations. It includes:
- Identifying whether there is a performance issue
- Assessing whether performance thresholds are met
- Conducting root cause analysis:
- What, why, how, when, who, how much?
- Correlating with actual construction information
- Correlating with contextual information – incidents, weather, project team input, public feedback, crash data, etc.
- Reaching out to the project team, district/regional traffic engineering teams, and traffic operations centers (TOCs) to better understand any project-specific or regional issues that may have an influence on the work zone
- Identifying potential fixes for performance issues:
- If there is a performance issue, can something be done to fix it – immediately, later, when, how?
- Is it something that better planning or design could have prevented?
- Setting up recurring checkpoints for reviewing performance with concerned stakeholders (e.g., monthly reviews with work zone mobility steering committee)
The Ohio Department of Transportation (ODOT) produces work zone speed charts on a monthly basis using probe vehicle data. Traffic managers evaluate the speed analysis charts monthly to identify any work zone performance issues. For example, ODOT determined for a particular project that the construction crew was closing lanes an hour earlier than allowed in the plan. ODOT engineers used this information to not only remind the project team about the importance of sticking to the permitted lane closure plans but also to re-emphasize the same message on a broader level to all project teams.15
Step 8 – Take Action/Intervention
This is the point in the process where agencies may choose to take action based upon the performance measurement results. Actions can include:
- Corrective actions to rectify project and process issues, including changes to project design and construction management (e.g., changing lane closure timings, allowed lane closures, alternate route changes)
- Investigative actions to further understand the underlying cause(s) of a performance issue
- Corroborative/Confirmative actions that lead to verification of cause and effect, peripheral/external influences, and other factors that may affect work zone performance
- Informational actions to communicate the performance measurement results to concerned parties including the traveling public
- Contractual actions to enforce existing contract terms aimed at ensuring better work zone performance and to modify contract terms towards delivering better work zone mobility
- Incentive or disincentive actions depending on whether performance expectations are met
- Procedural actions in response to unmet performance expectations (e.g., actions in response to delays caused by unallowed lane closures)
- Forward-looking actions aimed at improving future practices and policies
Step 9 – Develop Report
The purpose of this step is to appropriately document the results of the performance measurement and use them for ongoing process improvement. The reporting should be:
- Actionable
- Relevant
- Written with a learning and process improvement mindset
Step 10 – Document Lessons Learned
This is the final step in performance measurement. Just like in any other analysis and assessment activity, it is important to identify lessons learned and institute any changes or updates to existing policies and practices. Archiving lessons learned in easily accessible and searchable formats (e.g., checklists, searchable databases) enable practitioners to apply the lessons learned during the regular course of project planning, design, and implementation. In addition to lessons learned around overall work zone management, there may be lessons to apply to performance measurement processes and methods leading to more efficient and effective performance measurement.
Some of the considerations for documenting lessons learned include:
- What to archive:
- Raw data
- Curated and processed data
- Assumptions and contextual information
- Information and insights
- Performance results, issues, cause, and effect (root cause)
- Actions taken and any resultant benefits of the actions
- Lessons and suggestions for improvement
- What to keep in mind while archiving:
- Strive for easily searchable and retrievable information
- Preserve the context of the performance measurement
- Keep a learning and continuous improvement mindset → living process and artefacts
- Adapt policy, procedures, and processes based on lessons learned
Incorporating findings and lessons learned in agency process reviews. An effective channel for documenting lessons learned is to incorporate findings from performance management activities into the agency’s process reviews. Documenting the findings and actions in process reviews may help agencies to adopt a continuous, systematic approach to performance management by tracking action items across process review cycles and across program areas.
California Department of Transportation (Caltrans) resident engineers regularly conduct post-construction work zone mobility evaluations that document and evaluate the design/construction features of road projects in the context of contractor work windows, traffic delays, contractor claims for insufficient language in the specifications, contract change orders due to deficiencies in their lane requirements charts, and any special project provisions. Incidents in the vicinity of the work zone are evaluated to understand their impact on work zone performance, and any incidents caused by lane closures are appropriately attributed to the work zone. The information from the evaluations is used to identify lessons learned and improvement opportunities for work zone system planning and management. Caltrans districts include work zone performance evaluations in their quarterly and annual Mobility Performance Reporting and Analysis Program reports that cover broader congestion reports and metrics.16
12 Example FHWA work zone performance measurement resources are available at https://ops.fhwa.dot.gov/wz/decision_support/perf_measurement.htm and overall transportation performance measurement resources are available at https://ops.fhwa.dot.gov/perf_measurement/.
13 FHWA started the Work Zone Data Initiative (WZDI) to develop a recommended practice for managing WZAD; to create consistent language by initiating the development of a data dictionary and supporting implementation documents; and for communicating information on work zone activity across jurisdictional and organizational boundaries. The WZDI developed seven standardized and comprehensive WZAD categories, including: 1) Work Zone Planning and Project Coordination, 2) Work Zone Impact Analyses, 3) Construction and Maintenance Contract Monitoring, 4) Real-time System Management/Traveler Information Provision, 5) Safety and Mobility Performance Measurement, 6) Law Enforcement and Emergency Service Providers, and 7) Connected and Automated Vehicles Hardware Needs and System Readiness. FHWA-HOP-18-083. Available online: https://ops.fhwa.dot.gov/publications/fhwahop18083/index.htm.
14 See the Manual on Uniform Traffic Control Devices (MUTCD) for more information on the different areas in a work zone – https://mutcd.fhwa.dot.gov/.
15 Project communications with Ohio Department of Transportation.
16 Project communications with California Department of Transportation.
For more information on FHWA’s Work Zone Management Program, please contact Jawad Paracha, FHWA Office of Operations: Jawad.Paracha@dot.gov
Additional resources on work zone performance management can be found at: https://ops.fhwa.dot.gov/wz/ and https://www.workzonesafety.org/swz/
FHWA is engaging with State DOTs and conducting research to understand work zone mobility performance measurement best practices, challenges, and opportunities. The information will be used to increase awareness on data, tools, and methods for systematic work zone performance measurement across all stages of project development including planning, design, construction, and post-construction. Topics of interest include use of probe data and mainstreaming performance measures into agency policies and processes, including work zone process reviews and incorporating work zones into TSMO initiatives. This case study is one of a series of resources on work zone mobility performance measurement.
July 2021
FHWA-HOP-20-029
