Office of Operations
21st Century Operations Using 21st Century Technologies

APPENDIX A. ASSESSING PERFORMANCE ADVANCES ACROSS OBJECTIVES

The following table provides a summary of the performance advances across the twenty-seven performance measures and the eight program objectives.

Table 22. Performance Advances across the Twenty-seven Performance Measures and the Eight Program Objectives.
Objectives Performance Measures Highlights Assessment of Performance Measures
Objective 1: Build and sustain relationships with multi-disciplinary partners to expand RWM deployments PM #1: Number of agencies participating in road weather R&D projects
  • 4 State DOTs participated in the Pathfinder Project.
  • Seven public agencies have participated in the development and use of the RWMP Capability Maturity Framework.
  • 3 State DOTs have participated in the IMO program.
  • 7 State DOTs have been involved in V2I implementation activities.
  • 24 State DOTs have participated in weather data environment research activities.
  • 5 State DOTs have been involved in WRTM implementation support activities.
Measure adequately captured
  • R&D activities encompass all major initiatives of the RWMP, including the Pathfinder Project, RWMP Capability Maturity Framework, Weather Data Environment, IMO Program, V2I Application Development Efforts, and WRTM implementation support activities.
  • Data was collected through the FHWA RWMP via interviews with staff and review of the R&D program.
PM #2: Number of agencies participating in, and benefiting from, road weather management stakeholder meetings/workshops
  • The number of State DOTs attending the annual RWMP meetings has decreased with more than a 50 percent reduction over two years (2012 to 2014). However, the overall attendance in Stakeholder workshops is increasing with a greater proportion of private sector attendees.
  • Majority of participants expressed that the meetings and workshops met or exceeded their expectations and provided information that is useful and relevant to their duties.
Measure captured to extent possible given available data
  • From 2001 to 2003, focus of meetings was MDSS. From 2004-2009, both Clarus and MDSS were discussed. From 2010, focus was on broader RWMP activities. Now these meetings include annual RWMP stakeholder meetings and WRTM meetings held every two years.
  • In addition to States, other private and public agencies attend the stakeholder meetings. These agencies are not included in the measure, as this level of detail is not available for the earlier Clarus/MDSS meetings.
  • Data was collected from the FHWA RWMP records.
Objective 2: Ensure road weather management investments improve highway performance PM #3: Number of agencies that collect and report road weather-related performance measures to the public
  • Among the State DOTs surveyed, 23 DOTs reported regularly collecting and reporting some form of road weather performance measures. Eight States reported they did not collect and report road weather performance measures, and eight respondents were uncertain.
Measure adequately captured for State DOTs
  • This is a new measure.
  • Data was collected from the State DOT Survey.
  • Data shows that 58% of State DOTs report collecting performance measures.
PM #4: Number of agencies that have a process for evaluating the ROI or net benefit of their road weather management investments
  • The majority of States reported that they did not have a process or were not sure regarding evaluating ROI or the net benefits of road weather management investments.
  • Out of the respondents, five agencies do have a process in place.
Measure adequately captured for State DOTs
  • This is a new measure.
  • Data was collected from the State DOT Survey.
  • Data shows low level of ROI use for road weather management systems. 
PM #5: Reductions in agency costs of weather-related maintenance and operations activities
  • Since the publication of the 2012 report, limited evaluation reports have been published documenting savings, but published case studies continued to show that winter maintenance costs decreased as the use of weather information increased or its accuracy improved.
  • Michigan DOT's benefit-cost calculations for deployment of RWIS and Maintenance Tracking using GPS were positive.
  • Idaho DOT's use of RWIS to monitor winter maintenance response has demonstrated significant reductions in winter maintenance costs.
Measure captured to extent possible given available data
  • National level statistics are annually reported, but individual case studies do not have a longitudinal aspect.
  • National numbers for the cost of winter maintenance activities are hard to attribute to RWMP performance. Long term trends in the data can be indicative of overall performance; however, seasonal and geographic variation in weather and road weather conditions and local practices create significant variation in the data.
  • Data was collected from the Highway Statistics publication series (2001-2012), RITA ITS Benefits Database, and internet research for case studies.
PM #6: Reduction in number and types of fatalities and crashes attributed to adverse weather nationally
  • Although the national level data had been showing a decreasing trend of the number of fatal crashes occurring during inclement weather, 2013 shows a slight increase.
  • Practices such as variable speed management systems, ice warning systems, and automated anti-icing spray systems have demonstrated significant benefits. For example, in Colorado, implementation of a variable speed management system consisting of a complete RWIS, resulted in zero winter related crashes in one section of highway in Snowmass Canyon.
Measure captured to extent possible given available data
  • National level statistics are annually reported, but individual case studies do not have a longitudinal aspect. It is hard to determine the contribution of specific strategies on national crash rates that can be attributed to the RWMP.
  • Data was collected from FARS Database, NHTSA databases, Highway Statistics publication series (2001-2012), RITA ITS Benefits Database, and internet research for case studies.
PM #7: Reduction in the extent of capacity losses and delays due to fog, snow, and ice events including freight
  • Although limited evaluations have been found beyond those reported in 2012, systems have been implemented which demonstrated benefits on traffic flow.
  • One example came from Utah, which implemented a weather responsive signal control system. During severe winter weather events, travel times were improved by 3 percent and reduced overall stopped times by 14.5 percent.
Measure captured to extent possible given available data
  • Individual case studies do not have a longitudinal aspect.
  • National numbers for freight delays due to weather events are not readily available.
  • Data was collected via RITA ITS Benefits Database and internet research.
PM #8: Increase in travel time reliability or decrease in variability due to road weather management strategies during adverse weather scenarios
  • No new reliability-related studies for weather were found since 2012.
  • Very few agencies track reliability measures, and even the ones that do, do not distinguish between the various causes of reliability.
Measure captured to extent possible given available data
  • Individual case studies do not have a longitudinal aspect.
  • Data was collected via RITA ITS Benefits Database and internet research.
PM #9: Reduction in the number of tons of salt or chemical usage in the U.S. normalized by Winter Severity Index
  • The use of a Winter Severity Index (WSI) has continued to gain recognition as a way to gauge the relative severity of winter weather across various time frames or geographic regions.
  • Idaho DOT has reported a significant reduction in winter maintenance costs due to the use of a winter mobility index.
Measure captured to extent possible given available data
  • While national numbers for salt use are available, normalizing salt use by State for evaluation purposes is a challenge.
  • Data was collected from the USGS Minerals Yearbook: Salt (2006-2013), Salt Institute, AASHTO Standing Committee on Maintenance, RITA ITS Benefits Database, and internet research.
Objective 3: (Advance) Transportation, weather, and research communities' use of and reliance on fixed and mobile road weather observations PM #10: Number of State DOTs that are participants in the MADIS program
  • 12 States have participated in the MADIS program as of April 2015.
Measure adequately captured
  • This is a new measure.
  • The transition from Clarus to MADIS is a new activity since the 2012 update. The RWMP is supporting NOAA by working with State DOTs to secure data sharing agreements and helping to ensure data quality by integrating quality checking algorithms into the system.
  • Data was collected from the RWMP records.
PM #11: Number of State DOTs that subscribe to road weather products and services
  • Results show that subscription to National Weather Service Products held steady since 2013.
Measure adequately captured for State DOTs
PM #12: Number of State DOTs collecting mobile observations of road weather data from appropriate vehicle fleets
  • Overall, 50 percent of States surveyed collect real-time field data from maintenance vehicles.
  • Results of the survey show that collecting data fleet-wide is starting to become a practice; as many as three DOTs reported using 100 percent of the fleet to collect data, compared to zero in 2013.
Measure adequately captured from State DOTs
  • Major activities/projects of interest for this update included: IMO, VDT Research, and DMA.
  • Data was collected from State DOT Survey.
PM #13: Number of State DOTs reporting the use of ESS in operations and maintenance activities
  • In the State DOT survey, the respondents reported a total of 2,473 ESS, which is a slight decrease from the previous update.
  • Ninety-five percent of State DOTs reported using ESS data for decision-making. Majority of agencies also use ESS data to provide current conditions to traveler information systems (61 percent) and input for segment-level forecasts (58 percent).
Measure adequately captured from State DOTs
  • There is a slight change in definition in measure which makes it hard to fully compare with previous results. Previous updates tracked only the number of ESS sensors.
  • Data was collected from ITS Deployment Statistics and State DOT Survey.
Objective 4: Advance the state-of-the-art for mobile sensing and integrating vehicle data into road weather applications PM #14: Number of/percentage of responding agencies using mobile data-based applications in road weather management
  • Fifty-eight percent of States are considering applications to leverage data collected from mobile platforms.
  • Twenty-three percent of States have developed applications that input real-time data from vehicle fleets.
  • Five percent have developed applications that utilize data derived from vehicle fleets and vehicle-to-infrastructure connectivity.
Measure adequately captured from State DOTs
  • This is a new measure.
  • Data was collected from State DOT Survey.
  • Data shows strong interest but low level of current capability.
Objective 5: Advance the state-of-the-practice by promoting tailored management strategies for different regions PM #15: Number of States disseminating weather advisory and other road weather information to travelers
  • After significant increases between 2004 and 2007, the use of websites/social media and DMS steadied around the same level in 2015.
  • There was a significant increase in the use of 511 to disseminate information to travelers, while the use of HAR decreased.
  • Overall, providing road condition information on DMS is more prevalent, followed by agency hosted social media and other mobile applications. Road condition information on DMS and HAR are the most widely deployed.
Measure adequately captured from State DOTs
  • Data was collected from ITS Deployment Statistics and State DOT Survey. 
  • Trends have plateaued but a large number of responding states provide information to travelers during adverse weather. After significant increases between 2004 and 2007, the use of websites/social media and DMS steadied around the same level in 2015.
PM #16: Number of agencies using control and treatment strategies during weather events
  • The most widely deployed strategy, either partially or statewide, is employing traffic incident management practices (83 percent).
  • The least commonly used strategy is ramp meter adjustment (20 percent).
Measure adequately captured from State DOTs
  • Data was collected and State DOT Survey.
PM #17: Number of agencies that have participated in or conducted RWM capability maturity assessment exercises
  • The framework was recently developed in 2014. While participation in these exercises is currently low, the RWMP anticipates that capability maturity exercises will increase and intends to track the participation levels in the coming years.
Measure adequately captured
  • This is a new measure. Future updates will be able to track this measure over time.
  • Data was collected from RWMP records.
PM #18: Number of agencies that coordinate with their local forecast offices for road weather management and operations
  • 75 percent of respondents reported at least some coordination with the NWS local forecast office.
  • Nearly 8 percent of DOTs reported using publicly available information provided by the media and NWS, despite not having direct coordination.
Measure adequately captured from State DOTs
  • This is a new measure. Future updates will be able to track this measure over time.
  • Data was collected from State DOT Survey.
  • Data shows strong linkages with NWS among responding States DOTs.
Objective 6: Weather-related decision support technologies are integrated into traffic operations and maintenance procedures PM #19: Number of agencies adopting MDSS technologies and methods
  • The percentage of State DOTs with statewide MDSS deployment has remained constant, and partial MDSS use has decreased.
  • Perhaps more significant is that the number of State DOTs expressing a need for MDSS increased, with a corresponding decrease in those agencies reporting no need for a system.
Measure adequately captured from State DOTs
  • Data was collected from State DOT Survey.
  • MDSS use has stayed constant but program emphasis on MDSS adoption as decreased. Perhaps more significant is that the number of State DOTs expressing a need for MDSS increased, with a corresponding decrease in those agencies reporting no need for a system.
PM #20: Number of agencies using other weather-related decision-support tools
  • Respondents indicate an overall decrease in the use of weather-related decision support tools for road weather management, and a few states (12.5 percent) reported not using any tools
  • Providing traveler information remains the most used tool, followed by coordination with other agencies, support of non-winter maintenance, traffic control and management, and seasonal load restrictions.
Measure adequately captured from State DOTs
  • Respondents indicate an overall decrease in the use of weather-related decision support tools for road weather management, and a few states (12.5 percent) reported not using any tools.
  • Data was collected from State DOT Survey.
PM #21: Number of agencies reporting use of appropriate analysis tools to factor weather impacts and strategies
  • 50 percent of the respondents either did not use or were not aware of whether their agency used weather-responsive tools and models, which is less than the previous update, indicating an increased use of these types of tools and models.
Measure adequately captured from State DOTs
  • Potential for more widespread use of these models and tools for weather-related applications. Currently very low capabilities and interest to use analysis, modeling and simulation tools for weather
  • Data was collected from State DOT Survey.
Objective 7: Advance the state-of-the-practice by raising road weather capabilities and awareness across the transportation and weather communities PM #22: Number of agencies and attendees who have taken any of the training courses and workshops sponsored by the RWMP
  • Between July 2013 and May 2015, there were 13 training courses and workshops held for transportation practitioners – eight were offered by RWMP, and five by CITE.
  • A total of 267 attendees participated in these training courses. Participants included staff from State DOTs, local agencies, private consultants, and federal agencies.
Measure adequately captured
  • Training is a key approach to increasing knowledge and capabilities of the transportation community and supporting effective deployment of advanced road weather management strategies.
  • Data was collected from FHWA RWMP records (for stakeholder meetings) and attendance records from CATT Lab.
PM #23: Number of agencies and participants in road weather management webinars led by the RWMP
  • Three RWMP webinars have been held in 2014-2015.
  • 382 people have participated in the RWMP webinars.
Measure adequately captured
  • The RWMP has increasingly used webinars as an outreach tool to promote research results and raise awareness about the availability of guidance documents.
  • Data was collected from the ITS PCB Program records and other webinar sponsors/venues.
PM #24: Number of meetings, site visits or venues where road weather management presentations/briefings were made
  • In the 2013-2014 timeframe, RWMP was represented by program staff or support contractors in nearly 60 meetings.
  • The breadth of meetings that feature RWMP presentations, as well as consistent participation (i.e., multiple-year attendance) continues.
Measure adequately captured from State DOTs
  • RWMP representatives (staff and contractors) also provide presentations, briefings, and demonstrations at various meetings, site visits, or venues – extending the program's reach beyond its own activities.
  • This measure indicates the broader presence that RWMP holds in the transportation and weather community.
  • Data was collected from FHWA RWMP.
PM #25: Number of hits/visits to RWMP website
  • Limited website statistics indicate significant increase in RWMP website use.
Measure adequately captured from State DOTs
  • Data was collected from FHWA RWMP.
Objective 8: Operations community is engaged with climate change & sustainability communities PM #26: Number of public agencies meeting sustainability criteria related to road weather management
  • DOTs reported progress towards developing and implementing sustainability criteria related to road weather management as identified by INVEST.
  • An overwhelming majority (95 percent) of State DOTs are pursuing some sort of sustainability effort related to road weather management.
  • The most common sustainability activity among State DOTs is having a documented standard of practice or SOP for snow and ice control. The least common is having a dedicated road weather management program.
Measure adequately captured from State DOTs
  • This measure is being quantified differently than prior update.
  • Data was collected from the State DOT Survey.
  • DOTs reported progress towards developing and implementing sustainability criteria related to road weather management as identified by INVEST.
PM #27: Number of agencies conducting vulnerability/risk assessments, developing/implementing resiliency plans or adaptation plans, for their road weather management infrastructure and processes to respond to climate change and extreme weather
  • The results support this is an emerging area of practice.
  • 36 percent expressed being uncertain about their State's activities related to climate change and extreme weather.
  • 31 percent reported having developed/implemented process for responding to extreme weather.
Measure adequately captured from State DOTs
  • This is a new measure.
  • Data was collected from the State DOT Survey.
  • The results support this is an emerging area of practice.
AASHTO – American Association of State Highway and Transportation Officials
CATT – Center for Advanced Transportation Technology
CITE – Consortium for ITS Training and Education
DMA – dynamic mobility application
DMS – dynamic message sign
DOT – department of transportation
FARS – Fatality Analysis Reporting System
FHWA – Federal Highway Administration
GPS – global position system
HAR – Highway Advisory Radio
IMO – integrated mobile observations
INVEST – Infrastructure Voluntary Evaluation Sustainability Tool
ITS – intelligent transportation systems
MADIS – Meteorological Assimilation Data Ingest System
MDSS – maintenance decision support systems
NHTSA – National Highway Traffic Safety Administration
NOAA – National Oceanic and Atmospheric Administration NWS – National Weather Service
PCB – professional capacity building
PM – performance measure
R&D – research and development
RITA – Research and Innovative Technology Administration
ROI – return on investment
RWIS – road weather information systems
RWM – road weather management
RWMP – road weather management program
SOP – standard operating procedures
USGS – United States Geological Survey
V2I – vehicle to infrastructure
VDT – vehicle data translator
WRTM – weather responsive traffic management
WSI – winter severity index
Office of Operations