CHAPTER 12. CONCLUSION

Periodic assessments of the performance of transportation program activities and accomplishments have been a priority of the United States Department of Transportation (USDOT) as an essential tool for documenting goal attainment and providing guidance as programs evolve. The Road Weather Management Program (RWMP) established a set of performance measures beginning in 2006 and began collecting data in order to assess progress toward meeting each of their major program goals under the Safe, Accountable, Flexible, Efficient, Transportation Equity Act - A Legacy for Users (SAFETEA-LU). Performance measures have been quantified on a biennial basis in 2010 and 2012. This report documents a comprehensive review of the existing measures and identifies new measures intended to fill gaps created by recent adjustments to the program in light of new legislation, emerging initiatives, and refinement of program goals and activities. The result is an updated performance assessment document tracking continued progress in meeting each of the RWMP objectives focused on activities occurring in the 2012-2015 timeframe.

The measures used to assess the performance of the RWMP reflect both quantifiable outputs (e.g., number of agencies that have acquired an maintenance decision support systems (MDSS) or the number of training programs conducted) and qualitative outcomes (e.g., the extent to which agencies are using MDSS more effectively throughout their jurisdiction or the proactive incorporation of road weather information by transportation operators in decision making and the benefits experienced from these activities). Some of the RWMP objectives can be assessed quite adequately with quantitative output measures. For example, assessing success at building partnerships can be measured by identifying the number of agencies that are working together on road weather projects, jointly developing new operational strategies, and participating in joint-agency meetings and workshops. However, other objectives, such as enhancing road weather knowledge and capabilities, are more difficult to capture solely with quantitative output measures, such as attendance at training courses or RWMP website visits. It is assumed that actions taken by the RWMP to engage stakeholders and encourage their participation in various program activities will translate into the desired qualitative benefits, such as more effective use of tools or, ultimately, enhancements to traveler safety and mobility. A challenge for performance measurement is to gather the kinds of data that can support these more intangible qualitative outcomes; namely, measures that assess impacts and benefits.

The RWMP aims to promote the value of incorporating road weather data, tools, and research into State and local DOT operations in support of traveler safety and mobility. While these objectives can be met by both RWMP direct activities as well as by agency actions and factors external to the RWMP, the RWMP needs to understand the independent effects of its activities in achieving these objectives in order to implement continuous improvement in their programs and strategies. The previous performance assessments and this recent update assessment have sought to specify measures that can isolate the direct and indirect effects of the RWMP on goal attainment, though controlling for external effects remains challenging.

The RWMP desires to obtain performance measures that offer comparable indicators of progress across States. But there are many challenges to accomplishing this objective. For example, States use different indicators to measure how well they are managing and operating their transportation systems. Some States do not collect data to support performance measurement or use only a few indicators of performance. Assessing road weather management and operational performance directly is relatively new to State departments of transportation (DOT), and many do not yet include road weather into their metrics. The States focusing on measuring performance in managing their transportation system under weather conditions lack effective tools to allow them to compare performance across weather events or over time. That is, they have difficulty being able to ascribe changes in performance to the independent effects of their operational actions when there is a lot of variability event-to-event and time-to-time in the nature and severity of the weather conditions. The RWMP faces a similar challenge at the level of national performance assessment, comparing changes and benefits over time and variable weather conditions. The RWMP also desires to encourage consistency in performance metrics and methods across States and with their national approach to performance assessment.

This most recent assessment of progress across the country in meeting the RWMP objectives shows continuing adoption of advanced technologies, decision support tools, and more effective use of advanced road weather management strategies. The update received responses from 40 State DOTs which is a significant increase from the previous update of 28 State responses highlighting the primacy of road weather among State DOT operational concerns.

The update includes a number of challenges also encountered in the previous update of the measures, a number of which could not be overcome with the available data. These included:

• Assessing the impacts and benefits of partnerships, collaboration and training, such as increased awareness, knowledge, use and skills with regard to RWMP content (tools, research, etc.).
• The availability of mobile road weather data is increasing, but current availability and use are limited. As mobile data becomes more prominent, it will be important to employ measures of both the increased use of these data and assessment of their unique benefits over fixed data.

Appendix A highlights some of the significant performance advances across all the measures.