2.0 SELECTING USEFUL PERFORMANCE MEASURES
|
Points to Remember
|
Performance measures can be valuable tools for both public agencies and private-sector companies. However, it is important to select and use measures that can actually help improve work zone processes and procedures, can be understood by others who might be shown the measures, and can be determined within the data availability constraints that exist. Collecting data and computing measures, by itself, does not automatically lead to better results; the measures must align with what the agency or company deems important. At the same time, performance measurement choices cannot be made without regard to the type and amount of data that can be obtained and used to compute the measures. Thus, the decision as to what performance measures will be monitored should not be made lightly.
2.1 Characteristics of Useful and Effective Work Zone Performance Measures
|
Effective Performance Measure Characteristics
|
Ideally, work zone performance measures should have the following five characteristics:
Measures should show progress being made toward achieving desired results or goals – Measures that easily convey whether or not the agency or company is meeting or progressing towards its goals or targets (such as an agency’s work zone safety or mobility policy statements) will keep improvement efforts focused on those targets.
Measures should aid in making improvement decisions – In addition to tracking progress towards ultimate goals or targets, measures that relate to or imply possible courses of action for improvement are desirable. Measures without a purpose are undesirable.
Measures should provide clear feedback on the results of improvements or courses of actions that have been implemented – Similarly, measures that are sensitive enough to track whether improvement decisions have or have not achieved their intent help agencies and companies adjust and revise those decisions more quickly and efficiently.
Measures should be clear and easy to understand – One of the important uses of performance measures is to easily convey a “snapshot” of how an agency or company is doing and (hopefully) improving. Measures that are highly complex or which require significant technical background to understand, have more limited use than those which are inherently easy to understand. Also, most agencies will desire both project-level and program-level (aggregated across multiple projects) measures.
Measures need to be both accurate and precise – Performance measure accuracy defines how closely it can capture the “true” condition or improvement being measured; performance measure precision pertains to the level of confidence one can place on the performance measure value itself.
2.2 Users of Work Zone Performance Measures
Depending on how they are disseminated, work zone performance measures may be of interest to- and used by the following audiences:
- Practitioners responsible for designing the MOT plans for work zone projects, as well as other types of programmatic uses (such as establishing and revising work zone policies and procedures);
- Practitioners responsible for the day-to-day traffic safety and operations through the work zones; and
- The traveling public.
Work zone design practitioners are constantly tasked with striking a balance between providing space for traffic to use and providing space for work crews to complete the required project tasks. Often, certain less-than-optimum design features must be allowed on a temporary basis (e.g., narrowed lanes, temporary loss of shoulders, lane shifts) in order to accommodate the work tasks. The actual effects of these decisions are of interest to designers for consideration in future project designs.
Once a work zone has been established, practitioners responsible for its day-to-day operation desire constant feedback on both the safety and mobility effects in order to detect and correct any problems that may exist. Many work zones are very dynamic, involving multiple phases with different traffic-handling schemes, and a range of work activities that can affect traffic. These dynamics make it particularly challenging to detect when problems exist.
Finally, work zone travelers themselves desire information about work zone conditions and performance, particularly mobility performance. The value of real-time traveler information to motorists has long been recognized. Measures that reflect expected changes to normal travel conditions and which can be used as a surrogate of future conditions can aid in both departure time and route choice decisions. Measures reflecting the safety record through a project (i.e., number of injury or fatal crashes occurring in a work zone) can be useful as a reminder to motorists to drive safely and defensively.
2.3 Determining Work Zone Performance Measures of Greatest Interest and Use
As suggested above, the value and usefulness of a particular work zone performance measure will depend in part on the audience(s) for which it is intended.
- In general, the traveling public will be most interested in real-time (and forecasted) project-level, or if multiple projects exist along a particular route, corridor-level measures of the effects of work zones upon mobility and, perhaps, traffic and pedestrian safety.
- Practitioners with day-to-day work zone operational responsibilities will also be interested in real-time safety and mobility performance as well as in customer satisfaction, or perhaps more accurately, lack of satisfaction.
- Finally, practitioners who design work zones will be interested in how their design decisions ultimately affect safety, mobility, customer satisfaction, and the productivity and effectiveness of the work tasks that have to be accomplished.
Differences exist, within each audience group, of the level of detail required for various measures – While it is true that the types of measures of greatest interest will vary by audience, there will be differences in the level of detail desired in the various measures for certain subsets of those audiences. For example, travelers and higher-level manager practitioners responsible for overall operations in a given corridor may only be interested in the aggregated, corridor-level effect of all work zones upon peak-period travel times. Meanwhile, practitioners responsible for a given project may only be interested in how the project is impacting travel times during the peak periods, and may also be interested in how travel times are affected when lanes are temporarily closed for short-term work activities.
The level of performance measure aggregation or disaggregation needed can affect the amount and type of data needed to generate those measures – For certain measures, such as the number of work zone fatalities occurring during a year, aggregation across all work zones is important to allow an agency to assess its level of compliance with, or progress towards, its overall work zone safety goal. In other cases, the effect of a single work zone may be of greatest importance, such as the increase in peak period travel times during a specific phase of a complex project. The data needs for both types of performance measures are obviously much different.
Determining which work zone performance measures will be used also requires consideration of the data required to compute them. Ideally, an agency or company would simply identify the desired set of measures and the data necessary to develop those measures would be readily available. In reality, data availability must be considered as part of performance measure selection. As will be discussed later in this document, several data sources can be used for work zone performance measurement, if they are available to the practitioner, and more importantly, are appropriate for the conditions being monitored and measured.
Ultimately, work zone performance measures should be an integral part of an agency’s or company’s continuous improvement process to reduce work zone impacts. An example of how this could occur is presented in Figure 2-1. An agency establishes a work zone mobility or safety policy goal (perhaps no more than a 20 percent increase in crashes in a work zone relative to pre-work zone conditions). The agency should periodically assess what percent of its projects are meeting that goal. Those projects that did not meet the goal could then be examined in greater detail to help the agency understand the possible causes for the failure to meet the goal. In turn, this better understanding of causes could lead to improvements in either the design or operational procedures that the agency establishes for future projects.
Figure 2-1. Example Use of Performance Measures for Continuous Improvement of Agency Work Zone Policy
Battelle
Overall, work zone performance measure selection and use involves five main steps:
Step 1. Determine the categories of work zone performance that are of interest – An agency or contractor must initially decide which of the basic categories of possible work zone performance data and measures are of interest. In determining these categories, practitioners should consider both the work zone mobility policies and goals or targets already established, and the possible data sources available for the agency or contractor to use.
Step 2. Decide which work zones will be measured – Most agencies need not measure every work zone. A sampling of work zones, focusing on those of most interest to the agency, can actually be more useful from a continuous improvement process because it makes data collection, analysis, and interpretation more manageable. It should be noted that the same work zones need not be used for all performance measures. Rather, an agency or company may monitor some work zones because of concerns for their mobility impacts, monitor the safety impacts of several other work zones where certain crash countermeasures were implemented for testing, and monitor still other work zones located in a sensitive area in a downtown area for business and customer satisfaction, for example.
Step 3. Decide what work zone conditions are of most value to measure – Work zones are often complex, involving multiple phases and a wide range of work activities at different locations that vary in terms of their effects on traffic. Identifying which phases, locations, and work activities are of interest is needed to determine what performance measures are most appropriate. Measures may be computed for the entire project, for certain phases, or for during work tasks. In addition, these decisions can influence what data is needed to compute those measures. In some cases, data availability (or lack thereof) may require a change in either which work zones will be measured, or possibly a change in the categories of performance data of interest.
Step 4. Determine what data sources are available for computing performance measures at each work zone of interest, and assess their potential usefulness and limitations – Depending on the location of the work zone and the types of measures of interest, data that can be used to generate the measures may already exist within the agency. If not, one or more of several possible data collection methods may be employed. Opportunities may exist to leverage data collection efforts to meet multiple performance measure needs. Trade-offs between data accuracy/precision and costs to obtain must also be assessed.
Step 5. Identify and compute specific performance measures of interest – Once data that can be used for performance measurement have been located or collected, the final step is to identify the specific measures of interest, and then compute the measures. For most performance categories, measures can be defined to characterize the entire project (i.e., total number of hours when a traffic queue was present), specific times of interest (increase in average delay per vehicle during the peak hours), or for certain phases or work activities (average maximum queue length during hours when temporary lane closures were in place). Consideration must be given to the level of accuracy and precision attainable with the data when interpreting the results from a specific work zone as well as for an aggregated set of work zones.
< Previous | Next >
