Traffic Analysis Toolbox Volume VI:
Definition, Interpretation, and Calculation of
Traffic Analysis Tools Measures of Effectiveness

2.0 Current Practice

This chapter reviews current practice in the use of MOEs in traffic operations analysis and recommends a subset of MOEs and traffic analysis tools for further evaluation.

2.1 MOEs in Practice

This section identifies the measures of effectiveness most commonly used by transportation professionals in evaluating transportation design alternatives, operational performance, and ITS/traffic operations strategies. Three primary sources were consulted:

1. NCHRP Synthesis 311, Performance Measures of Operational Effectiveness for Highway Segments and Systems;
2. Interim report for NCHRP 7-15, Cost-Effective Measures and Planning Procedures for Travel Time, Delay, and Reliability; and
3. Interim report for NCHRP 3-68, Guide to Effective Freeway Performance Measurement.

NCHRP Synthesis 311 [Shaw, 2003] examined the use of performance measures for the monitoring and operational management of highway segments and systems. The study surveyed the current state of practice and assessed the relative strengths and weaknesses of the more than 70 measures identified.

The most successful measures related to conditions directly experienced by the traveler, e.g., travel time, speed, and delay. Measures derived from these, such as travel indices, were less relevant to the operational environment than to policy planners.

For purposes of this project, the most relevant finding is on performance measures that are used by public agencies. Table 6 shows the results of a survey of state DOTs and MPOs on the performance measures they use. LOS, traffic volume, and VMT were the measures cited most often. Travel time, speed, and incidents also were frequently cited. Most other measures were used by only one or a few of the agencies interviewed. All of the top six most frequently cited measures, except for incidents, can be estimated by current traffic analysis tools.

Table 6. Performance Measures Reported in Survey of State DOTs and MPOs

Performance Measure	Typical Definition	Number of Responses from Agencies
Level of Service (LOS)	A (best) to F (worst) based on measures of effectiveness	11
Traffic Volume	Annual average daily traffic, peak-hour traffic, or peak-period	11
Vehicle Miles Traveled	Volume times length	10
Travel Time	Distance divided by speed	8
Speed	Distance divided by travel time	7
Incidents	Traffic interruption caused by a crash or other unscheduled event	6
Duration of Congestion	Period of congestion	5
Percent of System Congested	Percent of miles congested (usually defined based on LOS E or F)	5
Vehicle Occupancy	Persons per vehicle	5
Percent of Travel Congested	Percent of vehicle miles or person miles traveled	4
Delay Caused by Incidents	Increase in travel time caused by an incident	3
Density	Vehicles per lane per period	3
Rail Crossing Incidents	Traffic crashes that occur at highway-rail grade crossings	3
Recurring Delay	Travel time increases from congestion; this measure does not	3
Travel Costs	Value of driver's time during a trip and any expenses incurred	3
Weather-Related Traffic Incidents	Traffic interruption caused by inclement weather	3
Response Times to Incidents	Period required for an incident to be identified, verified, and for an appropriate action to alleviate the interruption to traffic to arrive at the scene	2
Commercial Vehicle Safety Violations	Number of violations issued by law enforcement based on vehicle weight, size, or safety	1
Evacuation Clearance Time	Reaction and travel time for evacuees to leave an area at risk	1
Response Time to Weather-Related Incidents	Period required for an incident to be identified, verified, and for an appropriate action to alleviate the interruption to traffic to arrive at the scene	1
Security for Highway and Transit	Number of violations issued by law enforcement for acts of violence against travelers	1
Toll Revenue	Dollars generated from tolls	1
Travel-Time Reliability	Several definitions are used that include: 1) variability of travel times; 2) percent of travelers who arrive at their destination within an acceptable time; and 3) range of travel times	1

Source: Shaw, 2003.

NCHRP 7-15 [Cambridge Systematics 2005] is intended to develop structured, cost-effective methods and procedures that calculate measures of travel time, delay, and reliability. These methods are to be used by practitioners to measure, predict, and report on transportation system performance from a customer-oriented perspective and make decisions about policies, programs, or projects.

The interim report includes the findings of a review of current research and literature, a review of current agency and industry practice, and the results of several practitioner workshops.

The key findings on performance measures include the following:

• Several state DOTs use congestion and reliability measures.
• MPOs and smaller agencies still measure capacity and level of service.
• Most agencies surveyed acknowledge the growing importance of congestion and reliability performance measures.
• Some practitioners believe that the general public and high-level transportation officials are less interested in reliability measures than in total delay and congestion measures. The consensus opinion was that reliability has not yet gained the attention of the vast majority of the general public and decision-makers at the current time.
• Many organizations use economic measures such as benefit/cost ratios in their planning processes.

NCHRP 3-68 will develop a guidebook that codifies best practices in freeway performance measurement. The Interim Report [Cambridge Systematics, 2004] covers the results of interviews with more progressive agencies on their use of performance measures. The interviews covered the motivation for performance measures, the performance measures used, and use of the reported performance measures.

With regard to the types of performance measures used by agencies, the main findings were as follows [quoted directly from the NCHRP 3-68 Interim Report by Cambridge Systematics]:

• Both outcome and output measures are used by agencies. It is clear that agencies who have undertaken freeway performance measurement have accessed the literature on performance measurement because they use the outcome/output terminology.
• For outcome measures, derivatives of speed and delay are commonly used by both operating and planning agencies. The Travel Time Index is a popular metric. Level of service as a metric is still in use in both planning and operations agencies, though it is not as widespread as it might have been 10 years ago. Reliability metrics have not yet found their way into widespread use. (Seattle and Minneapolis are exceptions.) These metrics are usually formulated for short segments or at key locations. An exception is Seattle where a series of defined "freeway trips" have been defined – these can involve travel over multiple freeway routes for extended lengths.
• Some of the more interesting metrics used by agencies include:
  • The number of very slow trips (half of free flow speed) that occurs each year by time of day and major trip (Seattle);
  • Percentage of reduction in incident congestion delay; and
  • Percent of freeway lane-miles below congested volumes (based on volume per lane).
• Output measures are used primarily by operating agencies, and then primarily for incident management activities and the operation of field equipment (e.g., sensors, cameras).

Many areas are beginning to define more sophisticated measures for measuring congestion/mobility performance but have not yet implemented them. Overall, there appears to be a trend away from the general categories of performance (LOS) and toward continuous measures that are based on delay and travel time. Further, consideration of travel-time reliability is growing in acceptance, though its implementation is still problematic, primarily due to data requirements.

Florida Mobility Performance Measures Program

The Florida Department of Transportation (FDOT) has identified four dimensions for measuring the effectiveness of programs in meeting the objectives of FDOT. They are:

1. Quantity of Travel – The number of vehicles or people using a facility or service;
2. Quality of Travel – The degree of congestion;
3. Accessibility – The ease with which people can gain access to the transportation system; and
4. Utilization – The percentage of capacity utilized.

Table 7 shows the measures of effectiveness for highway performance. Table 8 shows the MOEs selected for transit systems. The performance measures are reported for two time periods: Peak Hour and Daily.

Table 7. FDOT Mobility Performance Measures for Highways

Dimension	Mobility Performance Measure	Definition
Quantity of Travel	Person miles traveled	Average Annual Daily Traffic * Length * Vehicle Occupancy
Quantity of Travel	Truck miles traveled	Average Annual Daily Traffic * Length * Percent Trucks
Quantity of Travel	Vehicle miles traveled	Average Annual Daily Traffic * Length
Quantity of Travel	Person trips	Total person trips
Quality of Travel	Average speed	Average speed weighted by person miles traveled
Quality of Travel	Delay	Average delay
Quality of Travel	Average travel time	Distance/mean speed
Quality of Travel	Average trip time	Door-to-door trip travel time
Quality of Travel	Reliability	Percent of travel times that are acceptable
Quality of Travel	Maneuverability	Vehicles per hour per lane
Accessibility	Connectivity to intermodal facilities	Percent within 5 miles (1 mile for metro area)
Accessibility	Dwelling unit proximity	Percent within 5 miles (1 mile for metro area)
Accessibility	Employment proximity	Percent within 5 miles (1 mile for metro area)
Accessibility	Industrial/warehouse proximity	Percent within 5 miles
Accessibility	Percent miles bicycle accommodations	Percent miles with bike lane/shoulder
Accessibility	Percent miles pedestrian accommodations	Percent miles with sidewalk
Utilization	Percent system congested	Percent miles at LOS E or F
Utilization	Percent travel congested	Percent daily VMT at LOS E or F
Utilization	Vehicles per lane-mile	Average Annual Daily Traffic * Length/lane miles
Utilization	Duration of congestion	Percent miles at LOS E or F

Source: FDOT Mobility Performance Measures Brochure (2000) [FDOT 2000]. http://www.dot.state.fl.us/planning/statistics/mobilitymeasures/mmbrochure.pdf.

Table 8. FDOT Mobility Performance Measures for Transit Systems

Dimension	Mobility Performance Measure	Definition
Quantity of Travel	Ridership	Total passenger trips
Quality of Travel	Auto/Transit Travel-Time Ratio	Door-to-door trip time
Quality of Travel	Reliability	On-time performance
Accessibility	Coverage	Percent person minutes served
Accessibility	Frequency	Buses per hour
Accessibility	Span	Hours of service per day
Utilization	Load Factor	Percent seats occupied

Source: FDOT Mobility Performance Measures Brochure (2000) [FDOT 2000]. http://www.dot.state.fl.us/planning/statistics/mobilitymeasures/mmbrochure.pdf.

Data on existing performance is obtained from FDOT's Roadway Characteristics Inventory (RCI) and Traffic Characteristics Inventory (TCI). Average vehicle occupancy data is obtained from the 1990 National Personal Transportation Survey (NPTS). Transit system data is collected from local transit authorities. A land use database (GIS) was used to measure accessibility. Special studies were performed to obtain reliability data. Travel demand modeling and the Highway Capacity Manual methods are used to forecast the MOEs.

2.2 Interpretation of MOEs by Agencies

Interpretation of MOEs consists of distinguishing between acceptable and unacceptable traffic operations.

Local and state public agencies primarily use the HCM LOS grades to determine if operations are acceptable or unacceptable. A certain letter grade is set as the agency standard. Then project designs and project impacts are compared against that letter grade standard. A project design is considered unacceptable if the letter grade level of service is below the agency standard. An impact is determined to be significant if the project changes the letter grade from an acceptable letter to an unacceptable letter.

The meaning of density on a freeway or highway is interpreted based upon its equivalent HCM LOS grade. Similarly the meaning of speed on an arterial street and delay at an intersection is determined based upon their equivalent HCM LOS grades.

Volume/capacity ratios are used by some agencies to determine acceptability of operations. A v/c less than one is acceptable. A v/c greater than one is unacceptable.

There is however no guidance or standard practice on the interpretation of the other MOEs that do not have HCM LOS equivalents, such as travel time, reliability, vehicle hours traveled, vehicle miles traveled, mean system speed, etc.

Agencies typically use non-HCM performance measures for comparing alternatives. In an alternatives analysis, less of an undesirable feature (such as vehicle hours traveled or variability of travel time) is considered better, but there is no threshold of acceptability. Interpretation of these non-HCM measures is currently difficult, in the absence of information on national averages or national practice that would help an agency characterize a particular value of a non-HCM MOE as acceptable.

2.3 Selection of MOEs for Further Evaluation

Volume 1 of the Traffic Analysis Toolbox [FHWA 2004a] lists numerous performance measures that are generated by most analysis tools. The list is a mix of input performance measures, basic performance measures, and derived performance measures. Some of the measures in this list, such as v/c ratio, are necessary for computing the basic travel time, speed, and delay performance measures. Travel time and speed are interchangeable. IF one is known the other can be computed. Other performance measures, such as delay or fuel consumption can be derived from the basic travel time/speed performance measure. LOS, in particular, is a derived performance measure. LOS is not strictly a performance measure, but a method of reporting one or more selected numerical performance measures in a system of easily understandable letter grades.

Basic Performance Measures

Basic performance measures are those measures, which cannot be derived from other performance measures. The major task of the analytical tool is to compute these basic performance measures. Once these measures have been computed, it is possible (at least in theory) to load the output into a spreadsheet and compute the other measures.

Capacity, although often considered to be an input, is a basic facility characteristic crucial to the calculation of many of the other performance measures. For simulation models, capacity is a calibratable output. For analytic models it is often a direct input or a computed input based on the geometric characteristics of the facility.

Speed, and its inverse, travel time, are the basic performance measures from which most all other performance measures can be computed.

Number of stops is another basic performance measure that cannot be derived directly from the other performance measures. It is crucial for predicting fuel consumption, noise, and air pollutant emission performance measures.

Travel-time reliability is another basic performance measure that cannot be directly derived from the other performance measures. The profession currently is exploring how "reliability" might be used in decision-making, so it has not seen much use so far. Many popular analytical tools do not produce measures of reliability.

Derived Performance Measures

Derived performance measures are those that could, at least in theory, be computed from the basic performance measures and other inputs. It is often not practical though for the user to go through model output to extract the data needed to make the computations. Thus it is a practical advantage to the user if the analytical tool computes these derived measures automatically.

Volume or throughput can be computed from demand and capacity, although upstream capacity constraints need to be considered in determining downstream volumes. Volume then becomes an input to several performance measure computations.

Delay is the difference between ideal travel time and the actual travel time.

Density can be computed from volume and speed from the "flow equals density times speed" equation.

The number of vehicles in queue can be derived from the volume arriving on red (for signals) and/or the difference between the volume and the capacity. The queue length is the number of vehicles in queue divided by the density (for freeways) or the average spacing between stopped vehicles (for signals).

Travel Distance and accumulated Travel Time ( VMT/PMT and VHT/PHT ) can be derived from the volume and the travel time. Distance traveled is the sum of link volumes and link distances or the volume divided by speed times the number of hours.

The volume capacity ratio (V/C) is computed from volume and capacity, which are inputs to the other performance measures.

The duration of congestion is the difference between the demand and the capacity, divided by the capacity.

Mode split is computed from the ridership, AVO, and auto volume performance measures.

Crashes are beyond the ability of commonly used traffic analysis tools to predict.

Noise, fuel consumption, and pollutant emissions are measures that can be calculated from the other measures of volume, speed, delay, stops.

Methods of Summarizing Performance Results

The methods for summarizing performance take quantitative physical performance results (e.g., stops, delays, etc.) and convert them into qualitative or quantitative results, either letter grades or monetary values. The innovative approaches considered in this research will look at different methods for summarizing facility and system performance.

Level of service (LOS) is computed based on delay or density depending on the facility type. A look-up table is used to convert quantitative numerical results into qualitative letter grades.

Benefit/Cost is computed from the other performance measures. The other measures are converted into monetary equivalents and summed to obtain benefits and costs.

Whatever method is used to summarize performance results, it should be capable comparing alternatives that differ along the following dimensions:

• Type. Increasingly, managers are faced with deciding between different types of alternatives: e.g., operations improvements versus capacity improvements, maintenance of existing facilities versus constructing new facilities.
• Physical scale, e.g., a large single interchange reconstruction on a freeway versus signalization improvements to all interchanges along a freeway.
• Time scale, i.e., alternatives with different lifetimes.

Recommended MOEs for Focus of Tool Evaluation

It is recommended that the following eight measures of effectiveness shown in Table 9 become the focus of analytical tool evaluations for this study. They were selected because they constitute the basic building blocks of most all performance measures currently used by public agencies to monitor agency performance. Thus, if one can predict HCM LOS, volume/capacity ratio, travel time, speed, delay, queues, stops, and travel-time variance, one can estimate most any performance measure currently used by public agencies in the transportation sector. Economic costs, air quality, noise, and energy use can be estimated from these basic eight MOEs.

Table 9. Recommended Focus MOEs

HCM Level of Service

Volume/Capacity

Travel Time

Speed

Delay

Queue

Stops

Density

Travel-Time Variance

2.4 Commonly Used Traffic Analysis Tools

This section provides an inventory of commonly used traffic simulation and analytical tools that are used to evaluate transportation design alternatives, operational performance, ITS, and traffic operation strategies.

Following the typology laid out by Volume 1 of the Traffic Analysis Toolbox, the tools fall into the following categories:

1. Sketch Planning Tools;
2. Travel Demand Models;
3. Deterministic Analytical Tools;
4. Traffic Signal Optimization Tools;
5. Macroscopic Simulation Models;
6. Mesoscopic Simulation Models; and
7. Microscopic Simulation Models.

Since the focus of this research effort is on tools for design and operation analysis, we will focus on the traffic signal optimization tools and simulation models (macro, meso, and micro). Within these four categories Volume 1 of the Toolbox identifies dozens of software.

The determination of "commonly used" was based upon the extent to which the tools are referenced in the published technical literature for papers accepted for presentation at the 2004 and 2005 Annual Meeting of the Transportation Research Board. The selection of analysis tools to be used in this research was then made from among the most popular tools with the intent of obtaining a representative sample of the range of commonly used analysis tools, while keeping the total number of tools evaluated to a tractable number (see Table 10).

Table 10. Initial List of Commonly Used Tools

Category of Analysis Tool	Representative Software
Deterministic Analytical Tools: Signalized Intersections	Synchro6
Deterministic Analytical Tools: Urban Streets	FDOT ARTPLAN
Deterministic Analytical Tools: Rural Two-Lane Roads	HCS
Deterministic Analytical Tools: Rural Multi-Lane Highways	HCS
Deterministic Analytical Tools: Freeways	HCS
Traffic Optimization Tools: Signalized Intersections	Synchro6
Traffic Optimization Tools: Freeway Ramp Metering	FREQ
Macroscopic Simulation Tools: Urban Streets	Transyt-7F
Macroscopic Simulation Tools: Rural Highways	None
Macroscopic Simulation Tools: Freeways	TRAF-CORFLO
Mesoscopic Simulation Tools	Dynasmart-P
Microscopic Simulation Tools	CORSIM
Microscopic Simulation Tools	SimTraffic
Microscopic Simulation Tools	Vissim
Microscopic Simulation Tools	Paramics
Microscopic Simulation Tools	Aimsum

2.5 Selection of Traffic Analysis Tools for Further Evaluation

It is beyond the budget and timeframe available for this research to investigate all tools and MOEs they produce, so the initial list of commonly used tools was narrowed down to the following recommended subset of analysis tools for evaluation during the remainder of the project (see Table 11).

Table 11. Recommended Tools for Evaluation

Tool Category	Representative Tool to Be Evaluated
Deterministic Analytical Tools	Synchro6 and HCS
Traffic Optimization Tools	Synchro6
Macroscopic Simulation Tools	Synchro6
Mesoscopic Simulation Tools	Dynasmart-P
Microscopic Simulation Tools	CORSIM, SimTraffic, Vissim, Paramics, Aimsum

The above list is not intended as an exhaustive list or a list of even the most commonly used tools for evaluation of traffic operations in the United States. Indeed, many popular tools are absent from this list. It would be beyond the resources available to this project to investigate all of the popular tools in the United States. This list is intended to be a representative cross-section of typical tools illustrative of the range and breadth of tools employed in the United States for traffic operations analysis at the macroscopic, mesoscopic, and microscopic levels.