Traffic Analysis Toolbox
Volume VI: Definition, Interpretation, and Calculation of
Traffic Analysis Tools Measures of Effectiveness

Executive Summary

This report presents the results of an investigation into the appropriate definition, interpretation, and computation of measures of effectiveness (MOEs) for traffic operations and capacity improvements.

Study Objectives

The goal of this study was to develop information and guidance on which MOEs should be produced, how they should be interpreted, and how they are be defined and calculated in traffic analysis tools. Specific objectives of the study were to:

Gain an understanding of the current use and interpretation, by transportation professionals, of some of the most commonly used MOEs generated by traffic simulation and analytical tools, such as HCM procedures;
Identify how field measurements are processed to estimate the MOEs used in conducting traffic analysis studies;
Provide guidance on how these MOEs are defined and calculated in the tools;
Develop an innovative approach to interpret these MOEs when conducting traffic analysis studies; and
Demonstrate the validity of the approach through a case study of representative tools.

Study Approach

The above objectives were accomplished in the following five tasks:

A review of current practice regarding the use and interpretation of commonly used MOEs (Task B);
A review of how MOEs are computed from field data (Task D);
A review of how MOEs are defined and computed within traffic simulation models and the Highway Capacity Manual (Task C);
Development of innovative approach to interpret MOEs produced by traffic studies (Tasks D and E); and
Demonstration of innovative approach on a case study (Task F).

This Report

The intended audience for this technical report is transportation planners, traffic analysts, traffic engineers, and decision-makers. The purpose of the report is to educate transportation professionals on how to correctly interpret their findings and present them in a manner that clearly supports their recommendation and is easy to comprehend by the decision-makers and the public.

Measures of Effectiveness (MOEs)

The purpose of computing one or more traffic performance measures of effectiveness is to quantify the achievement of a project's traffic operations objectives. This study identified seven basic measures of effectiveness, which are the building blocks of most existing and potential future systems for evaluating the traffic operations performance of highway facilities. They are:

Travel Time;
Speed;
Delay;
Queue;
Stops;
Density; and
Travel-Time Variance.

Travel time, speed, and delay are closely related measures of the amount of time that the general public must expend in order to complete their trips on the system.

Queues are indicators of operational problem spots within the system where capacity inefficiencies and/or safety problems may exist because of intersection blockages or turn bay overflows.

Stops, delay, and speed are inputs into signal timing optimization algorithms and into fuel consumption and air pollutant emission computations.

Density is used in the Highway Capacity Manual (HCM) to compute level of service for uninterrupted flow facilities (facilities without stop signs or traffic signals).

Travel-time variance currently is a little used performance measure that will soon become an important component of various travel-time reliability indices being proposed for the assessment of the benefits of traffic operations improvements.

In addition there are two commonly used indicators of performance that are used to communicate to decision-makers the quality of the facility performance:

Highway Capacity Manual (HCM) Level of Service; and
Volume/Capacity.

Poor letter grade levels of service (e.g., "F" ) and volume/capacity ratios greater than 1.00 are readily recognizable indicators of poor traffic operations.

Table 1 describes how these MOEs are typically used in practice. More details are provided in Chapter 2 of this report.

Table 1. Common Usage of MOEs in Practice

MOE Building Blocks of Performance Measurement Systems	Typical Usage
Travel Time	Used in long-range planning studies at regional or corridor level to evaluate traveler benefits of alternative improvements. Used to evaluate traveler benefits of signal timing improvements for individual facility.
Speed	Used to evaluate alternatives in long-range planning studies at regional or corridor level. Used to evaluate benefits of signal timing improvements for individual facility. Used to estimate fuel consumption and air quality impacts.
Delay	Used to evaluate alternatives in long-range planning studies at regional or corridor level. Used to evaluate benefits of signal timing improvements for individual intersection or facility. Used to compare different degrees of congestion (Level of Service F). Used to estimate fuel consumption and air quality impacts.
Queue	Used to identify hot spots, operations problems at points of facility (left turn bays, blockages, safety).
Stops	Used to evaluate fuel and air pollution savings of signal timing improvements for individual facility.
Density	An input to the Highway Capacity Manual level of service for uninterrupted flow facilities.
Travel-Time Variance	Rarely used due to difficulty of computation. Potential use for evaluating benefits of traffic operations improvements that reduce variability but not mean travel time or delay.

MOE Indicators of Performance	Typical Usage
HCM Level of Service	Used to determine if facility design (for a given set of signal control parameters) will provide acceptable traffic operations as reported for the peak 15 minutes.
Volume/Capacity	Used in planning studies to quickly determine if facility has sufficient number of lanes (sizing the facility) regardless of signal timing.

Field Measurements of MOEs

Table 2 summarizes the techniques commonly used to measure each MOE in the field. These field measurement methods are often mimicked in simulation models to compute MOEs. More details are provided in Chapter 3 of this report.

Table 2. Field Measurement of MOES

MOE Building Blocks of Performance Measurement Systems	Field Measurement Technique
Travel Time	Travel time can be measured in the field through a variety of instrumented vehicle or vehicle tracing techniques. Mean travel time is the simple average of the traced vehicle travel times. Mean travel time can be estimated from point measurements of speed (less accurate than tracing techniques), such as at video or loop detectors. The road segment length represented by the detector is divided by the harmonic mean speed measured at the loop detector to obtain mean travel time.
Speed	The mean speed for a road segment can be estimated by measuring travel time and dividing the segment length by the mean travel time. Instantaneous speed measurements along the length of a road segment can be obtained from instrumented vehicles. The mean speed also can be estimated from point measurements of speed (less accurate than tracing techniques), such as at video or loop detectors. The harmonic mean speed is computed for the speeds measured at the loop detector.
Delay	Delay over the length of a road segment is measured in the field using travel-time measurement techniques described above. Delay is the difference between the measured travel time and the free-flow travel time. Delay at an intersection is measured by counting the queued vehicles every 15 seconds (or so). The total delay is the number of queued vehicles times the interval between queue counts (15 seconds). Judgment is required to determine if a slow (but not stopped) vehicle is in a queue. This method misses deceleration/acceleration delay so the total delay is factored up to account for the missed delay. Stopped delay is multiplied by a factor greater than one to obtain control delay (stopped delay plus acceleration and deceleration delay). The average delay is the total delay divided by the number of vehicles arriving on the subject approach during the analysis period.
Queue	The best way to measure a queue in the field is to count the second by second arrivals at an upstream point and the second by second departures at a downstream point. The difference between the cumulative arrivals and the cumulative departures at time "t" (after shifting the departure pattern for the free-flow travel time between the upstream and downstream points) is the queue at time "t." The average number of vehicles queuing at an intersection can be measured in the field using the same technique as described above for measuring delay at an intersection. Judgment is required to determine if a slow (but not stopped) vehicle is in a queue.
Stops	Stops along a road segment can be measured using the same vehicle tracing techniques as described above for travel time. Stops can be measured at an intersection using the intersection delay method described above. Both methods require judgment as to when a slow vehicle is considered stop and when a vehicle is moving up in a queue (stops within a queue are not counted).
Density	The average density over time at a given point in space is computed by dividing the measured flow rate per hour by the measured arithmetic mean speed at the point. The average density over the length of a segment at a given point in time is measured using aerial photography and counting the number of vehicles present on the segment.
Travel-Time Variance	Travel-time variance is computed from the measured travel times (sum of the squared differences between the observation and the mean value).

MOE Indicators of Performance

Field Measurement Technique

HCM Level of Service

The measurement of HCM LOS in the field is described above for speed, delay, and density for freeways, highways, arterials, and intersections.
For 2-lane highways, the percent time spent following is estimated by measuring the percent of vehicles in platoons passing a given point for the peak 15-minute flow period within the analysis hour. A vehicle is defined to be in a platoon if it is following another vehicle by 3 seconds or less. The percent of vehicles in platoon is assumed to be roughly equal to the percent time spent following.

Volume/Capacity

If the demand exceeds capacity for at least 15 consecutive minutes then the capacity can be measured in the field. Capacity is the queue discharge flow rate observed for at least 15 consecutive minutes. Once the capacity of a facility is known, then the v/c ratio is measured by simply counting the hourly volume and dividing by the estimated hourly capacity.

Simulation of MOEs

There is, of course, a significant difference in how macroscopic, analytic tools compute MOEs and how microscopic simulation tools compute the same nominal MOEs. However, even among microscopic simulation tools, there are a wide variety of methods used to compute MOEs.

For example, some simulation tools compute the vehicle miles traveled only for vehicles that enter the link during the analysis period. Others include the vehicles present on the link at the start of the period, presuming that they traveled the full length of the link. Others include step-wise integrate the distance traveled on a second by second basis. Others include only the vehicles able to exit the link during the analysis period.

Another example is the computation of vehicle hours traveled (VHT). Some simulation tools include the delay incurred by vehicles denied entry to the system. Most others do not. The analyst must remember to add the delay in, when appropriate.

Most tools measure delay against the coded free-flow speed of the link. Some take into account the effect of maximum safe turning speed and reduce the computed delay accordingly. One tool measures delay against each driver's desired travel speed, not the link free-flow speed.

There is a great deal of variation among the tools in the reported queue lengths for a signal, even under low-flow, pretimed signal conditions when only uniform delay occurs. The variations are related to the geographic area included in the search for queuing vehicles and the definition of what constitutes a queuing vehicle.

Simulation model MOEs are usually not directly translatable into Highway Capacity Manual (HCM) level of service measures, because the HCM bases hourly level of service on the performance of the facility during the peak 15-consecutive-minute period within the analysis hour. Also, most all simulation tools report the density of vehicles, while the density used in HCM level of service (LOS) for uninterrupted flow facilities (no signals or stop signs) is the passenger-car equivalent in passenger car units (pcu) of the actual density of vehicles on the facility.

Chapter 4 provides more detail on the variety of methods used to compute the MOEs by the Highway Capacity Manual and various traffic analysis tools.

Limitations in How MOEs Are Currently Computed

Our ability to measure the MOEs in the field and the ability of simulation tools to estimate the MOEs are both severely strained by extreme congestion (see Table 3).

Table 3. Common Limitations of MOEs in Practice

MOE Building Blocks of Performance Measurement Systems	Limitations
Travel Time	Treatment of delays and travel time for incomplete trips or trips that cannot start within the analysis period is an issue. Difficult for a decision-maker to interpret (no value of time is obviously too high or too low without knowing more about the trip characteristics).
Speed	Treatment of incomplete or unbegun trips is an issue, as for travel time. Interpretation is facility-specific. Low speed may be quite acceptable on an arterial, but not on a freeway. Once all vehicles come to a stop, the mean speed conveys no further information on the severity of congestion (speed cannot go below zero). Speed is a poor indicator of how close the facility is to breakdown (capacity).
Delay	Treatment of incomplete trips or trips that have not started is an issue. Definition of free-flow speed, against which delay is measured, is a problem. Some use the posted speed limit for the free-flow speed; others use the mean speed under very low-flow conditions (can be higher or lower than the posted speed limit).
Queue	Definition of when vehicle joins a queue and when it leaves a queue is a problem. Interpretation is road segment and facility-specific. Queues are normal for signals, but queues that overflow turn bays or block cross streets are not desirable. Tallying of queued vehicles unable to enter the road segment is an issue for many tools. Many tools cannot report a queue longer than the storage capacity of the road segment or turn bay. Tallying of vehicles, which have not been able to enter the road network during the analysis period, is an issue.
Stops	Definition of what minimum speed is a stop is an issue. (One reviewer recommended that stop speed be defined as zero so that model results can be better compared to two-fluid models of traffic flow.) Since tallying of stops is normally suspended while a vehicle is moving up within a queue, the definition of a queue is an issue. Once all vehicles are queued on a road segment, further increases in congestion have no effect on stops.
Density	Once all vehicles are queued, then further increases in congestion have no effect on density.
Travel-Time Variance	Rarely used due to difficulty of computation. Prediction tools are close to nonexistent. Once all vehicles are queued on a road segment, the variation in travel time tends to drop towards zero. Variance will tend to peak when the volume is around capacity.

MOE Indicators of Performance

Limitations

HCM Level of Service

HCM LOS is designed to convey how close the facility is to reaching capacity or unacceptable LOS. Once LOS "F" reached, HCM LOS letter grades give no further information on relative severity of different degrees of congestion.
HCM LOS does not give information on the duration of the condition.
HCM LOS ratings do not exist for several situations: roundabouts, collector/distributor roads, local residential streets.

Volume/Capacity

Cannot measure in field or simulate volumes passing a given point that are greater than capacity (measured v/c is always less than 1.00). In such situations, must add to the measured flow rate the buildup in the queue.

Under severe congestion conditions it is very difficult to study a large enough geographic area over a long enough analysis period to ensure that congestion does not extend back beyond the limits of the study area and congestion is not present at the start or the end of the analysis period. Thus most simulation modeling of severe congestion conditions usually has queues that extend beyond the temporal and physical limits of the model.

Failure to include in the MOEs the queues that stretch beyond the geographic and the temporal bounds of the study area will bias the computation of travel time, speed, and delay, making capacity improvements within the study area appear to perform worse than no improvements.

There also are problems with reporting queues that extend beyond the turn bay and/or beyond the link. Only one simulation tool will track the buildup of queues beyond the subject link, and none of them will track congestion beyond the temporal or geographic limits of the model. With one exception, the reported queue is, by definition, never longer than the storage capacity of the turn bay or the link. Thus, the analyst cannot rely upon the reported queue length to identify queue overflow problems. The analyst must find the upstream links and review the reported queues there.

Chapter 5 of this report provides more details.

Inherent Limitations of Current MOEs

Most MOEs are satisfactory measures of traffic performance for uncongested conditions. Travel time, delay, queues, stops, density, and travel-time variance all increase as traffic demand increases relative to capacity and traffic operations worsen.

Among these MOEs, speed is a less satisfactory indicator of how close a facility is to breakdown, because speed is comparatively insensitive to changing traffic flow rates until capacity is reached.

Most all of the MOEs tend to break down under extreme congestion conditions. Speed, density, stops, and travel-time variance are invariant under "parking lot" conditions where no movement is possible (speed equals zero, density equals jam density, and travel-time variance is zero). The only MOEs that continue to function under parking lot conditions are travel time and delay, which continue to increase over the length of the analysis period (see Table 3 above).

Chapter 5 of this report provides more details.

Recommended Set of Key System MOEs

Traffic operations analyses can generate a great deal of numerical output. Any one of the numerical outputs can be important to the analyst depending upon the purpose and scope of the analysis and the alternatives being evaluated. The set of key systemwide MOEs recommended below is designed to be the "starting set" but not the end all set of MOEs for any traffic operations analysis.

This basic set is kind of like the first set of exams given to every patient when they first check into the emergency room. It does not matter what the patient's complaint, the health of every emergency room patient is measured by four indicators: temperature, pulse, blood pressure, and blood oxygen. These four basic indicators give the doctors a basic understanding of the general health of the patient, and some indication of general fields to investigate for the source of the problem. But they are not the definitive diagnostic tests. They are not the MRIs, X-Rays, or exploratory surgeries that come later.

Similarly, our basic system MOEs are designed to be that first round of tests every traffic engineer and planner should perform to assess the general health of his or her transportation system. They give an indication of the overall health of the system, and how serious the problems are. But they do not necessarily tell the engineer/planner precisely what is wrong. That is the purpose of additional analyses and additional tests.

This basic set of MOEs is not designed to replace the more detailed intersection and segment MOEs engineers and planners already are accustomed to perform to diagnose and solve traffic operations problems on the transportation system. This basic set is designed to help the engineer/planner and decision-maker rapidly assess the state of the system and identify key avenues of additional analysis to better identify needed improvements.

This basic set of MOEs also is good for rapidly assessing the benefits of alternative improvements at the system level, in a form readily understandable by the decision-maker.

The basic set of MOEs for decision-making consists of five basic measures:

Throughput;
Mean Delay;
Travel Time Index;
Freeway Segments at Breakdown; and
Surface Street Intersections with long queues, Turn Bay Overflows and Exit Blockages.

Each of these basic MOEs is highlighted in Table 4.

Table 4. Key Measures of Effectiveness for Decision-Making

Decision MOE	Use/Description/Interpretation
Throughput (Vehicles/Hour) and Percent Incomplete Trips	Use – Provides a measure of the relative productivity of the system compared to an alternative. This MOE also is required to compute mean delay. Definition – The number of vehicles present at the start plus those attempting to enter and successfully entering the system during the analysis period. Computation – At start of analysis period tally number of vehicles present within system. During analysis period tally number of vehicles attempting to enter the system but never successfully enter. Tally number of vehicles that both enter and exit the system during the analysis period. Tally number of vehicles entering the system during the analysis period that are remaining on system at end of analysis period. Reporting – Number of vehicles at start that successfully exit system during analysis period (V1). Number of vehicles at start that fail to exit the system during the analysis period (V2). Number of vehicles that entered but did not exit during the analysis period (V3). Number of vehicle generated but never entered the system during the analysis period (V4). Number of vehicles that successfully both entered and exited the system during the analysis period (V5). Report the percent of incomplete trips as the ratio of (V1+V2+V3+V4)/(V1+V2+V3+V4+V5). Interpretation – The throughput is compared for different alternatives to determine the relative productivity of each alternative. Higher values are desired. If the ratio of incomplete trips exceeds 5 percent then the decision-maker should consider lengthening the analysis period or investigate the MOEs computed for solely the through trips (V5) to see if they might better represent system performance.
Delay/Trip	Use – Comparison of traveler perceptions of average traffic performance of alternatives. Definition – The travel time (for all vehicles entering and attempting to enter the system during the analysis period) minus the theoretical travel time at the free-flow speed. This difference is divided by the number of vehicle trips to obtain mean delay per trip. The free-flow speed is defined as the minimum of the maximum safe speed or the analyst coded speed limit. Note that turning vehicles at an intersection would have a lower maximum safe speed than through vehicles. Computation – Accumulate distance traveled (VMT) and time expended (VHT) each second by vehicles on system (or attempting to enter system) during analysis period. Simultaneously accumulate theoretical VHT that would have been expended if vehicle had been able to travel the same VMT at free-flow speed. At the end of the analysis period the free-flow VHT is subtracted from the accumulated VHT to obtain Delay VHY. The Delay VHT is converted to seconds and then divided by the total number of vehicle trips over which the delay was accumulated (V1+V2+V3+V4+V5). If the number of incomplete trips (V1+V2+V3+V4) is greater than 5 percent of all trips, then either the analysis period should be lengthened, or Delay VHT should be accumulated separately for through trips (V5) and averaged over only the through trips (V5). Reporting – Delay is reported in vehicle hours traveled (VHT) and mean seconds per trip. If incomplete trips account for over 5 percent of all trips, that should be noted. Interpretation – Subjective. If trips are long in system, then high mean delays may be acceptable. If trips are short, even low mean delay may be unacceptable.
Travel Time Index	Use – To determine if facility operation during peak periods is unacceptably worse than for off-peak periods. Definition – Ratio of observed vehicle hours traveled (VHT) to theoretical VHT at free-flow speed for same VMT. Computation – See notes above on computation of mean delay. The travel time index is the accumulated VHT divided by the theoretical free-flow VHT. Reporting – System travel time index plus a few words qualifying the result, such as "Good," "Bad." Interpretation – The TTI minus 1.00 gives the mean delay as a percentage of the mean trip time. The inverse of the TTI gives the mean speed as a percentage of the free-flow speed for the system. For an entire metropolitan area (including local streets, arterials, and freeways) a peak-period TTI of 1.4 or better was typical of 90 percent of the metropolitan areas of United States in 2005. For more focused systems of mixed freeway and arterial facilities (no local streets) a TTI of under 2.5 is roughly indicative of generally uncongested conditions and good signal coordination. For a system of solely unsignalized facilities (freeways, highways, 2-lane rural roads) a TTI of over 1.4 is indicative of a facility that is overcapacity for the entire length of analysis period. For a system of exclusively signalized arterials a TTI of over 2.5, is generally indicative of overcapacity conditions and/or poorly coordinated traffic signals. A qualifier of "Good" for TTIs <= 1.5, "Potentially Acceptable" for TTIs between 1.5 and 2.5, and "Less Desirable" for TTIs > 2.5 would help the decision-makers.
Freeways: Percent Breakdowns	Use – To identify geographic and temporal extent of unsatisfactory operations for freeways in system. Description – Breakdown is defined as a density of vehicles per lane greater than the density at capacity flow rate. The HCM assigns LOS "F" to these densities higher than capacity. Two percents are computed: Maximum percent of directional freeway miles at LOS F at any onetime during the analysis period. Percent of analysis period when any directional freeway segment is at LOS F. Computation – Determine maximum LOS "E" density for each directional freeway segment in system according to the Highway Capacity Manual. Determine PCE values for each vehicle type in system for each directional freeway segment according to the Highway Capacity Manual. For each second in the analysis period tally number of vehicles on directional segment and multiply by appropriate PCE value to obtain equivalent PCUs. At end of model run compute 15-minute running averages of PCUs for each directional segment. Identify start and end times when 15-minute average PCU exceeds HCM LOS E maximum density. Find earliest start time of LOS F and latest end time of LOS F for system. The percent duration of LOS "F" is equal to the latest end time minus the earliest start time of LOS F, divided by the length of the analysis period. The maximum percent LOS "F" is the maximum number of directional segment miles operating at LOS "F" (averaged over 15 minutes) at any one time divided by the total directional miles of freeway in the system. Interpretation – The geographic and temporal percents LOS "F" are indicators of congested freeway operations. The lower, the better.
Surface Street: Percent of Intersections Exits Blocked	Use – To identify geographic and temporal extent of unsatisfactory operations for urban streets in system. Intersection exit blockages due to downstream queues extending back to an upstream intersection reduce through capacity and are potential safety concerns. Definition – Two percents are computed: Maximum percent of intersection exits that are blocked at any onetime during the analysis period. Percent of analysis period when any intersection exit is blocked. Computation – For every directional street link and for each second in the analysis period compare the distance of back bumper of last car in any lane of the directional street link to the total estimated distance to the nearest upstream intersection (If the upstream node has only one link feeding the subject downstream link, that node is not an intersection. Abandon the test on the subject link and perform the test instead on the upstream link.). Determine if the back of queue is within 25 feet of storage length of the directional link. Accumulate number of seconds during analysis period when one or more directional links are full. The percent of the analysis period with one or more full directional links is the number of seconds when one or more directional links are full divided by the number of seconds in the analysis period. Tally number of directional links meeting this "full" criterion at any one time. The maximum percent full links is the maximum number of full directional links at any one time divided by the total number of directional street links (exclude freeway links which have their own LOS F test) in the system. Report – Maximum percent of full directional street links at any one time. Percent of analysis period during which one or more directional street links are "full." Interpretation – The geographic and temporal percents of "full" links are indicators of potential capacity and safety problems. The lower, the better. Since a common goal for designs of new turn bays is to provide storage for the 95 percentile demand per cycle, a goal of no more than 5 percent of the analysis period with full street links would be desirable for a new design.
Surface Street: Percent Turn Bay Overflows	Use – To identify geographic and temporal extent of unsatisfactory operations for urban streets in system. Turn bay overflows reduce through capacity and are potential safety concerns. Definition – Two percents are computed: Maximum percent of full turn bays at any onetime during the analysis period. Percent of analysis period when any turn bay is full. Computation – For each second in the analysis period compare the distance of back bumper of last car in turn bay from stop bar to total estimated turn bay storage length. Determine if the back of queue is within 25 feet of storage length of turn bay. Accumulate number of seconds during analysis period when one or more turn bays are full. The percent of the analysis period with one or more full turn bays is the number of seconds when one or more turn bays are full divided by the number of seconds in the analysis period. Tally number of turn bays meeting this "full" criterion at any one time. The maximum percent full turn bays is the maximum number of full turn bays at any one time divided by the total number of turn bays in the system. Report – Maximum percent of full turn bays at any one time. Percent of analysis period during which one or more turn bays are "full." Interpretation – The geographic and temporal percents of "full" turn bays are indicators of potential capacity and safety problems. The lower, the better. Since a common goal for designs of new turn bays is to provide storage for the 95 percentile demand per cycle, a goal of no more than 5 percent of the analysis period with full turn bays would be desirable for a new design.

Their use, definition, computation, reporting, and interpretation are presented in this table.

There are, of course, many other MOEs that every good planning or engineering study also will want to consider, such as, user benefits, user costs, fuel consumption, noise, air pollutant emissions, and safety. However, the 5 basic system MOEs listed above will give the decision-maker and the analyst a good idea of how well the system is performing and how one alternative improvement compares to another. Chapter 6 describes them in more detail.

Additional Key System MOEs for Future Research

The technical review suggested four additional key MOEs that are likely to be valuable to decision-makers but could not be included at this time due to several uncertainties about them requiring more research and additional practical experience in order to better define them. They are:

Buffer Index;
Fuel Consumption Ratio;
NOx Emissions Ratio; and
Safety.

The buffer index is one of a dozen different measures of travel-time reliability. Future research needs to address the selection of the appropriate set of reliability measures for decision-making, as well as the extent to which the variability already incorporated into microsimulation models can be expanded to include the much larger variability associated with demand fluctuations and capacity reducing incidents in the real world.

The fuel consumption and NOx emissions ratios are an attempt to incorporate energy consumption and air pollution considerations in an easily understandable form for decision-making. However, there are several issues of how these indices might be computed, and consequently there are concerns as to their real meaning. Indeed, the United States Environmental Protection Agency may not accept air quality conformity analyses results that are produced by any tool except EPA approved tools.

Safety is one measure of effectiveness that many analysts would like to include in operations analysis considerations, however; the tools to do so are still very limited and in the development stage, where available.

Comparison of MOEs between HCM and Other Analysis Tools

Since each microscopic and macroscopic analysis tool (including the Highway Capacity Manual method) has notably different definitions of what constitutes stopped and queued vehicles and because the tools also vary significantly in the determination of which vehicles to include in the computations (vehicles that both entered and exited during the analysis period, vehicles that did neither, or vehicles that either entered or exited but not both during the analysis period), it is not feasible for an analyst to take the macroscopic output from one tool, apply a conversion factor (or procedure) and compare the results to that of another tool. The analyst simply does not have access to sufficient data at the macroscopic level to be able to compare MOEs across tools.

This conclusion means that looking up Highway Capacity Manual levels of service using MOEs produced by a different analytical method is not appropriate. It is prone to bias and error.

For similar reasons, the analyst should not be switching tools in the middle of a comparison between alternatives. One set of tools should be consistently used to evaluate MOEs across all alternatives. One should not evaluate one alternative with one tool, another alternative with a second tool, and then use the MOEs produced by both tools to select among the alternatives.

Comparison of results between tools and methods is possible only if the analyst looks at the lowest common denominator shared by all field data collection and analytical tools: vehicle trajectories. [The author is indebted to Dr. Hani Mahmassani for suggesting this concept.] At this microscopic level, the analyst can compare field data to analysis tool outputs, whether the tool is microscopic or macroscopic. By computing aggregate, macroscopic MOEs from the vehicle trajectory data the analyst can compare the results of macroscopic and microscopic tools to field data and to each other in a consistent manner. This is the only appropriate method for comparing results between tools, validating the model results against field data, or using the outputs of other tools to compute level of service as defined by the Highway Capacity Manual.

Consequently, it would be highly desirable if all microscopic analysis tools had the option of generating vehicle trajectory data in a universally readable format (for example, a generally accepted database format, such as DBASE), so as to make post-processing and comparison of results across tools possible.

Practical Application of Recommended MOE System

This report concludes with a chapter illustrating the computation and interpretation of the recommended systemwide MOEs for a freeway and an urban arterial street.

Traffic Analysis Toolbox Volume VI: Definition, Interpretation, and Calculation of Traffic Analysis Tools Measures of Effectiveness