Office of Operations
21st Century Operations Using 21st Century Technologies

Measures of Effectiveness and Validation Guidance for Adaptive Signal Control Technologies

Contact Information: Operations Feedback at OperationsFeedback@dot.gov.

Printable version [PDF 10 MB]
You may need the Adobe® Reader® to view the PDFs on this page.

Table of Contents

Foreword

Chapter 1. Introduction

Purpose

Background

Chapter 2. Traffic Signal Operations Objectives that can be applied by ASCT

Goals for Traffic Signal Operations

Minimize Congestion

Prevent or Delay Oversaturation

Accommodate Long-Term Variability

Manage Incidents and Events

Traffic Signal Operations Objectives

Smooth Flow

Maximize Throughput

Access Equity

Manage Queues

Variable Objectives

Changing Objectives by Time of Day

Long Term Variability and Events

Summary of Operational Objectives for ASCT

Chapter 3. Methods for Validation of ASCT Deployments

Summary of Literature Review

Chapter 4. Candidate Measures of Effectiveness for Validation of Operational Objectives

Route Travel Time

Route Average Speed

Link Travel Time/Link Average Speed

Route Travel Delay

Link Travel Delay

Route Traffic Volume

Time to Process Equivalent Volume

Green Bandwidth on a Route

Percent Arrivals on Green

Platoon Ratio

Number of Stops per Mile

Phase Green Occupancy Ratio

Served Volume to Capacity Ratio

Queue Length by Movement

Oversaturation Metrics

Reliability of Travel Time

90th or 95th Percentile Travel Times

Buffer Time

Planning Time

Reliability Indices

Reliability of Phase Performance

Derived MOEs

Fuel Use

Emissions

Net Present Value and Benefit/Cost Ratios

Down-Selection of Metrics for Assessment of Operational Objectives

Chapter 5. Validation Guidance

Summary of Data and Measures of Effectiveness to Validate Operational Objectives

Smooth Flow Objective

Configuration of Travel Time Routes and Conducting Travel Time Runs

Access Equity Objective

Throughput Objective

Travel Time Reliability Objective

Improvements to Validation Processes

Data Collection During Off-Peak Periods

Validation of Objectives Instead of Comparison with Existing Operations

Consideration of More Traditional Engineering Modifications that Cost Less

Consider an ON and OFF Study

Consider Measurement of Failure Modes and Incident Conditions

Consider a Longer-Term Program of Performance Monitoring

Chapter 6. Future Research and Development

Chapter 7. References

Chapter 8. Literature Review

Route Travel Times and Stops

Side-Street Performance

Before and After Format

On/Off Format

Quality of "Before" Timings

Reporting of Results

Extrapolation to Benefit/Cost

Explicit Statement of Goals

Chapter 9. Annotated Bibliography of Representative ASCT Evaluation Studies

Alphabetical Listing by Primary Author or Agency

Appendix A. Technical Approach for Collecting MOEs

High-Resolution Phase Timing and Detector Data

Vehicle Re-Identification Data

GPS Probe Data

Midblock Detection Data

Appendix B. Tool Design and Development

Processing Components

Configuration Data

GPS Probe Data

High-Resolution Phase Timing and Detector Data

Bluetooth Scanner Data

Tube Counter Data

MOE Calculations

Probe Travel Time Data

Bluetooth Scanner Data

Tube Counter Data

Phase Timing and Detector Data

Before and After Comparisons for Validation of Operational Objectives

Reliability Estimation

Throughput Measurement

Statistical Significance Computations

Summary

Appendix C. Field Testing of the Validation Methodology

Site Location

Site Characteristics

Location of Traffic Signals

Characteristics of Baseline Traffic Signal Operation

Detection Layout

Characteristics of Adaptive Traffic Signal Operation

Location of Bluetooth Detectors

Location of 24-Hour Tube Counters

Probe Travel Time Routes

Adaptive On/Off Schedule

Data Collection Discussion

Phase Timing Data

Bluetooth Travel Time Data

24-Hour Tube Count Data

Probe Travel Time Data

Appendix D. Measures of Effectiveness Analysis and Findings

GPS Probe Travel Time Findings

Buffer Time Analysis

Summary of GPS Probe Data Analysis

Findings Using Bluetooth Travel Time Detectors

Correlation of Travel Times to Route Volumes

Summary of Bluetooth Travel Time Analysis

Findings Using High Resolution Phase and Detector Data

Summary Results for Time of Day Analysis of GOR Performance

Summary Results for Time of Day Analysis of Percent Arrivals on Green

Examples of More Detailed Analysis of High-Resolution Phase Timing Data

Example of Pitfalls in Reporting and Consideration of Percent Improvement Data

Volume Counter Findings for Throughput Estimation

Appendix E. Findings for Validation of ASCT Operational Objectives

Findings for the Access Equity Objective

Findings for the Pipeline Objective

Findings for the Travel Time Reliability Objective

Findings for the Throughput Objective

Qualitative versus Quantitative Validation

Appendix F. Additional Supporting Data from the Field Testing

Average Route Travel Time Comparisons

Appendix G. Green Occupancy Ratio Performance Summaries

Appendix H. Throughput Performance Metrics

List of Tables

Table 1. Mapping of MOEs to Objectives and Data Sources.

Table 2. Issues Identified in Literature Review.

Table 3. Candidate MOEs for Validation of Operational Objectives.

Table 4. HCM Arrival Type to Platoon Ratio Assignments.

Table 5. Example Quantitative Measures of Oversaturation.

Table 6. Proposed MOEs for each Operational Objective.

Table 7. Identification of Data Sources and MOEs for each Operational Objective.

Table 8. Mapping of MOEs to Data sources.

Table 9. Coordinated Timing Pattern Schedule.

Table 10. Probe Travel Time Schedule.

Table 11. Travel Time Performance Comparisons For GPS Probe Runs.

Table 12. Qualitative Comparison of Average Trip Time and Reliability – GPS Probe Data.

Table 13. Summary of Travel Time Reliability Comparisons from 11/4-11/15 (adaptive) versus 11/16-11/21 (Coordination).

Table 14. Qualitative Comparison of Average Trip Time and Trip Reliability – Bluetooth Data.

Table 15. Difference between ASCT and Coordination for GOR by Time of Day – Southern and Superstition Springs (Weekdays).

Table 16. Difference between ASCT and Coordination for Percent Arrivals on Green by Time of Day – Power and Southern (Weekdays).

Table 17. Difference between ASCT and Coordination for Percent Arrivals on Green by Time of Day – Power and Southern (Weekends).

Table 18. Difference between ASCT and Coordination for GOR by Time of Day – Southern and Superstition Springs (Weekends).

Table 19. Difference between ASCT and Coordination for GOR by Time of Day – Power and Southern (Weekdays).

Table 20. Difference between ASCT and Coordination for GOR by Time of Day – Power and Southern (Weekends).

Table 21. Difference between ASCT and Coordination for GOR by Time of Day – Power and Hampton (Weekdays).

Table 22. Difference between ASCT and Coordination for GOR by Time of Day – Power and Hampton (Weekends).

Table 23. Difference between ASCT and Coordination for GOR by Time of Day – Power and U.S. 60 (Weekdays).

Table 24. Difference between ASCT and Coordination for GOR by Time of Day – Power and U.S. 60 (Weekends).

Table 25. Difference between ASCT and Coordination for GOR by Time of Day – Power and Superstition Springs (Weekdays).

Table 26. Difference between ASCT and Coordination for GOR by Time of Day – Power and Superstition Springs (Weekends).

List of Figures

Figure 1. Diagram. Systems Engineering Process with Needs and Validation Processes Highlighted.

Figure 2. Line Graph. Comparison of normal (top) and special event traffic (bottom).

Figure 3. Illustration. Illustration of Data Used to Calculated Percent Arrivals on Green.

Figure 4. Illustration. Arrival Type Performance Before and After Offset Modifications.

Figure 5. Illustration. Illustration of Comparison of Detection Length on Calculation of GOR and V/C.

Figure 6. Equation. Served Volume to Capacity.

Figure 7. Illustration. Example Queue Length Estimation Results.

Figure 8. Illustration. Representation of the Value of Reliability Improvements.

Figure 9. Illustration. Reliability Measures Compared to Average Congestion Measures.

Figure 10. Scatter Graphs. V/C Ratio by Phase by Time of Day.

Figure 11. Screen Shot. Example Split Monitoring Summary Statistics.

Figure 12. Illustration. Example of Speed Versus Distance and Time-Space Diagram Displays in Studies.

Figure 13. Line Graph. Example Comparison of Volume and Distance-Weighted "Total System" Travel Times.

Figure 14. Table. Example of Results Reported as Time Spent in Congested Conditions.

Figure 15. Line Graph. Example Comparison of Travel Times by Time of Day.

Figure 16. Exhibit. Example of Side-Street Performance Summary.

Figure 17. Table. A Typical Evaluation Report Summary Table.

Figure 18. Table. Summary of Performance in "Better," "Worse," and "Same."

Figure 19. Table. Summary of Performance Adjusted for Volume Changes.

Figure 20. Diagram. Example of Aggregate Performance Indicators For Summary Performance of a Network.

Figure 21. Exhibit. Graphical Example of Statistically Different Performance by Movement.

Figure 22. Exhibit. Example Assumptions for B/C Estimation.

Figure 23. Exhibit. Example Benefit/Cost Exhibit.

Figure 24. Diagram. Generic Illustration of the Data Collection and Processing Methodology.

Figure 25. Screen Shot. Hourly Phase and Detector Data Files from a Controller.

Figure 26. Message. Example XML Feed from BT Scanner Server.

Figure 27. Table. Sample GPS Tracking Data.

Figure 28. Diagram. Focus of the Tools Development Process.

Figure 29. Screen Shot. Link Configuration Interface.

Figure 30. Screen Shot. Detector Configuration Interface.

Figure 31. Screen Shot. Android Smartphone Application Screens.

Figure 32. Screen Shot. Import Interface.

Figure 33. Table. Sample Traffic Counter Data Format.

Figure 34. Screen Shot. Report Selection Dialog for Before and After Comparison.

Figure 35. Screen Shot. Sample Link by Link Travel Time and Delay Comparison Table.

Figure 36. Screen Shot. Sample Travel Time Reliability Display.

Figure 37. Map. Test Site Location.

Figure 38. Map. Test Site Traffic Signals.

Figure 39. Diagram. Detector Configuration and Phasing.

Figure 40. Line Graph. Example ASCT Cycle Adjustment Profile for Power and Southern.

Figure 41. Line Graph. Example ASCT Split Adjustment Profile for Power and Southern (Fridays, 7 am-7 pm).

Figure 42. Map. Bluetooth Detector Locations.

Figure 43. Map. 24-Hour Tube Count Locations.

Figure 44. Map. Probe Travel Time Routes.

Figure 45. Calendar. ASCT On/Off Schedule.

Figure 46. Schedule. Data Availability Over the Study Period.

Figure 47. Screen Shot. Comparison of Average Speed for Route 1.

Figure 48. Bar graph. Comparison of Buffer Times for the Eight Routes.

Figure 49. Bar graph. Buffer Time Differences for Each of the Six Bluetooth Pairs.

Figure 50. Line Graph. Comparison of Route Travel Times for Varying Volumes – Southbound.

Figure 51. Line Graph. Comparison of Route Travel Times for Varying Route Volume – Southbound.

Figure 52. Table. Comparison of Travel Times on Power Road Before and During Black Friday.

Figure 53. Line Graph. Average GOR Over Time for ASCT Operation at Power and Southern on Saturdays.

Figure 54. Line Graph. Average GOR Values Over Time for Coordinated Operation at Power and Southern on Saturdays.

Figure 55. Line Graph. Comparison of Average Platoon Ratio 7-10 AM on Representative Weekdays – North and Southbound at Power and Southern.

Figure 56. Line Graph. Percent Differences in GOR between Adaptive and Coordination on Sundays at Intersection 1.

Figure 57. Line Graph. Average GOR Values on Sundays at Intersection 1.

Figure 58. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "C", Northbound.

Figure 59. Line Graph. Comparison of Average Speed for Route 2.

Figure 60. Line Graph. Comparison of Average Speed for Route 3.

Figure 61. Line Graph. Comparison of Average Speed for Route 4.

Figure 62. Line Graph. Comparison of Average Speed for Route 5.

Figure 63. Line Graph. Comparison of Average Speed for Route 6.

Figure 64. Line Graph. Comparison of Average Speed for Route 7.

Figure 65. Line Graph. Comparison of Average Speed for Route 8.

Figure 66. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "C", Southbound.

Figure 67. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "B", Eastbound.

Figure 68. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "B", Westbound.

Figure 69. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "D", Northbound.

Figure 70. Line Graph. Throughput Performance by Time of Day – Volume Counter "D" Northbound.

Figure 71. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "D", Southbound.

Figure 72. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "A", Westbound.

Figure 73. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "A", Eastbound.

United States Department of Transportation Federal Highway Administration

U.S. Department of Transportation
Federal Highway Administration
Office of Operations
1200 New Jersey Avenue, SE
Washington, DC 20590
www.ops.fhwa.dot.gov

July 2013

Publication #: FHWA-HOP-13-031

Office of Operations