Measures of Effectiveness and Validation Guidance for Adaptive Signal Control Technologies
Contact Information: Operations Feedback at OperationsFeedback@dot.gov.
Printable version [PDF 10 MB]
You may need the Adobe® Reader® to view the PDFs on this page.
Table of Contents
Foreword
Chapter 1. Introduction
Purpose
Background
Chapter 2. Traffic Signal Operations Objectives that can be applied by ASCT
Goals for Traffic Signal Operations
Minimize Congestion
Prevent or Delay Oversaturation
Accommodate Long-Term Variability
Manage Incidents and Events
Traffic Signal Operations Objectives
Smooth Flow
Maximize Throughput
Access Equity
Manage Queues
Variable Objectives
Changing Objectives by Time of Day
Long Term Variability and Events
Summary of Operational Objectives for ASCT
Chapter 3. Methods for Validation of ASCT Deployments
Summary of Literature Review
Chapter 4. Candidate Measures of Effectiveness for Validation of Operational Objectives
Route Travel Time
Route Average Speed
Link Travel Time/Link Average Speed
Route Travel Delay
Link Travel Delay
Route Traffic Volume
Time to Process Equivalent Volume
Green Bandwidth on a Route
Percent Arrivals on Green
Platoon Ratio
Number of Stops per Mile
Phase Green Occupancy Ratio
Served Volume to Capacity Ratio
Queue Length by Movement
Oversaturation Metrics
Reliability of Travel Time
90th or 95th Percentile Travel Times
Buffer Time
Planning Time
Reliability Indices
Reliability of Phase Performance
Derived MOEs
Fuel Use
Emissions
Net Present Value and Benefit/Cost Ratios
Down-Selection of Metrics for Assessment of Operational Objectives
Chapter 5. Validation Guidance
Summary of Data and Measures of Effectiveness to Validate Operational Objectives
Smooth Flow Objective
Configuration of Travel Time Routes and Conducting Travel Time Runs
Access Equity Objective
Throughput Objective
Travel Time Reliability Objective
Improvements to Validation Processes
Data Collection During Off-Peak Periods
Validation of Objectives Instead of Comparison with Existing Operations
Consideration of More Traditional Engineering Modifications that Cost Less
Consider an ON and OFF Study
Consider Measurement of Failure Modes and Incident Conditions
Consider a Longer-Term Program of Performance Monitoring
Chapter 6. Future Research and Development
Chapter 7. References
Chapter 8. Literature Review
Route Travel Times and Stops
Side-Street Performance
Before and After Format
On/Off Format
Quality of "Before" Timings
Reporting of Results
Extrapolation to Benefit/Cost
Explicit Statement of Goals
Chapter 9. Annotated Bibliography of Representative ASCT Evaluation Studies
Alphabetical Listing by Primary Author or Agency
Appendix A. Technical Approach for Collecting MOEs
High-Resolution Phase Timing and Detector Data
Vehicle Re-Identification Data
GPS Probe Data
Midblock Detection Data
Appendix B. Tool Design and Development
Processing Components
Configuration Data
GPS Probe Data
High-Resolution Phase Timing and Detector Data
Bluetooth Scanner Data
Tube Counter Data
MOE Calculations
Probe Travel Time Data
Bluetooth Scanner Data
Tube Counter Data
Phase Timing and Detector Data
Before and After Comparisons for Validation of Operational Objectives
Reliability Estimation
Throughput Measurement
Statistical Significance Computations
Summary
Appendix C. Field Testing of the Validation Methodology
Site Location
Site Characteristics
Location of Traffic Signals
Characteristics of Baseline Traffic Signal Operation
Detection Layout
Characteristics of Adaptive Traffic Signal Operation
Location of Bluetooth Detectors
Location of 24-Hour Tube Counters
Probe Travel Time Routes
Adaptive On/Off Schedule
Data Collection Discussion
Phase Timing Data
Bluetooth Travel Time Data
24-Hour Tube Count Data
Probe Travel Time Data
Appendix D. Measures of Effectiveness Analysis and Findings
GPS Probe Travel Time Findings
Buffer Time Analysis
Summary of GPS Probe Data Analysis
Findings Using Bluetooth Travel Time Detectors
Correlation of Travel Times to Route Volumes
Summary of Bluetooth Travel Time Analysis
Findings Using High Resolution Phase and Detector Data
Summary Results for Time of Day Analysis of GOR Performance
Summary Results for Time of Day Analysis of Percent Arrivals on Green
Examples of More Detailed Analysis of High-Resolution Phase Timing Data
Example of Pitfalls in Reporting and Consideration of Percent Improvement Data
Volume Counter Findings for Throughput Estimation
Appendix E. Findings for Validation of ASCT Operational Objectives
Findings for the Access Equity Objective
Findings for the Pipeline Objective
Findings for the Travel Time Reliability Objective
Findings for the Throughput Objective
Qualitative versus Quantitative Validation
Appendix F. Additional Supporting Data from the Field Testing
Average Route Travel Time Comparisons
Appendix G. Green Occupancy Ratio Performance Summaries
Appendix H. Throughput Performance Metrics
List of Tables
Table 1. Mapping of MOEs to Objectives and Data Sources.
Table 2. Issues Identified in Literature Review.
Table 3. Candidate MOEs for Validation of Operational Objectives.
Table 4. HCM Arrival Type to Platoon Ratio Assignments.
Table 5. Example Quantitative Measures of Oversaturation.
Table 6. Proposed MOEs for each Operational Objective.
Table 7. Identification of Data Sources and MOEs for each Operational Objective.
Table 8. Mapping of MOEs to Data sources.
Table 9. Coordinated Timing Pattern Schedule.
Table 10. Probe Travel Time Schedule.
Table 11. Travel Time Performance Comparisons For GPS Probe Runs.
Table 12. Qualitative Comparison of Average Trip Time and Reliability – GPS Probe Data.
Table 13. Summary of Travel Time Reliability Comparisons from 11/4-11/15 (adaptive) versus 11/16-11/21 (Coordination).
Table 14. Qualitative Comparison of Average Trip Time and Trip Reliability – Bluetooth Data.
Table 15. Difference between ASCT and Coordination for GOR by Time of Day – Southern and Superstition Springs (Weekdays).
Table 16. Difference between ASCT and Coordination for Percent Arrivals on Green by Time of Day – Power and Southern (Weekdays).
Table 17. Difference between ASCT and Coordination for Percent Arrivals on Green by Time of Day – Power and Southern (Weekends).
Table 18. Difference between ASCT and Coordination for GOR by Time of Day – Southern and Superstition Springs (Weekends).
Table 19. Difference between ASCT and Coordination for GOR by Time of Day – Power and Southern (Weekdays).
Table 20. Difference between ASCT and Coordination for GOR by Time of Day – Power and Southern (Weekends).
Table 21. Difference between ASCT and Coordination for GOR by Time of Day – Power and Hampton (Weekdays).
Table 22. Difference between ASCT and Coordination for GOR by Time of Day – Power and Hampton (Weekends).
Table 23. Difference between ASCT and Coordination for GOR by Time of Day – Power and U.S. 60 (Weekdays).
Table 24. Difference between ASCT and Coordination for GOR by Time of Day – Power and U.S. 60 (Weekends).
Table 25. Difference between ASCT and Coordination for GOR by Time of Day – Power and Superstition Springs (Weekdays).
Table 26. Difference between ASCT and Coordination for GOR by Time of Day – Power and Superstition Springs (Weekends).
List of Figures
Figure 1. Diagram. Systems Engineering Process with Needs and Validation Processes Highlighted.
Figure 2. Line Graph. Comparison of normal (top) and special event traffic (bottom).
Figure 3. Illustration. Illustration of Data Used to Calculated Percent Arrivals on Green.
Figure 4. Illustration. Arrival Type Performance Before and After Offset Modifications.
Figure 5. Illustration. Illustration of Comparison of Detection Length on Calculation of GOR and V/C.
Figure 6. Equation. Served Volume to Capacity.
Figure 7. Illustration. Example Queue Length Estimation Results.
Figure 8. Illustration. Representation of the Value of Reliability Improvements.
Figure 9. Illustration. Reliability Measures Compared to Average Congestion Measures.
Figure 10. Scatter Graphs. V/C Ratio by Phase by Time of Day.
Figure 11. Screen Shot. Example Split Monitoring Summary Statistics.
Figure 12. Illustration. Example of Speed Versus Distance and Time-Space Diagram Displays in Studies.
Figure 13. Line Graph. Example Comparison of Volume and Distance-Weighted "Total System" Travel Times.
Figure 14. Table. Example of Results Reported as Time Spent in Congested Conditions.
Figure 15. Line Graph. Example Comparison of Travel Times by Time of Day.
Figure 16. Exhibit. Example of Side-Street Performance Summary.
Figure 17. Table. A Typical Evaluation Report Summary Table.
Figure 18. Table. Summary of Performance in "Better," "Worse," and "Same."
Figure 19. Table. Summary of Performance Adjusted for Volume Changes.
Figure 20. Diagram. Example of Aggregate Performance Indicators For Summary Performance of a Network.
Figure 21. Exhibit. Graphical Example of Statistically Different Performance by Movement.
Figure 22. Exhibit. Example Assumptions for B/C Estimation.
Figure 23. Exhibit. Example Benefit/Cost Exhibit.
Figure 24. Diagram. Generic Illustration of the Data Collection and Processing Methodology.
Figure 25. Screen Shot. Hourly Phase and Detector Data Files from a Controller.
Figure 26. Message. Example XML Feed from BT Scanner Server.
Figure 27. Table. Sample GPS Tracking Data.
Figure 28. Diagram. Focus of the Tools Development Process.
Figure 29. Screen Shot. Link Configuration Interface.
Figure 30. Screen Shot. Detector Configuration Interface.
Figure 31. Screen Shot. Android Smartphone Application Screens.
Figure 32. Screen Shot. Import Interface.
Figure 33. Table. Sample Traffic Counter Data Format.
Figure 34. Screen Shot. Report Selection Dialog for Before and After Comparison.
Figure 35. Screen Shot. Sample Link by Link Travel Time and Delay Comparison Table.
Figure 36. Screen Shot. Sample Travel Time Reliability Display.
Figure 37. Map. Test Site Location.
Figure 38. Map. Test Site Traffic Signals.
Figure 39. Diagram. Detector Configuration and Phasing.
Figure 40. Line Graph. Example ASCT Cycle Adjustment Profile for Power and Southern.
Figure 41. Line Graph. Example ASCT Split Adjustment Profile for Power and Southern (Fridays, 7 am-7 pm).
Figure 42. Map. Bluetooth Detector Locations.
Figure 43. Map. 24-Hour Tube Count Locations.
Figure 44. Map. Probe Travel Time Routes.
Figure 45. Calendar. ASCT On/Off Schedule.
Figure 46. Schedule. Data Availability Over the Study Period.
Figure 47. Screen Shot. Comparison of Average Speed for Route 1.
Figure 48. Bar graph. Comparison of Buffer Times for the Eight Routes.
Figure 49. Bar graph. Buffer Time Differences for Each of the Six Bluetooth Pairs.
Figure 50. Line Graph. Comparison of Route Travel Times for Varying Volumes – Southbound.
Figure 51. Line Graph. Comparison of Route Travel Times for Varying Route Volume – Southbound.
Figure 52. Table. Comparison of Travel Times on Power Road Before and During Black Friday.
Figure 53. Line Graph. Average GOR Over Time for ASCT Operation at Power and Southern on Saturdays.
Figure 54. Line Graph. Average GOR Values Over Time for Coordinated Operation at Power and Southern on Saturdays.
Figure 55. Line Graph. Comparison of Average Platoon Ratio 7-10 AM on Representative Weekdays – North and Southbound at Power and Southern.
Figure 56. Line Graph. Percent Differences in GOR between Adaptive and Coordination on Sundays at Intersection 1.
Figure 57. Line Graph. Average GOR Values on Sundays at Intersection 1.
Figure 58. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "C", Northbound.
Figure 59. Line Graph. Comparison of Average Speed for Route 2.
Figure 60. Line Graph. Comparison of Average Speed for Route 3.
Figure 61. Line Graph. Comparison of Average Speed for Route 4.
Figure 62. Line Graph. Comparison of Average Speed for Route 5.
Figure 63. Line Graph. Comparison of Average Speed for Route 6.
Figure 64. Line Graph. Comparison of Average Speed for Route 7.
Figure 65. Line Graph. Comparison of Average Speed for Route 8.
Figure 66. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "C", Southbound.
Figure 67. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "B", Eastbound.
Figure 68. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "B", Westbound.
Figure 69. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "D", Northbound.
Figure 70. Line Graph. Throughput Performance by Time of Day – Volume Counter "D" Northbound.
Figure 71. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "D", Southbound.
Figure 72. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "A", Westbound.
Figure 73. Line Graph. Throughput Performance for ASCT versus Coordination – Volume Counter "A", Eastbound.
U.S. Department of Transportation
Federal Highway Administration
Office of Operations
1200 New Jersey Avenue, SE
Washington, DC 20590
www.ops.fhwa.dot.gov
July 2013
Publication #: FHWA-HOP-13-031
