Traffic Analysis Toolbox Volume III: Guidelines for Applying Traffic Microsimulation Modeling Software 2019 Update to the 2004 Version
Chapter 4. Error Checking
The error correction step is essential in developing a working model so that the calibration process does not result in parameters that are distorted to compensate for overlooked coding errors. This is Step 4 in the Microsimulation Analytical Process (Figure 8). The calibration process relies on the elimination of all major errors in demand and network coding before calibration.
Error checking involves various reviews of the coded network, coded demand, and default parameters. Error checking involves the following three stages:
Review Software Errors
The analyst should review the software and user group web sites to ensure that he or she is aware of the latest known "bugs" and user workarounds for the software. The analyst should ensure that he or she is using the latest version and "patch" of the software, if any.
Review Input Errors
A checklist for verifying the accuracy of the coded input data is provided below:
Driver behavior and vehicle characteristics:
The following techniques may be useful to increase the efficiency and effectiveness of the error-checking process:
Animation output enables the analyst to see the vehicle behavior that is being modeled and assess the reasonableness of the microsimulation model itself. Running the simulation model and reviewing the animation, even with artificial demands, can be useful to identify input coding errors. The analyst inputs a very low level of demand and then follows individual vehicles through the network. Aberrant vehicle behavior (such as unexpected braking or stops) is a quick indicator of possible coding errors. At this stage, the analyst is not required to perform multiple runs of the model by changing the random number seeds; a single random-number-seed run will suffice.
A two-stage process can be followed in reviewing the animation output. Run the animation at an extremely low demand level (so low that there is no congestion). The analyst should then trace single vehicles through the network and see where they unexpectedly slow down. These will usually be locations of minor network coding errors that disturb the movement of vehicles over the link or through the node. This test should be repeated for several different O-D zone pairs.
Once the extremely low demand level tests have been completed, then run the simulation at 50 percent of the existing demand level. At this level, demand is usually not yet high enough to cause congestion. If congestion appears, it may be the result of some more subtle coding errors that affect the distribution of vehicles across lanes or their headways. Check entry and exit link flows to verify that all demands are being correctly loaded and moved through the network.
The animation should be observed in close detail at key bottleneck areas to determine if the animated vehicle behavior is realistic. If the observed vehicle behavior appears to be unrealistic, the analyst should explore the following potential causes of the unrealistic animation in the order shown below:
A comparison of model animation to field design and operations is highly essential. Some of the things to look for include:
If the analyst has field-verified his or her expectations of traffic performance and has exhausted all possible input errors, and the simulation still does not perform to the analyst's satisfaction, there are still a few possibilities. The desired performance may be beyond the capabilities of the software, or there may be a software error.
Software limitations can be identified through careful review of the software documentation. If software limitations are a problem, the analyst might seek an alternate software program without the limitations. Advanced analysts can also write their own software interface with the microsimulation software (called an "application program interface" (API)) to overcome the limitations and produce the desired performance. Any changes made to override the simulation software's capabilities should be documented in the Methods and Assumptions document.
Software errors can be tested by coding simple test problems (such as a single link or intersection) where the result (such as capacity or mean speed) can be computed manually and compared to the model. Software errors can only be resolved by working with the tool developer.
Key Decision Point
The completion of error checking is a key decision point. The next task-model calibration-can be very time-consuming. Before embarking upon this task, the analyst should confirm that error checking has been completed, specifically:
Once the error checking has been completed, the analyst has a working model (though it is still not calibrated).
Example Problem: Error Checking
Continuing with the Alligator City problem from the previous chapters, the task is now to error check the coded base model.
Software: The latest version of the software was used. Review of the model documentation and other material in the software and user groups' web sites indicated that there were no known problems or bugs related to the network under study and the scenarios to be simulated.
Review of Input Data and Parameters: The coded input data were verified using the input files, the input data portion of the output files, static displays, and animation.
First, the basic network connectivity was checked, including its consistency with coded geometry and turning restrictions. All identified errors were corrected. For example, there was a fatal error that one of the SB Riverside Parkway links didn't exist. It was found that the link number had a typographical error.
Static network displays were used extensively to verify the number of lanes, lane use, lane alignment (i.e., lane number that feeds the downstream through link), and the location of lane drops. At this step, the consistency of link attributes was checked. For example, is the free-flow speed of ~90 km/h (55 mi/h) coded for all freeway links?
Next, the traffic demand data were checked. Input volumes at the network entrances were specified in four time slices. The input values were checked against the collected data.
Traffic signals coded at each intersection were reviewed. For fixed-time signals, the phasing and signal settings were checked. There was a fatal error that indicated Phase 1 at one of the intersections in Downtown Alligator City was incorrect. Phase 1 was defined as left-turns from the E-W street. But the cross street (N-S street) was coded as one-way. So, left turns are not allowed from the eastbound street. This was fixed.
Special attention was given to the traffic patterns at the interchange ramp terminals to avoid unrealistic movement. The software provisions (and options) were exercised to force the model not to assign movements to travel paths that were not feasible.
Vehicle characteristics were reviewed.
Review Animation: Following the checking of the input data, the model was run using very low demand volumes to verify that all of the vehicles travel the network without slowdowns. This step uncovered minor errors in the link alignments that needed adjusting.
Next, the traffic demands were specified to about 50 percent of the actual volumes and the simulation model was rerun. Animation was used to verify that all demands were properly loaded in the network links and the traffic signals were properly operating. The link and system performance measures (travel time, delay) were also checked for reasonableness (i.e., they should reflect free-flow conditions).
Careful checking of the animation revealed subtle coding problems. For example, the coded distance of a warning sign for exiting vehicles from the Victory Island Bridge affected the proper simulation of driver behavior. These problems were corrected.
Key Decision Point: The model, as revised throughout the error-checking process, was run with all the input data (actual demands) and the default model parameters. The output and animation were also reviewed and discussed with the Alligator City agency staff who were familiar with the study area. The conclusion was that the model is working properly.
In summary, when checking errors, the analyst should:
United States Department of Transportation - Federal Highway Administration