Data Needs, Availability, and Opportunities for Work Zone Performance Measures Webinar
March 19, 2013
Transcript
Printable Version [PDF 112KB]
PDF files can be viewed with the Acrobat® Reader®.
Contact Information: WorkZoneFeedback@dot.gov
Jennifer Symoun
Good afternoon or good morning to those of you to the West. Welcome to today's webinar on Data Needs, Availability, and Opportunities for Work Zone Performance Measures. My name is Jennifer Symoun and I will moderate today's webinar.
Before I go any further, I do want to let those of you who are calling into the teleconference for the audio know that you need to mute your computer speakers or else you will be hearing your audio over the computer as well. For those of you calling into the phone line, please note that your phone lines are listen-only.
Today's webinar is scheduled to last 90 minutes. We will have four presenters: Jawad Paracha of the Federal Highway Administration Work Zone Mobility and Safety Team; Gerald Ullman and Geza Pesti of the Texas A&M Transportation Institute; and Rachel Klein of Battelle. If during the presentation you think of a question, please type it into the chat area. Please make sure you send your question to "Everyone". Presenters will be unable to answer your questions during their presentations, but we'll take a few minutes at varying points throughout the presentation for questions. If we run out of time and there are unanswered questions we will attempt to get written responses from the presenters that will be emailed to all attendees.
The PowerPoint presentation used during the webinar is available for download from the file download box in the lower right corner of your screen. The presentation will also be available online within the next few weeks, along with a recording and a transcript of today's webinar. I will notify all attendees once these materials are posted online.
We're now going to go ahead and get started with our first presenter, Jawad Paracha of the Federal Highway Administration (FHWA) Work Zone Mobility and Safety Team.
As a reminder, if you have questions during the presentations please type them into the chat box and they will be addressed at the appropriate point during the presentation.
Jawad Paracha
Good afternoon. I'm Jawad Paracha with the Work Zone Management Team and Transportation Operations at FHWA. I would first like to thank you all for participating in the webinar. This webinar is based on a soon-to-be published report titled Guidance on Data Needs, Availability and Opportunities for Work Zone Performance Measures. This report builds on an earlier document published in September 2011, A Primer on Work Zone Safety and Mobility Performance Measures. We plan to notify participants of this webinar as soon as the report is available online.
First, let me go over the webinar structure. We will first discuss some basic work zone performance management concepts and information related to the guidance documents. This will be followed by four categories of work zone performance measures: safety, mobility, customer satisfaction, and agency/contractor-related measures. We will have some time for Q&A after the mobility measures and after the safety measures, and again at the end of the webinar.
As we know, performance measures in general help in determining progress towards specific agency-defined goals and objectives. Work zone performance measures can be simply defined as metrics that help to quantify work zone impacts on travelers, residents, businesses, and workers. Work zone performance measures can be at the project level and agency program level. For example, if you are considering safety, it can be crash rate during the project duration, while at the program level it can be percentage of projects that exceed an acceptable crash rate in work zones. In the case of mobility, it can be queues observed during the project, or at the agency level it can be the percentage of projects which experience queues above a certain threshold. That’s just to emphasize that it's both at the project level and at the agency/program level.
Work zone performance measurement is a tool that helps agencies accomplish their mission more effectively. This slide lists some of the ways we can use work zone performance measures. These applications can range from work zone impacts assessment; developing impact mitigation strategies and related benefit-cost analysis; identifying trends (in the safety and mobility areas, for example); detecting problems as well as finding what's working well; managing and improving processes; and using performance measurement data for running effective public information and outreach campaigns.
To further reiterate why we need to focus on work zone performance management, here is some language from the Rule on Work Zone Safety and Mobility, Subpart J that requires agencies to collect, analyze, and use work zone crash and operational data at both project and process levels. This also helps in doing effective work zone process reviews, which are required by the Rule to be done every two years. Also, I would like to mention the recent emphasis on performance management requirements in MAP-21, such as establishing national goals, determining performance measures, setting targets, and reporting. Overall, this is an important area to run an effective program and agency. With this, I'm going to request Gerald Ullman of the Texas A&M Transportation Institute to talk more about the guidance document.
Gerald Ullman
Thanks, Jawad. Good job of setting the stage for everyone. I want to go through our guidance document structure a little bit and where we came from in developing this particular document. We started with what most of you all have probably already acknowledged in your own agencies as some of the challenges. Great, we can do this. We know there are a lot of potential measures we could do — which ones are the most important? Based on the most important measures, what data do we need and where do we get that data? What we hopefully generated through this document and provide a glimpse of today is that in many cases, there are multiple ways to find the data that can support these measures. There are trade-offs, which are a function of cost as well as manpower and those kinds of things. We want to make sure we provide a broad breadth of examples of some of the things you can use. We want to focus to the extent we can on some of the more upcoming data sources. We had a discussion about data in the earlier performance measure primer that Jawad mentioned. There has been change and growth in the area of data, and this is offering more recent, more broader-based, and maybe more easily accessible data sources than we've had at any time in history.
How we got to the guidance document is it started with a list of 13 possible measurement categories that have been generated as part of previous work for FHWA. We looked at that initial data set, put structure around it, and came up with three key categories or dimensions of data and measures. Then we convened a practitioner panel of folks that deal in this area quite a bit and had them help us identify and prioritize specific measures that they saw as having the most value along these key dimensions and categories that we'll be talking about today. From that standpoint, it presents an advance over the primer in that we have a practitioner-oriented list of recommended measures. You'll find out as we go through that most of these are things you probably have looked at yourselves, or at least thought about, and it represents a geographically broad base of practitioners and their opinions on those measures.
The three dimensions of data that we refer to include those relating to performance. We call those the measures that quantify the amount of effects. The four basic categories of performance data that we'll be talking about today are mobility, safety, customer satisfaction, and agency/contractor productivity. Exposure data quantifies who or what was affected. An important aspect of performance measurement is being able to have exposure data when you're generating these measures themselves. Some data that's used for one of these categories — performance data, for example — can also be considered in another set of measures. Throughput would be one example; if you have the amount of throughput going through a work zone for measuring whether or not there was any diversion, that would be a performance measure on the mobility side of things, but you might also want that data as exposure data for estimating any effects on crashes. So we have categorized it this way, but keep in mind that these dimensions are somewhat fluid.
Indicator data are further stratification types of data. You may only be interested in queues or delays during peak periods, in some cases. If you're only worried about temporary lane closures, the time and duration of those temporary lane closures would be the indicator data on which you would focus the collection of your other data. In and of itself, it could also ultimately be a performance measure. To track how often a contractor had to close lanes or the total amount of hours used versus what it was projected would be, you could move the indicator data up and it becomes performance data. We'll spend most of our time on the performance data category for this webinar.
We presented in the guidance document a general performance measure selection process. It's very straightforward, with five key steps. You might argue that the order of these may change depending on your specific situation. In some instances, the work zones that you're going to measure may not be the second thing you decide, it may be the first thing you decide because you're going to focus on the highest priority work zones, and you know what you want to measure and monitor. In other instances, the decisions of which work zones to measure might be step 3 or 4 of the process as far as how you structure your monitoring and measurement program. I wouldn't worry too much about the order, but keep in mind that the key steps are here.
What we're going to talk about today is hopefully one of the more important things you're interested in, which is where can we get data that we can use for work zone performance measurement? We have three options. We can extract it from things we already have. One of the most important things to keep in mind is that there are a lot of data sources already out there under each key performance category. You have to extract it out of where it resides within the structure of the agency, or in some cases outside the agency, and pull it in and use it for performance measurement. If it doesn't exist or it doesn’t exist at the level it's desired, the next option is to collect it specifically for performance measurement purposes. The third option is to interpolate the data from existing or collected data. Maybe we have collected and/or existing data that doesn't fit for the set of performance measures I'm looking at. Maybe I collected it for mobility, but also want to use it for safety purposes. I may have to do some data manipulation to get what I want out of data that has already been collected.
As for the structure of the document, we have a general introduction and key tips on performance measure selection. Then we have chapters on each of the four performance measure categories that we mentioned earlier: mobility, safety, customer satisfaction, and agency/contractor productivity/efficiency measures. Each chapter will touch upon data sources, the measures themselves, some other tips, and those types of things.
With that, Geza is going to talk about the mobility-related performance measures chapter, and after that we'll break and take questions.
Geza Pesti
The mobility impacts of work zones are typically measured in terms of the following performance measures: throughput, delays, travel times, travel time reliability, and vehicle queues. There are several data sources for each of these performance measures. In the following slides we'll review both the existing data sources and some of the potential, near future options for data sources that we may be able to use.
Let's start with the throughput. Existing data sources for throughput may include traffic operation centers (TOC) and signal systems on arterials equipped with vehicle detection. These cameras can be used to monitor traffic and do some traffic counts from the video feed provided either from the detection at the intersection or at the TOC. Other typical data sources include toll facility usage data. Of course, this has to be fairly close to the work zone. If it was too far upstream and there is a significant distance between the toll facility and the work zone, then we may not be able to use this data source. Automatic traffic recording stations is another data source that can be used. Again, the location is very important; we may measure demand or throughput, depending on the capacity-demand ratio. For planning and programming purposes, we can also use average annual daily traffic (AADT). This is more applicable to look at the impact of a work zone. It can also be used for throughput estimation for a longer period of time if the demand is not exceeding capacity and there is no significant diversion.
Work zone-specific throughput data can be obtained from ITS deployment at the work zones. For example, sensors can measure speeds and can count traffic and classify vehicles. We can also use temporary mechanical data collection devices like tubes or in-pavement sensors, which are portable and can be removed. It's important that you archive the data you collect. Manual vehicle counts are more applicable to very short-term work zones with fairly small impacts.
If in addition to the vehicle throughput you're also interested in person throughput data, you may need look at per-vehicle occupancy levels, which can be manually counted by taking samples of vehicles passing through the work zone. You can also use video detection of pedestrians if pedestrian throughput is critical (in urban areas, for example).
For future potential data sources for throughput, we should consider connected vehicle technology, which may provide very good data, but it requires sufficient market penetration and it will be far in the future before we get there. We need both vehicle-to-vehicle and vehicle-to-infrastructure communications in order to get reasonable estimates of throughput. For example, if there are platoons of vehicles traveling in a cooperative adaptive cruise control environment, we could get from the connected vehicles the number of vehicles in a platoon, the platoon size, and the headway between the vehicles. When the platoon arrives at a roadside device, out would dump this data and we would know how many vehicles passed through the point. Again, it is important to have a good market penetration to use this data source. If we have a good estimate of the proportion of vehicles equipped with vehicle-to-vehicle and vehicle-to-to infrastructure communication technology then we can estimate the throughput from the data that they provide.
I mentioned that the location where we measure the throughput is important. In case of non-congested conditions, it really doesn't matter. We can measure anywhere, but we always measure the throughput or the demand, which are the same. However, if we have congestion and demand exceeds capacity, we have to make sure that throughput is measured downstream of the work zone, or if you want to get demand, you have to make sure that you measure the volume upstream of the longest estimated queue. This is an important thing to remember when you need to do some traffic counts or volume counts.
The guidance document contains several tables like this (and we will see it in the safety and the other performance measure sections too) that show the key considerations that you have to account for when you choose appropriate data sources. For example, in the case of throughput for any type of data source, where you measure is important, because you can measure throughput or demand depending on the location. If you want to account for day-to-day variation, you need to collect data for multiple days; otherwise, you are not able to get the data you need. If you use work zone ITS data, you must be certain that the data collected are archived for consequent performance measure calculations. These are just a few considerations. I'm not going through all of them because I don't have the time, but you will find it in the guidance report.
The next performance measure for which we looked at the data sources are delay, travel time, and travel time reliability. Existing agency data sources include — among others — the TOC spot speed sensor data. These could be loops or hi-definition radars, which count and classify vehicles at multiple points on the freeway system or arterials. We could also use TOC video cameras to track vehicles and estimate travel times. We could also use other technologies like automatic vehicle identification (AVI), where you identify vehicles at one point on the freeway and then re-identify them at another point downstream, and you can calculate the speed from the distance and the travel time. Also, you can use license plate recognition, which is very similar to the AVI.
In addition to the available agency data, we can also use work zone-specific travel time and delay data sources. These are extracted from ITS deployments at the work zone, typically speed sensors, which can be used to provide variable speed messages or queue warning systems. At the same time, they can archive data, which we can use to calculate performance measures. We can also use portable point-to-point travel time data collection devices or portable speed sensors which we can deploy in the work zone. Manual spot speed sampling can also be collected using radar and LIDAR devices. Travel time runs using floating car techniques through the work zone is another way to collect travel time and delay data and speed performance measures in the work zone. In addition to these, we can also estimate delays from the queue. If you just observe the queue and you don't have sensors in the work zone, you may be able to estimate the delay. In order to get the delay from the observed queue, you need to know the length of the queue, the length of the work zone, the speed in the work zone, and the speed in the queue. Assuming a linear speed-density relationship in the work zone area, the actual speed in the work zone can be estimated as half of the free-flow speed. As a function of the free-flow speed, work zone capacity, and normal capacity, we can estimate the speed within the queue. The queue delay and the delay within the work zone can be calculated with this formula: the travel time with the queue speed minus the travel time with the work zone speed limit, and similarly, travel time with the speed in the work zone minus the travel time with the work zone speed limit. With this, you can provide some delay estimation.
Potential future data sources for travel time and delay data include, in addition to the connected vehicle data, Bluetooth address matching, which has been used in several states (e.g., Texas, Indiana). They have Bluetooth reader stations along the roadway and they do anonymous address matching of vehicles and take some samples of those vehicles, which have Bluetooth-enabled devices in the car. Another potential data source already used in several departments of transportation (DOT) (e.g., Virginia DOT) is to use some third party or private data sources from companies that collect GPS-enabled mobile device data from vehicles. They use them as probe vehicles in the network and they sell the data to agencies.
This is an example of the use of Bluetooth data for estimating travel times and delays, from I-35 in Texas. There was a work zone, indicated by the red section. It's in the northbound direction and it was nighttime work zone from 7 p.m. to 7 a.m. You can see the work zone information in the table. There are four Bluetooth reader stations in the vicinity. There are more, but these are the stations which were affected by the work zone. We identified the affected Bluetooth segments where the speed dropped; you can see the speed drop within the speed profile of the segments. Based on these three segments, the Bluetooth-measured travel time is plotted as a blue line. Assuming the 65 mph was the free flow travel time, the normal travel time, we subtracted the actual travel time from the 65 mph travel time, and then we got the delay for this time period. The maximum delay in this case was 28.6 minutes, assuming the 65 mph free-flow speed. Of the three Bluetooth segments, one of them was 1 mile, one was 3 miles, and one was 5 miles. We also knew that the queue extended somewhere between these two locations because they were the affected Bluetooth segments. We could use the same approach to look at the impact of multiple work zones in the corridor. In this case, there are two work zones and one incident here. It's not only work zones, but incidents can be accounted for as well. Again, we have several Bluetooth stations between these two points, and the sum of the delays is plotted here. It was almost 30 minutes on an about 50 mile corridor.
This is another table to summarize what kind of considerations and trade-offs we have to account for when we select data sources for delay, travel time, and travel time reliability estimation. For example, TOC spot speed sensor data tends to be less accurate when congestion is present. Also, we need to make sure to that data is available and archived in order to perform these calculations. If you look at work zone ITS data, they have to be archived, not only collected for the operation of the work zone. If you want to do manual spot speed data collection, it is very labor-intensive, it is most useful for work zones with small impacts, and you want to assess the impact in a short period of time.
The fourth performance measure is traffic queues. Existing data sources for traffic queues are speed data extracted from work zone ITS deployments, or observation of queues from traffic cameras at TOCs, or we could use field personnel to observe the queue length at work zones.
If we use spot speed sensors to estimate queue location, these are the four steps we can use. We divide the roadway into regions, assuming uniform speed between the sensors. Then we constantly examine speeds and volumes hour by hour at each sensor location, and we compare the hourly speed/volume profiles across the sensors to identify queue length. Then we sum up region lengths at those regions that were impacted by the queue.
Here's an example that's easier to see. This is the work zone, and we can see that one lane is closed in a two-lane section. There are three sensors: one at 0.2 miles, one at 0.8 miles, and one at 1.3 miles upstream of the temporary lane closure. The project diary information indicates that there's a lane closure began 9 a.m. and 3:30 p.m. How can we use spot speed data to determine the queue locations? We said that we look at the hourly spot speed averages at each sensor location. For example, at the first sensor you can see that at 11:00, there is no queue, depending on how you define the queue. If you have a speed threshold of 30 mph, you can see that there is no queue because it is 40 mph. However, if look between 11 a.m. and 1 p.m., there is a queue at sensor number 1. So you can see what the queue is between sensor 1 and sensor 2, if sensor 2 did not have a queue at 11:30 a.m. Using this approach, you can identify the location of the queue at each hour. We said that at noon the queue was between sensor 1 and sensor 2. The distance between sensor 1 and sensor 2 was 0.6 miles. If we know that the queue was somewhere in there, but we don't know exactly where, we assume it was halfway between the two sensors. This is why we apply the 0.2 miles distance plus half of the difference of the next two sensors, which is 0.5 miles, is the expected length of queue at noon. We can do the same for each time interval and we can estimate the queue formation during the entire construction period.
Potential future traffic queue data sources can be captured from third party traveler information providers; for example, Google traffic maps. We can use third party data sources for travel time and speed data for queue estimation. Connected vehicle technology can provide data because they record vehicle position and speed at all times, so we can get speed profiles for each vehicle equipped with this technology. Again, this is still somewhat far in the future. I think I'm running too long in my time so I'm not mentioning this table, but it is the list of considerations and trade-offs for traffic queue detection.
Once we identify the data sources that are available, the agency has to decide which to use. This is the last step after they've identified all the available data sources. This is an example where an agency may develop a dashboard type application where they show the performance measures and data sources in real-time, but also archive it for future calculation of performance measures.
Jennifer Symoun
Thank you. We'll take about five minutes for questions. If we don't get through all of them, we'll try to get back to them at the end. The first question is, aren't we missing valuable data if vehicles are not equipped with electronic sensors?
Geza Pesti
I mentioned that it is necessary to have good market penetration of the tracking device or the technology that you are using. In the case of Bluetooth-based travel time estimation, we noted that we already have a quite reliable data source. However, if you want to use connected vehicles for throughput estimations, you need to have much better market penetration.
Jennifer Symoun
Are the readers set up as part of agency data collection, or are they specific to this project only and will be removed once the project is complete?
Gerald Ullman
That was given during Geza's discussion of the I-35 example. Those devices were installed as part of a construction traveler information system for the series of projects that were going on along I-35. However, Texas DOT is in process of developing a long-term vision of ITS monitoring and management along the entire corridor. There's a good chance that while they might not be in the exact same locations and on the same structures, the intent is that the use of Bluetooth monitoring for I-35 will continue once construction is complete on that corridor and it will feed into the overall management of the system.
Jennifer Symoun
Are hard copies of the guidebook available and how can I obtain one? Does it include generic sample calculations/examples or maybe some "best practices"? Is there generic companion software?
Gerald Ullman
There is not companion software. The guidebook is still being finalized and will then go to publication. Once that's ready to go and it's available, FHWA will be sending an email to everybody participating today on how to get the guidebook. It should be available in the next few months.
Jennifer Symoun
For the queue data example, did you set the sensor locations based on predicted "length of queue" or historical experience for this roadway? We typically use a uniform spacing on our sensors, say 0.5 miles apart.
Gerald Ullman
That's a good question. That example came from a permanent TOC location where a work zone was placed in there. They were permanently installed Wavetronix, and the system was installed prior to the work zone, so that's why it looks like there's not uniform spacing. If it's a set system, wherever the work zone starts is the start of the work zone and its location to the nearest sensor is going to depend on where the lane closure is relative to that. An important thing to keep in mind if you are going to use permanent sensor data for doing things like this is that you need to know where the start of the lane closure occurs. Not every system has uniform spacing throughout its region. Another factor that comes into play here is that the farther apart the sensors are, the less accurate they will be. If a sensor is out and you have a big gap between a couple of sensors, that reduces your accuracy. You have to work around those kinds of difficulties if you are using existing data collection sources for your measurement purposes.
Jennifer Symoun
Can't statistical sampling methods be applied to account for lack of vehicles with onboard Bluetooth devices?
Geza Pesti
Even if you use some sampling techniques, we don't have sufficient data. For example, if there is a complete stop and the vehicles are not moving, then we simply do not have any data available for that time period. This happened on I-35; for example, when there was a full closure, vehicles were diverted to a frontage road and they had to turn back, so there are points when they do not have data. However, if at nighttime there are not many vehicles and the sample size would be too small, we could use some sampling techniques based on previous timing intervals to improve or fill in the gaps.
Gerald Ullman
It's also important to keep in mind that the Bluetooth antennae themselves are not really a point sensor. It's sending out a signal and it's detecting a signal, and the ability to detect Bluetooth devices varies based on its proximity to the travel lanes, how many travel lanes across you have, weather conditions, etc. So even if we think 7 percent of vehicles have Bluetooth, extrapolating enough to say that this number of Bluetooth reads equivalent to this amount of volume, it's going to give you a ballpark, but it can have quite large errors. That's why we didn't bring it up too much, but it's a good point.
Jennifer Symoun
We'll continue at this point and come back to the remaining questions so we have time to get through the whole presentation.
Gerald Ullman
I will move on to the next category: safety-related performance measures. When we talk about safety performance measures, we generally are talking about three major categories: crash data itself; some type of safety surrogate measures, which are generally operational measures; and worker accidents.
Existing sources of crash data statewide crash database. If you are really into the details of a crash, you can even get down to the electronic or hard copy narratives and report forms that are filled out on crashes. We are increasingly seeing that agencies that have operations centers are collecting extensive amounts of data themselves on the incidents. A lot of incidents are crashes, and you can see use those incident databases for performance measurement purposes, as well. The nice thing about those is that the agency has it right away for much more real-time monitoring. A lot of states have indicated challenges in getting crash data from work zones. It can take several weeks or months before you can get to it, and by that time you've missed your opportunity to monitor things. The whole idea of using operations center incident data is very interesting. A similar category is that response agencies in the region and/or service patrols that are dispatched to go assist motorists or roam corridors in work zones can be a good source of data for crashes and for crash monitoring. The challenge with that is having a baseline against which to compare so you know whether your numbers have gone up or not relative to either where they were before or in the previous phase.
Right now there are a few agencies that collect their own work zone crash data. New York is the most classic example of a State DOT that collects their own set of data, as a companion to the law enforcement officer report forms. It allows them to pull data that they are interested in, monitor it, and analyze it on their basis. They also gather additional data elements you don't typically see in a crash report form. Depending on how crash report forms evolve over the next three years, you may see other states taking this approach. It does take quite a bit of work to set up and commitment from a high level of administrative support to make it happen, but it has allowed New York to make a number of changes to their policies because they had that data source available.
Geza mentioned that connected vehicles are coming down the road. It's going to be very good when that does come on and becomes an accessible data source; it has the opportunity to provide a rich source of safety-related, crash-related data. There will certainly be issues of privacy and other things that will come first before it's going to be easily accessible to agencies and others for performance monitoring, though.
I think I have already summarized most of the key considerations. There are time lags. One thing about the incident information we were mentioning from operations centers is that obviously you're going to have non-accident data as well, from vehicles stalls and that kind of stuff, so you'll have to work through that. With electronics, there's a simple coding effort to extract those out of your databases. It is somewhat uncertain at this time as to when connected vehicle data will be available for use by agencies as a performance measurement data source.
Safety surrogates are another popular source of data and type of measure that agencies and contractors like to use. There are things that you can target, like specific locations, and get the data much more quickly. There's a listing here of some of the very common ones: going out and using LIDARs or radars to get some spot data, looking at speed data predominantly from ITS sensors, even travel times — looking at where things slow down, stop, and/or speed up can be indicative of safety issues and used as a safety surrogate. A lot of agencies do very thorough reviews of work zones and even go as far as rating the quality of various devices and the overall setup and use those inspection scores as a surrogate, the idea being that if I get a better score year after year, that should result in improved safety. We believe that's the case. For the most part, very few of these surrogate measures or data sources have been correlated strongly to changes in actual crashes. It doesn't mean that they don't exist, just that data hasn't been gathered and processed in a way to demonstrate those associations.
A couple of future sources that we see in the surrogate area that I think need to be mentioned include the connected vehicle data initiative that's under way. In addition to crash data, the initiative is going to have near-miss data — high levels of braking activity, swerving, and all those kinds of things. We don't know exactly when those are going to be available, and privacy concerns and interaction with the private sector/ automobile companies will have to be worked out, but I think it has potential down the road.
Another data source that has potential is the use of microscopic traffic simulation output for safety surrogate measures. That is in a lot of ways similar to the idea of using connected vehicle data in that you're using very detailed MOEs, short headways, severe braking requirements, braking levels, etc., that you track in a simulation model and report as indicators of less safe conditions. I think there's been some work by FHWA and others to look at this. It doesn't have a real strong backing within practitioners yet. We found that out in our practitioner panel. A lot of folks are very nervous about this. I think there will need to be some more trial and error with this before people feel more comfortable with it, but it certainly does provide a way to look at things before they get built and to look for problem areas and those kinds of things that would help an agency or contractor develop a safer work zone.
I think I have touched upon these considerations for the most part. It's very easy to collect data with hand-held devices. Travel times are also pretty easy. You can do videotaped traffic behavior. It's very labor-intensive, but if you think you have a problem location, a camera along with videotaped data can be a very useful method of monitoring and conducting your own erratic maneuver/vehicle conflict study to get at numbers, and then you could assess the extent to which you could improve that through some type of a countermeasure.
The third category of safety measures deals with worker accidents. To this point, we have been talking predominantly about the traveler and the public. Worker safety is paramount to all agencies and contractors, and tracking that is a very important aspect of safety monitoring and performance measurement. Data sources include agency and contractor worker injury records. All states also have a worker compensation commission that compiles crash statistics. Federally, the Bureau of Labor has a statistics database. They don't necessarily have all injuries of all types for all workers — it's much more of a sampling process — but you can use these to assess general trends at the program level. If it comes down to wanting to track at the project level, you can rely on the contractor's or agency's ability to get specific injury records. Privacy issues are critical here, as we mentioned elsewhere.
It's tougher to see the connected vehicle initiative as a future data source in this area, but we theorized that the potential does exist, particularly if work vehicles are equipped with connected vehicle data so that we can track the proximity of multiple vehicles working out in the field and those kinds of things for monitoring purposes. This probably has a farther way to go than any of the other ones we've listed so far.
Privacy concerns are very critical. With the higher-level State worker compensation statistics and the Bureau of Labor statistics, the level of detail that you'll be able to get down to as far as what was going on with the particular project at the particular time and what the traffic conditions were and those kinds of detailed supporting data will be much more difficult to get than they will be when looking something that comes from an individual agency's report, which will show date, time, locations, etc. That's something to keep in mind.
The highest priority measures related to safety that we got from our practitioner panel are listed here. Certainly just monitoring changes in crash frequency, typically by type, is one of the easiest and most important ways to monitor safety. Speed limit compliance and speed variances are high priority safety surrogate measures, as rated by our practitioner panel. Frequency of worker accidents and injury rates are also listed. If you are doing crash frequency using just crash numbers as a performance measure, it is important that you keep in mind that you have to measure over the same time period or do a comparison to a baseline of the same time period for the measures to mean anything. Simply seeing you had four crashes at a work zone in the last month doesn't mean anything unless you know whether that's a lot or less compared to previous months, what was happening before construction, and those kinds of things. But it does illustrate how these don't have to be very complex and you don't need a high-level statistical modeling analysis and background performance functions to do this; you can have those, but it can be done much more simply than that.
As an example, one of the things we have in our document is a simple trend line tracking cumulative crashes over time relative to what crashes would have been if a work zone had not been in place, along with some confidence intervals. In this example, we want to know if we're getting significantly more crashes than we did before construction. We're on a road that normally had five crashes per month, and then over the first four months of this project, we had 7 crashes one month, 3, 10 and 7. Are crashes increasing significantly, or is it just normal random month-to-month variation? With this chart, you just plot month-by-month. At each month you're going to add 5, so the first month we would have 5 crashes, second month we would have 10, third month 15, fourth month 20 etc. All we do is add the month-by-month to see where we're tracking along this line. Even though we had 7 crashes the first month, that could have been due to normal noise in crash data. The fact that it went down to 3 the next month actually lined up with expectations; we have no evidence to say that we're having a crash issue at all in this work zone. The next month we had 10 crashes, so we have bumped above the even money line, which is this lowest number that would be a 50/50 yes or no increase. We're above the 75th percentile confidence interval that crashes have increased. Then we add the next month and we're up to where we're 90 percent sure that crashes are higher than they would have been if the work zone had not been in place. Depending on the level at which you would be concerned that crashes were something you were going to look at, then once you get into fourth month here it would be signaling that numbers are significantly higher than what they would have been had the work zone not be present. You would then institute whatever reaction you want to as an agency or contractor to look into that.
Questions?
Jennifer Symoun
Let's take a few minutes for questions. Is this data only for the mainland, or is it for all of the U.S., like Hawaii, Guam, and American Samoa?
Gerald Ullman
Data comes from wherever you are. There's nothing in the guidebook that is mainland-oriented only. The principles apply regardless of where you are, so I would say that it is valid for those locations as well.
Jennifer Symoun
For collecting throughput data, is anybody using video cameras mounted on drones?
Gerald Ullman
That's a good question. I heard some discussions about that, but I have not heard about it actually being done. Geza or Rachel, have you heard of that actually being done at this point?
Geza Pesti
I haven't.
Gerald Ullman
Okay. That doesn't mean it won't be done in the future. It was not actually something we even saw as a possibility, though I guess it certainly could be. I would be a little nervous about it, given what we know as far as drones and what they are being used for by the military.
Jennifer Symoun
Will this data also be shared with the Bureau of Labor Statistics?
Gerald Ullman
I think all fatalities have to be reported to the Bureau of Labor Statistics. When it comes to injuries, I think it's dependent upon each day — somebody could check if I'm incorrect — at what level you have to and what types of things you have to report. There is also some self-reporting; if you want to report this the Bureau of Labor Statistics takes those reports as well. I do know it's not a 100 percent sampling activity at this time, unfortunately.
Jennifer Symoun
Does anyone out there actually videotape traffic behavior?
Gerald Ullman
I don't know that we've identified any state or local agencies that have done that. We have been hired as a research agency in the past to investigate, and in fact one of the things we did on I-35 is put cameras up to monitor some high crash locations, both to allow Texas DOT to have quicker detection and to understand why those are a high crash locations. I think the answer is it depends. I don't know that I could name an agency that does it as a normal practice in day-to-day operations.
Jennifer Symoun
Work zone crashes are difficult to pin down, as getting a definition of a "work zone" that our State and local police departments understand is a challenge. Plus, is it workers present or is it the road geometry being modified from existing conditions that constitutes an active zone?
Gerald Ullman
That's absolutely correct. It's written as a question, but I think Neil made a good statement there that it is a challenge in each state. The best I can say is that it really dirties the analysis of a work zone safety assessment because it is tough to get a definition. For that reason, some states have put an effort into limiting the definition to the limits of the lane closure and when workers are present, because that seems to be the one that gets identified correctly. When you get into whether it was a queue that extended a mile upstream of the lane closure that caused a rear-end crash, does that get coded as a work zone crash? A lot of times it doesn't. You have to go in and make manual decisions —look at the project plus a mile on each side for specific dates in order to get at this and almost not use the work zone crash code that most states have in their analysis. It's a much more involved process, but he is absolutely correct.
Jennifer Symoun
Wouldn't we expect more crashes in a work zone than in a normal roadway?
Gerald Ullman
That is absolutely correct. I'm glad you brought that up. I used that example because I know that there are several states that have stated that they want to maintain as part of their policy no increase in crashes in the work zone. I think that's an admirable goal, but it does contradict what we expect when you temporarily reduce the quality of the roadway to get work done. Undoubtedly, you would expect crashes to increase and all the evidence that we have seen in the past indicates that does happen. I didn't go through it because the math wouldn't do you any good on a webinar, but the guidance document has some information on how to modify that graph to use whatever you want to use as a more realistic threshold. If you wanted to say I want to know whenever crashes get more than 20 percent above my baseline, there is a way to generate that. The 50/50 line would be the norm — the before plus 20 percent — but then there's a way to estimate the 75th and 95th percentile values that would indicate that the numbers are more than 20 percent above what was going on before. I would encourage you to get a copy of the guidance document when it's available, and for whatever percent increase you want to assume for your agency, you can do that same kind of analysis.
Jennifer Symoun
I think we are going to move on at this point. There are a few other questions, but I think there are two questions for which I think anybody online could type in their thoughts. Someone is looking for a good definition of what a work zone is. Feel free to type in your thoughts. At this point we'll continue and we can come back to any questions if we have time.
Gerald Ullman
Good deal. We are going to turn it over to Rachel Klein. She is going to move onto the customer satisfaction category of performance measures.
Rachel Klein
When we talk about customer satisfaction, what we're really looking at are the folks who drive through, work in, or live in the areas where there are work zones. The reason that we want to look at this is because any potential delay, congestion, or inconvenience could create issues in maintaining relationships with those said customers. The reason that this is important is because it's those customers who fund and essentially own most of the infrastructure for which State agencies and some contractors are responsible.
We have about five different existing data sources that we're looking at in this guidance document. The first is focus groups. When we talk about focus groups, we're talking about six to ten participants with a trained facilitator, and the participants are going to usually be diverse in terms of socioeconomic background, age, gender, and education. Sometimes they may be part of a particular traveler group if there are certain concerns about that group's opinions. When I say a specific traveler group I may mean commuters or local residents, perhaps business owners or transit bus operators. The value of focus groups is that particular participants can provide their own opinions, experiences, or suggestions, which on some occasions can actually provide information on creating questions for larger surveys. While this information is helpful, it clearly doesn't represent the overall driving population, and essentially the findings are nothing more than anecdotal.
The next data source is in-person or telephone interviews. This is a little bit more qualitative in terms of the sample size. In-person interviews are likely going to be for one particular work zone. The telephone interview is best suited if you want to get information on a large group of work zones in a particular region. An important thing to note here is that for in-person interviews, they are likely going to be happening directly downstream of a work zone, so you might have an interviewer standing in a public area and approaching people who have just passed through the work zone. Something to note here is that as conditions change in a particular work zone, you're going to get different responses. To get a comprehensive understanding of people's feelings, you're going to have to continue the interviews several times over the course of some period of time. In-person interviews should be short, probably three to four questions. People don't want to stand there for a long time. They're probably not going to want to entertain your questions if it takes them a long time. Regardless of whether or not you do in-person or telephone interviews, it will be labor-intensive to administer and there are costs involved with that. Some particular interview topics might be people's feelings on the work zone, things like that. In my next slide I have an example of a South Dakota telephone interview script. In this particular incidence, you have the interviewer asking about people's feelings on the construction and maintenance, whether it's increased, whether it's decreased, whether it's caused delays, what types of delays, and people's tolerance with those delays.
We also have surveys as a data source. You can have mail, email, or web site surveys. With any type of survey, it can be quantitative in nature or qualitative. Most of these surveys are going to have predetermined options, although you may also include a section where folks can include additional comments. One thing that's useful about surveys is that they tend to be statistically significant and you can use them to baseline the effectiveness of your performance measures. The downside is that they may be high in cost, depending on the type of survey you conduct and the sample size. Another thing you want to consider when you decide on the type of survey that you do conduct is if you do something that is web-based, you're going to have a slight selection bias. You're likely to have younger, more educated and tech-savvy users respond. One other thing to consider is that folks who decide to respond are likely going to have a somewhat negative bias in their answers, because as most of us know, when you're opposed to something, you're more likely to respond than if you're satisfied with it.
This is an example from Missouri DOT. I believe this is currently still on their web site. The survey asks about work zone signage, whether or not people feel that they had enough warning, and things of that nature. It's short and to-the-point.
The last data source we have listed is customer complaint database entries. Most agencies have a formal process for lodging complaints, and some agencies have databases for that. The thing that will be most important is the way that database is structured to determine how easily it will be to use complaints in order to monitor your performance. It's important to note that some of the complaints will be clear and easily associated with work zones, where some complaints may be more indirect. An easily understood work zone complaint would be that you closed a ramp or this work zone is difficult to get through, but something that might be more indirect is a complaint over intersection busyness, which is the result of people rerouting themselves because of a work zone activity.
While complaints are often not indicative of overall driver satisfaction in an area, they are very helpful in identifying specific operational and safety problems and may help an agency over time evaluate as those particular complaints are dealt with.
We also have future data sources, although both of these data sources mentioned are being used on a small scale now by agencies in regards to work zones. If you look at social media technologies — most of you are probably familiar with both Facebook and Twitter — right now agencies are those technologies to make their customers aware of current traffic conditions, travel conditions, anything like that. In some cases people are actually lodging surveys on Facebook to ask customers how they feel about work zones. Again, something important to consider is the folks who are going to respond to these surveys are likely going to be younger, more educated, tech savvy customers who are comfortable using the internet and who are familiar with the younger social media tools. Like with traditional surveys, you will have selection bias and you will have the tendency to have negatively skewed responses, so again you want to make sure you have trained survey designers creating the questions to try to reduce or eliminate bias if possible.
One other data source is an online focus group web-based tool. There are a few of these available. Their capabilities vary, but in general some capabilities include the ability for the facilitator to poll the group and to have private chat sessions, and related to what Facebook does is the idea of a "group think" area where folks can present ideas and everyone else in the focus group can comment on them.
I went over most of the trade-offs and key considerations already, but just a few things to think about when you're deciding on what data source you want to use is whether or not it will provide statistically significant findings and whether or not that matters for you. What types of information are you going to need? Do you want information that's on a specific work zone? For example, if you do a one-on-one interview, you can get information as it happens from folks as they pass the work zone. For things like complaints, that might be a better option if you want to get specific operational or safety issues. Something to keep in mind is that again, especially with complaints, people tend to embellish the issues and they are not going to provide information to you when they're content or happy.
Other things you might want to consider when you want to use the future data sources, especially for online focus groups, is that the effectiveness of doing things online versus things in person isn't particularly known. As I mentioned before, you always want to think about using trained survey designers in everything you do to make sure that you don't get any bias.
Lastly, our expert panel identified high priority performance measures, so those would be rating the quality of work zone features that folks see when they are driving through a work zone, be it signs or any other information regarding delays, queues, or other construction activities. The last three bullets are related to satisfaction ratings or complaints. The thing to note about the last three bullets is that people's actual feelings of satisfaction or anger over work zones can be easily manipulated by exogenous factors such as media reports, or for example if some sort of a traffic incident occurs in a work zone and people attribute the congestion to the work zone when in fact it was the traffic incident. There are things we need to consider when dealing with customer satisfaction that are a little more subjective than when we're dealing with other categories that are more objective. Regardless, they're all really important measures to consider when deciding on your work zone performance measures set.
With that, I believe it's back to you Gerry.
Gerald Ullman
Our last category is agency contractor productivity and efficiency measures. This is an area that I think most agencies have been doing, and you can find them on most of their web sites. I think there are some really good dashboards, and a lot of states have those already developed. Emphasis for those has been predominantly just making sure money is being spent adequately, at a good rate, and those kinds of things. This data also has some safety and mobility related value; tracking the speed at which things are getting done or the number of lane closures that are needed relative to what was estimated is valuable because such lane closures typically have an impact on mobility. At the same time, this kind of data also is used very often as a means of stratifying or as indicator data. Knowing what nights lane closures were occurring, knowing what weekends full road closures occurred, and where we had everybody pushed off onto surface streets are all examples of the ways in which this kind of data becomes valuable for safety and mobility performance monitoring and measurement.
Existing data sources include the standard construction management system databases — Site Manager is very common one a lot of agencies use. A few states actually have very detailed lane closure request/approval processes and databases to track those kinds of things. The third category, which we here at TTI have spent quite a bit of time poring over on the last several projects, are the daily project diary notes. These vary in terms of quality and accuracy, but a lot of times of that's where you'll find specific details on everything, from when lane closures were occurring to the date at which a phase change was made and we switched type of barrier and/or whether a barrier was there or not. All of that comes into play.
Going forward, we're seeing more and more efforts to develop mobile data collection. I won't call them apps, but I guess that's the closest thing I can come up with. These are developed for agencies and contractors to more closely monitor their work from a productivity standpoint just for the bottom dollar, but at the same time it is a source of data that will allow us to track things that have been difficult to date to get at; the dates of major changes in traffic control, the speed at which paving is getting done relative to what we thought and/or what other contractors do, and those kinds of things all will be available. I think it has a very value as a potential source.
I don't know how many states have detailed electronic maintenance work databases. Several years ago, it was rather limited, though it may have increased in recent years. They track their in-house State force maintenance activities and not just payroll, but actual locations, times, number of vehicles etc. They do it for tracking charge accounts and that kind of stuff, but it serves as a way how to track how often maintenance activities are being done, and obviously that can be tied back to other data sources, for example if you wanted to assess mobility on a section where you had traffic surveillance and wanted to know which disruptions were due to maintenance, which were due to incidents, etc.
The key characteristics listed here are fairly straightforward. Most everything we're talking about is related to contract data. Right now, we're getting at stuff that relates to safety and mobility. A lot of times it's the narratives and the project diaries that we consider tough to get, but opportunities do exist to operationalize those to get at them more easily in the future — with some of the future applications we mentioned, for example.
Three key things that our practitioner panel saw as valuable for monitoring and measurement of agency and contractor productivity were percentage of days, either the allowable or the total depending on which way your contract is written; percentage of lane closure hours that occur outside your allowable or identified acceptable "work windows" (hopefully those are very small percentages); and general production rates. All those are valuable and ultimately useful for safety and mobility purposes in addition to just tracking contractor and agency productivity.
That concludes what we wanted to present here. These are the resources. The FHWA Office of Operations web site is listed here if you want to jot that down. Go to FHWA, go to the Operations Group, and then there's work zone management in that group. As soon as the guidance document is published, Jawad will let you know that it's available and how to get it. With that, do we have time for questions?
Jennifer Symoun
We are actually out of time. I think there are only two other questions, so if everybody has time to hang on a few extra minutes we can get through those. Have any States established baseline crash rates for differing work zones (urban vs. rural, etc.)? We are having difficulty establishing what constitutes and normal work zone crash rate and therefore being able to determine what is a high rate.
Gerald Ullman
I am not familiar with any States that have done that yet. I think that is something that would occur as a natural outcome of folks moving towards performance measurement. By tracking work zones year after year you could figure out what the baseline rates are, or at the very least what the increases are that we could expect for different types of facilities. I don't know that I'm aware of states that have done that thoroughly yet, unfortunately.
Jennifer Symoun
Is an agency supplying its inspectors or resident engineers with laptops and apps to record work zone performance measures?
Gerald Ullman
We have not found anybody who has done that yet, not to say that it's not a good idea or that it's not something that could happen down the road. I do know that some states are moving towards getting their folks tablets, predominantly for field testing and for being able to enter slump test results and those types of things in the field. It could certainly translate very quickly to some of these performance metrics, it just hasn't happened yet that I'm aware of.
Jennifer Symoun
Were winter and summer work zones considered?
Gerald Ullman
We didn't consider time of year separately in the guidance. We're not familiar with too many ways that data collection would vary whether it was a winter or summer work zone. I don't think I have a good answer for that, unfortunately.
Jennifer Symoun
We are out of time, so we will end now. I want to thank everybody for attending and I want to thank all of our presenters as well. The recording and presentation from today's webinar will be available online in a few weeks, and I will send out an email to all who registered once they are available. Thank you everybody and enjoy the rest of your day.
