Office of Operations
21st Century Operations Using 21st Century Technologies

Freight Performance Measure Primer

APPENDIX C. FREQUENTLY USED FREIGHT DATA

TRUCK PROBE DATA

The Federal Highway Administration (FHWA) has been a leader in developing freight data and performance metrics. Since 2002, FHWA has worked with private sector trucking operations to collect data through a trusted third party, the American Transportation Research Institute (ATRI.) What began as the collection and analysis of Global Positioning System (GPS) signals or probes for a limited set of trucks on only a few highways has become a robust data source of probes collected from over 600,000 trucks all across the United States. This information has been used since its inception to form the FHWA Freight Performance Measurement program (FPM). Analysis of probes has been used to understand speeds and reliability on the Nation's highways, to identify bottlenecks, and to support State- and regional-level analyses of truck movements with the goal of improving freight performance. FHWA has learned a great deal from evaluating the probes and is continuously finding new ways to develop the data and use it in innovative analytical methods. Use of these data has produced many successful analyses of freight movement, both in continuous monitoring and in incident analysis. The information revealed by the data has helped numerous decision-makers determine both operation and capital improvements for truck movements. FPM gathers information only on truck movements at this time. FHWA is working with other modal operating administrations to attempt to build probes for other modes and to evaluate freight movement from a multi-modal perspective.

While FHWA's FPM probe data are available to FHWA, FHWA will provide access to the data to support innovative research conducted by States, Metropolitan Planning Organizations (MPOs), regional authorities, and academics. States and MPOs may also acquire the data directly for their own needs from the data provider. Additionally, it is important to note that these data provides unique identifiers only for the truck probe, latitude, longitude, speed, and heading. There is no information about the truck type, truck owner, fleet size, commodities, or other characteristics. FHWA is not aware of any data source that currently provides truck probes and commodity information at this time, and is working with public and private stakeholders to improve on the data to obtain more detail in the future that could support freight analysis. Analyses using the FHWA FPM probe data are publicly available on FHWA's website. FHWA uses the probe data to routinely measure the following:

  • Speed and reliability of the top 25 domestic freight corridors.
  • Freight Efficiency Index, which consists of the following components:
    • Measures of known bottlenecks.
    • Metropolitan area speeds.
    • Intermodal and port locations.
    • Border crossings.

In addition to the routine measures, FHWA has supported a number of freight analyses with the probe data that are published on the FHWA website. Most of the resources are located at http://ops.fhwa.dot.gov/Freight/freight_analysis/perform_meas/index.htm.

NATIONAL PERFORMANCE MANAGEMENT RESEARCH DATA SET

Though FHWA does not provide external entities with access to this truck probe data, FHWA does make available a derivative of the data in the form of travel times to States and MPOs for free. The National Performance Management Research Data Set (NPMRDS), or vehicle probe data, is a historical archive of average travel times measured in five-minute increments. This data set covers the entire National Highway System (NHS) and is available on a monthly basis, with each month's data usually available by the middle of the following month. NPMRDS data covers all 50 States, Washington, D.C., and Puerto Rico. It also covers crossings into both Canada and Mexico. The data set contains three distinct travel times: freight, passenger, and a combination of all traffic. NPMRDS data are divided into States and regions, making it easy for States to conduct corridor analyses.

There are a number of potential uses for the NPMRDS. States and MPOs are encouraged to use the NPMRDS data to produce system performance measures that cover operational and freight management strategies. NPMRDS can also be used to produce performance measures that will allow a more efficient highway system to improve the movement of freight. NPMRDS can be used to produce measures that will better inform States of where congestion is regularly occurring or how congestion occurs when there is an incident or work zone operation. This information helps them to identify where operational improvements or long-term investments can be made.

FHWA makes the NPMRDS available to State departments of transportation (DOTs), MPOs, and Federal agencies. States and MPOs are required to sign a licensing agreement with the NPMRDS data provider. More information on the NPMRDS, including webinars and reference documents, is located on FHWA's website under the vehicle probe data section at http://www.ops.fhwa.dot.gov/freight/freight_analysis/perform_meas/index.htm.

COMMODITY FLOW SURVEY

The Commodity Flow Survey (CFS) is the primary source of national and State-level data on domestic freight shipments by American establishments in mining, manufacturing, wholesale, auxiliaries, and selected retail and services trade industries. Data are provided on the types, origins and destinations, values, weights, modes of transport, distance shipped, and ton-miles of commodities shipped. The CFS is a shipper-based survey and is conducted every five-years as part of the economic census. It provides a modal picture of national freight flows and is the only publicly available source of commodity flow data for the highway mode. The CFS was conducted in 1993, 1997, 2002, 2007, and most recently in 2012.

FREIGHT ANALYSIS FRAMEWORK

The Freight Analysis Framework (FAF), produced through a partnership between the Bureau of Transportation Statistics (BTS) and FHWA, integrates data from a variety of sources to create a comprehensive picture of freight movement among States and major metropolitan areas by all modes of transportation. Starting with data from the 2012 CFS and international trade data from the Census Bureau, FAF incorporates data from agriculture, extraction, utility, construction, service, and other sectors. The FAF version 4 (FAF4) baseline edition provides estimates for tonnage and value by regions of origin and destination, commodity type, and mode for 2012, the most recent CFS year. Data are available through the Data Extraction Tool, for download as a complete database, as well as summary files. Throughout 2016, releases of additional FAF4 products have provided forecasts through 2045; State-to-State flows for 1997, 2002, and 2007; truck flows assigned to the highway network for 2012 and 2045; and domestic ton-miles and distance bands.

This compilation of data creates a comprehensive picture of freight flows, trends, and a baseline forecast to support analysis and performance for all modes of transportation among States and major metropolitan areas. FAF forecasts are a reasonable estimation of current trends; however, they do not reflect major shifts in the national economy, future capacity limitations, or changes in transportation costs or technology. FAF data would need to be supplemented with additional models to identify such trends.

FHWA has an online tabulation tool that is useful for States and MPOs who want to understand what freight, how much, and by what mode is occurring in their area. The tool is available at http://faf.ornl.gov/fafweb/Extraction0.aspx.

COMMODITY FLOW SURVEY MICRODATA

The first generation 2012 Commodity Flow Survey (CFS) Public Use Microdata (PUM) file is now available. This new, experimental data product contains information for approximately 4.5 million shipments obtained from businesses included in the 2012 CFS. The PUM file provides access to shipment-level characteristics while continuing to protect the confidentiality of individual business information. PUM file users can create customized tables and build models to track and analyze the movement of commodities. (http://www.rita.dot.gov/bts/sites/rita.dot.gov. bts/files/publications/commodity_flow_survey/index.html)

HIGHWAY PERFORMANCE MONITORING SYSTEM (HPMS)

The Highway Performance Monitoring System (HPMS) is a national level highway information system provided to FHWA by State DOTs. The HPMS data are used extensively in the analysis of highway system conditions, performance, and investment needs that make up the biennial Condition and Performance Reports to Congress. The HPMS contains administrative and system information on all public roads. Extensive data are available for interstates and principal arterials, while a mix of sample data is reported for minor arterial and collector functional systems. Limited information on travel and paved miles is included in summary form for the local functional systems.

The HPMS was developed in 1978 as a continuing database and has been modified several times since its inception. The system has been altered to reflect changes in the highway systems, legislation, national priorities, and new technology, and to consolidate or streamline reporting requirements.

For freight, States and MPOs often use HPMS data to derive Annual Average Daily Truck Traffic (AADTT) counts, Percent Peak Combination Trucks, or other truck count information for roads included in the HPMS data. FAF uses HPMS in its network assignments for freight flows. (https://www.fhwa.dot.gov/policyinformation/hpms.cfm)

HIGHWAY STATISTICS

FHWA's Highway Statistics Series consists of annual reports containing analyzed statistical information on motor fuel, motor vehicle registrations, driver licenses, highway user taxation, highway mileage, travel, and highway finance. This information is presented in tables and charts and has been published annually since 1945. Most highway data are submitted by the States directly to FHWA. Each State's data are analyzed for completeness, reasonableness, consistency, and compliance with data reporting instructions contained in "A Guide to Reporting Highway Statistics." While the FHWA Office of Highway Policy Information is responsible for preparing this publication, a number of the statistical summaries are prepared by other units within the FHWA. More information is available at http://www.fhwa.dot.gov/policyinformation/statistics.cfm.

MARITIME DATA

The U.S. Maritime Administration (MARAD) provides a number of data resources through the MARAD Open Data Portal and link to Maritime Data and Statistics. Below is a list of the available resources.

Reports


  • 2013 Vessel Calls in U.S. Ports and Terminals (Uploaded 6/17/2015).
  • 2012 Vessel Calls in U.S. Ports, Terminals and Lightering Areas Report (Released 10/8/2014).
  • 2002–2012 Vessel Calls in U.S. Ports, Terminals, and Lightering Areas Report (Released 10/8/2014).
  • 2011 Coastal Tank Vessel Market Snapshot.
  • 2011 U.S. Water Transportation Statistical Snapshot.
  • 2011 Vessel Calls at U.S. Ports Snapshot.
  • Guide Market Research for Marine Transportation Services.

Fleet Statistics


  • U.S.-Flag Privately Owned Fleet (As of 4/1/2016).
  • U.S.-Flag Integrated and Articulated Tug-Barge Units.
  • Merchant Fleets of the World Report for 2015 (As of January 2015).
  • Top 25 Flags of Registry.
  • 2000–2014 U.S.-Flag Privately Owned Fleet Summary.
  • 1946–2014 U.S.-Flag Privately Owned Fleet.

MARAD U.S.-Flag Fleet Lists


  • Listing of U.S.-Flag Carriers.
  • Maritime Security Program List.
  • Agricultural Preference Trade Fleet List.

Historic Fleet Reports and Fleet Lists

MARAD has electronically imaged some historic Vessel Inventory Reports on the U.S.-Flag Fleet. The Vessel Inventory Reports from July 1990 to January 2016 are available for research purposes.

Trade Statistics


  • U.S. Waterborne Foreign Trade by U.S. Customs Ports 2000–2015.
  • U.S. Waterborne Foreign Trade by U.S. Customs Ports 1997–1999.
  • U.S. Waterborne Foreign Trade by U.S. Custom Districts.
  • U.S. Waterborne Foreign Trade by Trading Partners.
  • U.S. Waterborne Foreign Container Trade by U.S. Customs Ports 1997–2014 Report.
  • U.S. Waterborne Foreign Container Trade by Trading Partners.

Historical Trade Statistics


  • Containerized Cargo on Selected Trade Routes CY 1970.
  • Containerized Cargo Statistics CY 1971 (unavailable).
  • Containerized Cargo Statistics CY 1972–1983.

OPA-90 Related Statistics


  • Coastal Tank Vessel Market Snapshot.
  • Tanker Calls at U.S. Ports.
  • U.S.-Flag Oil Pollution Act of 1990 (OPA-90) Phase-Out.
  • U.S.-Flag Tank Vessels Removed.
  • U.S. Tank Vessel Trades.

Survey Series


  • Mainstream Container Services.
  • Coastal Tank Barges.
  • Deck Barges.
  • Inland Tank Barges.

MARINE TRANSPORTATION SYSTEM DATA INVENTORY


U.S. ARMY CORPS OF ENGINEERS NAVIGATION DATA CENTER

The U.S. Waterway Data is a collection related to the navigable waters in the United States, including inland waterways, off-shore waters, the Great Lakes, and the Saint Lawrence Seaway. Data on commerce, facilities, locks, dredging, imports and exports, and accidents are included, along with the geographic waterway network. The data were compiled by several agencies, including the U.S. Army Corps of Engineers Navigation Data Center, the U.S. Bureau of the Census, the U.S. Coast Guard, Oak Ridge National Laboratory, and Vanderbilt University. One of the objectives of this coordinated effort is to make waterway data more widely available and easily accessible. The data included are in standard file formats that can be easily imported into other software tools such as spreadsheets, databases, and Geographic Information Systems (GIS).

The Navigation Data Center had several objectives in developing the U.S. Waterway Data. These objectives support the concept of a National Spatial Data Infrastructure and will include:

  • Providing public access to national waterway data.
  • Fostering interagency and intra-agency cooperation through data sharing.
  • Providing a mechanism to integrate waterway data (U.S. Army Corps of Engineers Port/Facility and U.S. Coast Guard Accident Data, for example).
  • Establishing a basis for intermodal analysis.
  • Assisting standardization of waterway entity definitions (ports/facilities, locks).
  • Supplying public access to the National Waterway Network, which can be used as a base map to support graphical overlays and analysis with other spatial data (waterway and modal network/facility databases, for example).
  • Making available reliable data to support future waterway and intermodal applications (http://www.navigationdatacenter.us/data/data1.htm)

PIPELINE DATA


Oil Pipeline Statistics

The U.S. Energy Information Administration (EIA) publishes statistics on crude oil, gasoline, diesel, propane, jet fuel, ethanol, and other liquid fuels. Information collected includes petroleum prices, crude reserves and production, refining and processing, imports/exports, stocks, and consumption/sales. (http://www.eia.gov/dnav/pet/pet_move_pipe_dc_R20-R10_mbbl_m.htm)

Pipeline Safety Statistics

The Pipeline and Hazardous Materials Safety Administration's (PHMSA's) Office of Pipeline Safety provides a variety of data about federally regulated and State-regulated natural gas pipelines, hazardous liquid pipelines, and liquefied natural gas plants. The operators of these pipeline facilities report this data in accordance with Part 191 and Part 195 of PHMSA's pipeline safety regulations. PHMSA provides downloads of the raw data, yearly summaries, multi-year trends of safety performance metrics, and inventories tracking the removal of aging and other higher-risk infrastructures. (http://www.phmsa.dot.gov/pipeline/library/data-stats)

RAILROAD DATA


Surface Transportation Board Rail Waybill Sample

The Rail Waybill Sample is a stratified example of carload waybills for all U.S. rail traffic submitted by rail carriers terminating 4,500 or more revenue carloads annually. Because the Waybill Sample contains sensitive shipping and revenue information, access to this information is restricted. Access is granted when the Waybill Sample is the only source of the data, obtaining the data from other sources is burdensome or costly, and/or the data are relevant to issues pending before the State Transportation Board (STB). Waybill data are used by transportation practitioners, consultants, and law firms with formal proceedings before the STB or other State boards. There is also a group designated as "other users." To obtain the Waybill Sample, the requester must submit a written request explaining the need for the data and how it will be used.

There is a Public Use Waybill File. The Public Use Waybill File is created from the confidential Waybill Sample File and is a non-proprietary version of the Waybill Sample that can be used to support freight analysis. (http://www.stb.dot.gov/stb/industry/econ_waybill.html)

Association of American Railroads

The U.S. freight rail network consists of nearly 140,000 miles of rail and is operated by over 560 different railroads. Freight railroads are predominantly privately owned and operate almost exclusively on tracks owned and maintained by the private owners.

The Association of American Railroads AAR compiles and distributes information on North American freight railroads, including finances, operations, performance, input cost indexes, traffic, U.S. carloads, intermodal traffic, Class 1 rail tons, and more. The AAR can supply data on safety statistics, hazardous materials, and grade crossings as well. Data is collected weekly, monthly, and annually depending on the data required (AAR website 2015). (https://www.aar.org/pages/freight-rail-traffic-data.aspx)

Railroad Performance Measures

The Railroad Performance Measures website is where six major North American freight railroads have voluntarily reported three weekly performance measures—Cars On Line, Train Speed, and Terminal Dwell—since 1999, in a commitment to improve communications with their customers. The six Class I railroads are shown in Table 9.


Table 9. Class 1 Railroads.
Burlington Northern and Santa Fe (BNSF Kansas City Southern
Canadian National (CN) Norfolk Southern
Chessie and Seaboard (CSX) Union Pacific

All six railroads use the same definitions to calculate their performance data; however, because the level of one railroad's performance relative to another's may differ sharply because each railroad is unique, these measures are most useful for examining trends and relative changes, and least useful as absolutes. Differences between railroads include terrain, physical routes and network design, traffic mix and volume, the extent of passenger operations, and operational practices. External factors such as weather and port operations can also cause variations between railroads over time. In addition, individual differences in the collection and reporting of operational data may affect the absolute level of the measures to some degree. (http://www.railroadpm.org)

AVIATION DATA

Air Cargo Summary Data

This monthly freight summary includes both freight and mail carried by U.S. airlines in all service classes (scheduled and non-scheduled). Data in this report goes back to October 2002, when a change in reporting requirements increased the number of reporting carriers and collection of monthly all-cargo service data started. (https://www.transtats.bts.gov/freight.asp)

Landing Weights

The Bureau of Transportation Statistics provides information on landing weights for cargo- bearing airports throughout the United States. Data is available through the BTS Summary data. (https://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/publications/state_transportation_statistics/summary/index.html)

BORDER CROSSING DATA


Transborder Freight Data

The North American Transborder Freight Database, available since April 1993, contains freight flow data by commodity type and by mode of transportation (rail, truck, pipeline, air, vessel, and other) for U.S. exports to, and imports from, Canada and Mexico. The database includes two sets of tables; one is commodity-based while the other provides geographic detail. The purpose of the database is to provide transportation information on North American trade flows. This type of information is being used to monitor freight flows and changes to these since the signing of the North American Free Trade Agreement (NAFTA) by the United States, Canada, and Mexico in December 1992, and its entry into force on January 1, 1994. The database is also being used for trade corridor studies, transportation infrastructure planning, marketing and logistics plans, and other purposes. It allows users to analyze movement of merchandise by all land modes, waterborne vessels, and by air carriers. (http://transborder.bts.gov/programs/international/transborder/)

Freight Performance Measurement Border Data

As previously mentioned, the Freight Performance Measurement (FPM) program collects information on select border crossings for North America using truck probes. The probe data, collected by FPM and the NPMRDS, can be used to analyze truck movements and travel times, respectively, for border crossings. The NPMRDS extends into the five-mile radius of arterials in Canada and Mexico.

Travel Time Index Border Data

The Texas A&M Transportation Institute maintains the Border Crossing Information System, which computes average travel times of U.S.-bound commercial vehicles for the selected segment over the selected time period, calculated at 10-minute intervals. (http://bcis.tamu.edu/Commercial/en-US/queryArchivedData.aspx)

GENERAL AND ECONOMIC FREIGHT DATA RESOURCES

The following freight data resource links are provided as additional resources that can be consulted for freight data and analysis. While most analysts use the above data sources for freight performance measures and the Commodity Flow Survey or FAF for economic information, the following could supplement analysis, depending on the objective. Although not detailed in this primer, the interested reader may wish to consult these sources:

Economics:


Classification Systems Used for Freight Data:


Office of Operations