Freight Performance Measure Primer

CHAPTER 5. CHALLENGES WITH FREIGHT PERFORMANCE ANALYSIS

Federal, State, and Metropolitan Planning Organization (MPO) practitioners regularly cite challenges in analyzing freight data. These challenges, which are impediments to measuring freight, include setting goals and understanding the State or MPO role in freight planning and funding, not having multi-modal data, inconsistency in data quantity and quality, and not having reliable freight models. The following section describes these challenges in more detail. This primer aims to help solve these issues by providing information on the most commonly used data and approaches to measurement from which practitioners can work to grow the state of practice for freight performance measures.

SETTING FREIGHT GOALS

The identification of goals is the first step in bringing about change and improvements. Goals are the desired results—the purpose of a program. Objectives are the methods by which the goals will be achieved. Performance measures help agencies evaluate if they are being successful in achieving the established goals and objectives. By understanding what the result should be, agencies can better select improvement projects or programs that will help the agency achieve its goals.

Differences between freight transportation and passenger transportation present challenges to setting goals and defining objectives. Freight is most often carried by private, for-profit companies, which may have different goals than government agencies. Private companies focus on such things as profitability, competitiveness, and customer retention.

Government agencies focus on outcomes that positively impact the public good such as increasing safety for the traveling public and reducing congestion and air pollution. To understand the differences between public and private goals, transportation agencies should gather input from stakeholders with different perspectives to find shared interests that benefit one another. The Florida Department of Transportation's (FDOT's) outreach for the Freight Mobility and Trade Plan is a good example and can be found at http://freightmovesflorida.com/wp-content/uploads/2016/11/FMTP-Investment-Element_2014-09-11.pdf.

LACK OF CONSISTENCY

Freight stakeholders and practitioners continue to investigate how best to measure freight performance. Transportation agencies (State departments of transportation [DOTs], MPOs, and local governments) have mostly developed their own freight performance measures based on existing data available and their agency's goals.

There is currently no standard set of universally accepted freight performance measures. In addition to the inconsistencies in performance measures between different agencies and States, there can also be inconsistency between data sets that support freight performance measures. Cities, counties, regions, or States may all collect similar data but may do so using different methodologies. This makes comparisons of freight performance between them difficult.

ABUNDANCE OF INCOMPLETE DATA

There is a large amount of data available that can be used to develop performance measures. Data on infrastructure conditions and general travel data aggregated from all modes is widely available. Data for measuring freight performance is much more scarce. Many transportation agencies use general performance on key freight corridors as a proxy for measuring the movement of goods. The partnership between the Federal Highway Administration (FHWA) and the American Transportation Research Institute (ATRI), which is providing truck volume and speed data collected through a sample of Global Positioning System (GPS) transmissions, is making additional freight data available. However, even this data is incomplete. This data sample is drawn mostly from large national trucking firms and independent truckers. Recent trends indicate the larger companies are handing off deliveries to smaller delivery companies, which could mean the data is less reliable for important last-mile connections, as these companies are not tracked by ATRI data.

FREIGHT MODELING CAPABILITIES AT THE REGIONAL SCALE

Another challenge is the inability to accurately model the effects of various transportation improvements on freight performance. Many regions have truck forecasting models which were developed from a patchwork of data. These models do not typically cover all modes of freight and are currently incapable of modeling how different improvements or changing economic, demographic, or land use conditions might change freight mode share. There have been, however, some recent improvements in this area. For example, the Tampa Bay Regional Planning Model has a freight component improved by FDOT District 7 to better estimate truck travel. However, freight transportation can cover very long distances. In these cases, even a strong regional truck model may not accurately capture freight that is passing through the region.

APPLYING THE MEASURES

The 2011 the National Cooperative Freight Research Program (NCFRP) Report 10, "Performance Measures for Freight Transportation," noted most of the agencies that reported using freight performance measures did not rely on these measures to guide decision-making. The freight performance measures were used as indicators of how freight (typically truck freight) was moving through the region, but the measures did not influence the decision-making processes or provide the impetus to amend agency goals or objectives.

DATA CAPTURE

It is important to have common definitions of freight performance measures to ensure consistency and integrity. Several common terms regarding freight data, such as "merging data," "governing data," "quality of freight-related data," and "quantity of freight-related data" are not always clearly understood. Definitions for these concepts are included in the glossary in Appendix E. Terminology referring to freight performance measures data should always be examined and understood before credence is given to any trends identified in performance measure analysis.

The development of performance measures typically include difficult tradeoffs between their predictive or descriptive usefulness, the cost and time involved in acquiring data, and the ability for citizens and elected officials to easily understand the measure's meaning.

INTEGRATING DATA

The FHWA defines data integration as "the process of combining or linking two or more data sets from different sources to facilitate data sharing, promote effective data gathering and analysis, and support overall information management activities in an organization (Transportation Research Board [TRB], NCFRP 2011).

The "Freight Performance Measure Approaches for Bottlenecks, Arterials, and Linking Volumes to Congestion" Report (Cambridge Systematics 2015) noted that methodologies for producing performance measures from data are very similar, but would benefit from consistent/standardized processing procedures. Developing high-level performance measures from low-level data requires multiple processing steps, and there are usually multiple ways to perform each step. Default values also are often necessary, and these can vary depending on the methodology. The result is that different values can result from processing the same basic data.

SUSTAINING FREIGHT PERFORMANCE MEASURES

Much of the literature on performance measures and the experience of the practitioners interviewed for the NCFRP Report 10 indicated that performance measuring systems tend to mature and improve over time. Input from stakeholders indicated that of the agencies that had comprehensive measurement systems, few had those systems from the beginning.

"Begin with what you have" was a common recommendation from the performance management practitioners interviewed for the report. It was also acknowledged that there is no one proposed reight performance measures system that will meet the needs of all stakeholders. Thus, the practitioners predicted that stakeholders would advocate for additional measures that could be added over time. A perceived benefit of publishing the "Freight System Report Card" was that flaws in the current data would be uncovered as the report card was examined, which should result in improvement of the data.