Practices for Improving the Coordination of Information Technology and Transportation Systems Management and Operations Resources: A Reference Document

Appendix A. Common Challenges in Coordinating TSMO and Information Technology

This appendix describes the common challenges experienced by Transportation Systems Management and Operations (TSMO) and Information Technology (IT) groups working together within a transportation agency that were identified through the project research. The individual challenges are categorized within functional groups based on common issues and all groupings are further categorized into two large classes: institutional challenges and business and technical challenges. Where feasible, specific agency examples are provided for individual challenges based on interviews and outreach efforts.

A.1. Institutional Challenges

The interaction between TSMO staff and information technology (IT) staff is often influenced by the organization itself, including professional culture, organizational structure, staff capabilities, and resource allocation. Challenges can arise when the organization’s history, the underlying culture of its staff, or the available resources do not align with the current mission or task. Common institutional challenges can be encountered at all levels of the organization from leadership decision-making to frontline implementation.

A.1.1. Culture

This subsection discusses common challenges related to professional culture. Culture consists of the values and behaviors that influence personal relationships or interactions between the differing agency functions or groups.

Culture-1: Lack of Mutual Understanding between TSMO and IT Staff

TSMO units and IT units, like many areas within a transportation agency, are separate not only in function but also in location. The physical and functional separation can limit interaction and ultimately limit the understanding and familiarity of the other unit. Staff shortages in some agencies can further distance the groups by reducing the available time to meet and coordinate. While several of these contributing factors are addressed individually in other challenge areas, the resulting lack of mutual understanding is a common challenge that inhibits progress.

During normal business, TSMO staff is focused on optimizing traffic operations and ensuring traffic safety. These functions are external in nature and employ various technology platforms to monitor, manage, and communicate with drivers. In delivering these functions, TSMO staff may perceive IT staff strictly as internal technical support who may be able to quickly assess and resolve any technical shortcomings during crises. TSMO staff may not understand the backlog of other agency IT functions or the complexity involved in resolving a malfunctioning technology system.

Conversely, IT staff is responsible for building and maintaining a vast array of business platforms (hardware and software) across many functional areas within an agency. While IT staff is focused on delivering service to their agency customers, they may not appreciate the urgency involved in TSMO functions. TSMO functions are directly linked to public safety, and an IT malfunction may have significant impacts on vital traffic management functions. IT staff may not understand the public safety ramifications of TSMO-related systems being inoperative for an extended timeframe.

As an example, the research identified one agency in which the priorities between the State department of transportation (DOT) and their central IT group were not always aligned on project implementation and maintenance needs of new technology. While the overall lack of understanding in this case was largely attributed to staffing constraints and a shortage of time to coordinate, there was a strong recognition and desire to increase mutual understanding.

Culture-2: Lack of IT Staff Availability when TSMO Staff Need IT Support

TSMO functions, by definition, center around the management and operations of transportation systems ranging from freeways to arterials to transit. These environments operate 24 hours per day and (typically) do not have downtime. As such, the underlying systems that monitor and manage the operations can also be available and function 24 hours per day. Agencies have found that any issues with these systems need to be resolved expeditiously to avoid potential issues with traffic safety or efficiency across major transportation infrastructure.

Traditional agency staff, including most IT units, work during typical business hours, Monday through Friday. This traditional schedule may be satisfactory for many agency functions that have long-lead projects, but a strict adherence to this schedule may limit support during evening or night and weekend hours when TSMO functions are equally critical as normal weekday hours.

TSMO functions are also continuous operations rather than stand-alone projects. As such, agencies have found that systems requiring updates or repair need to minimize downtime to the extent possible. While planning these upgrades can be possible, some updates or repairs may be unplanned and urgent with little to no notice. An orderly response to providing support and resources to these unplanned issues may not be feasible through a ticketing method or first-in, first-out approach.

As an example, one TSMO group that participated in the agency outreach effort of this project highlighted the fact that their Statewide IT unit was not accustomed to supporting a real-time 24/7/365 business such as TSMO, which operates continuously. This lack of available support was especially evident during times of regional or Statewide weather events and on State holidays.

Culture-3: Different Staff Backgrounds and Roles Lead to Silos

TSMO and IT staff typically come from very different backgrounds and career paths. Both units are diverse and include many different staffing roles, but in general, TSMO staff are traffic engineers or technicians, while IT staff are computer or systems engineers.

Traditional education and career paths for TSMO staff are focused on civil engineering or the built environment. This perspective on public infrastructure is highlighted through delivering safety and efficiency in physical resources. Within TSMO, the focus is more directed at safe and efficient operations of the transportation system.

Traditional education and career paths for IT staff are focused on computer science and information management. This includes computer and network equipment, software development, and database management that are scalable and replicable. With further emerging technology services and applications, many of these systems can also be virtual and located remotely.

These differing backgrounds and skillsets can result in silos when the IT and TSMO functions overlap. These silos can lead to inefficiencies or solutions that do not take full advantage of available technology. One example is that TSMO staff often develop communications network to transfer data and video to and from the field devices back to a central hub. This can lead to inefficiencies in overall network operation. IT staff is more likely to develop the network from the “top down” taking advantage of the best centralized equipment for the network and then building to the field equipment. Middle switches and edge switches can be matched with the latest central equipment.

Culture-4: Shared Resources and References for TSMO Related IT Needs Are Scarce

Technology has served the TSMO sector for several decades but has undergone drastic changes in the recent past. The rate of change in available types and volume of available TSMO data and the systems needed to collect and analyze this data are increasing. Agencies have responded to these new technologies based on their individual capacities and internal lessons learned.

There are few national resources or reference guides available to assist agencies in navigating the IT requirements for TSMO. One reason may be the lack of an organizational body dedicated to research and practice within the overlap of both TSMO and IT. Another reason may be the speed at which technology is changing compared to the time required to publish material.

The limited amount of shared knowledge often leads agencies to repeat approaches taken by others and learn similar lessons without the awareness of the commonalities. The lack of reference material can also slow the adoption of newer approaches that may increase depth and breadth of services.

A.1.2. Staff and Financial Resources

This subsection discusses common challenges associated with staff and financial resources. Staffing and financial resources are essential components to fulfilling TSMO and IT needs and are required for nearly all aspects of acquisition, development, and maintenance of IT systems.

Resource-1: Funding Sources for IT-TSMO Projects Are Not Well Established

While TSMO as a stand-alone function or group has gained significant traction in recent years, funding sources within State DOTs have not kept pace. Many established TSMO groups have dedicated funding for staff, operations, and maintenance of existing features but encounter difficulties identifying funding sources for new projects or efforts.

Agency programming and funding allocations have traditionally been centered around infrastructure and capital investments. Most projects—whether planning, design, or maintenance—have an element that supports constructing or rehabilitating “external” physical assets, and TSMO intelligent transportation systems (ITS) field elements can typically fit within these parameters. However, ongoing funding for maintenance and operations of the new features is often not included in these programs, and other sources are needed for future sustainability.

TSMO efforts that predominately center on software, data, networking, or other “internal” technology platforms often encounter constraints in funding. Significant portions of transportation funds are strictly allocated within an agency’s program. If allowable funding is identified, it is typically not dedicated or recurring and TSMO efforts will likely have to compete against other priorities.

As an example, the research identified one agency in which securing funding for TSMO technology projects was problematic. The justification for funds was particularly difficult for this group as the policy-makers did not recognize the importance of the IT elements within TSMO projects. IT elements are generally hidden components that may not be readily visible. This type of problem can apply specifically to expenditures to upgrade communication networks and replace network devices, operator workstations and video walls, and a variety of maintenance equipment to enhance network or device troubleshooting.

Resource-2: Recruiting and Retaining IT Staff is Difficult with Competition from the Private Sector

IT has become integral to nearly all aspects of the modern world. Data, computing, communication, and advanced analytics are commonplace in most industries, and the demand for talent has followed. Professionals who specialize in IT are highly sought after, and demand is forecast to remain high.

Public agencies that rely on IT professionals compete with private industry, which has a different and competing business model for staff. Private companies, operating on behalf of shareholders or owners, typically offer higher salaries, bonuses, and more flexibility, both in terms of hours and location. Public agencies, operating on behalf of the taxpayers, are typically constrained in terms of salaries, location, and hours.

For many agencies, retaining IT staff is as challenging as hiring. The IT industry is ever evolving, and external opportunities can entice staff if they do not see future opportunities to grow internally. Like any industry, staff turnover can also occur once the individuals have been trained and gain experience.

As an example, one TSMO group that participated in the agency outreach effort of this project highlighted the fact that staffing was their top priority. Recruiting for this group had been an issue due to pay levels not being competitive with the market and the restrictions created by the position classifications. Due to the poor pay structure, it was common for staff to leave after a few years once they obtained training and marketable technology skills.

Resource-3: Training Requirements and Programs for TSMO Staff and IT Staff Are Not Typically Provided Internally

Agency-wide training programs mostly focus on general administrative and safety protocols such as building safety (fire, etc.), working within public right-of-way, computer basics, and similar topics. These training programs are developed to ensure basic understanding across large numbers of staff and to minimize agency risk. Specialized internal training is typically developed for the largest contingencies and common practices—for transportation agencies, these may include construction practices, traffic control, and maintenance protocol.

TSMO and IT are both typically smaller units within a transportation agency and established training programs are not common. External training options are available, but there are few standards on what an agency would require, and there may not be requirements established by position (knowledge, skills, abilities [KSA]). In addition, registering for external training typically requires initiation by the individual or manager and approval for funding. Because these trainings are not required, requests may require special justification and may not be supported on a wide scale (or may be considered personal education and the responsibility of the individual).

The breadth of topics within TSMO and IT are significant and not necessarily standardized across either industry. Staff looking for external training may have difficulty finding training programs specific to the public agency environment or their role. It is more common for staff to select specific training related to a task or project as the need arises.

Resource-4: Use of Private Firms to Provide Services Requires Funding, Long-Range Planning, Strict Oversight, and Approval

As agencies consider options to adequately support IT needs, some have explored or implemented external services to bolster resources. The reasons to employ private services can vary, but agencies may find that they need an increase in the knowledge base, manpower (number of staff), or availability (specific timeframe or temporary assignment). Other reasons to outsource may be directed by political or funding constraints.

The use of private firms can be beneficial but are typically more expensive on a per-hour basis. In addition to the direct costs of the external staff, indirect costs—such as oversight for contracting, invoicing, supervision, and quality control—are often incurred. These costs require a funding source that can support the additional staff for the duration of the need, whether short term or long term. Inadequate funding can result in disrupting projects or gaps in operations.

A.1.3. Organizational Structure

This subsection discusses common challenges related to organizational structure. Organizational structure includes the formal and informal structural arrangements around which staffing, roles, and responsibilities are managed and carried out.

Org-1: TSMO is Limited in its Direct Interaction with IT Leadership Given the Hierarchical Structure

Each State has its individual organizational structure that could include IT units within a State DOT or not. In States where the IT unit is outside the State DOT, interaction may be limited due to the organizational and physical separation.

Consolidating IT services across an entire public agency into a single enterprise group creates efficiencies in managing resources and consistency across the agency. However, it can also limit the interaction between units and cause them to be more transactional. For TSMO, the separation may result in reduced formal and informal coordination and limit the ability to build strategic partnerships or planning efforts that are critical for technology systems that support continuous transportation operations.

Org-2: Lack of Clarity in Defining the Roles of Departments of Transportation IT Organizations Versus the Enterprise IT Organization and Associated IT Ownership

Enterprise groups are designed to maximize efficiencies in technology infrastructure, software platforms, services, and staffing through integration and replication. However, those efficiencies may not be applicable within a DOT/TSMO unit and some States (e.g., Tennessee and Washington) have maintained separate IT units within individual agencies in addition to the centralized IT office or department. This can provide more local support but lead to confusion over the differing roles and ownership.

The larger centralized IT office or department and smaller DOT-focused IT unit typically have overlapping technical capabilities and functions. The result may be confusion over which group has authority on purchasing, staffing, and service, as well as ownership of the system or data. Without clearly defining specific roles, the IT units and their clients may not understand the correct methods to obtain resources. In addition, the enterprise group may set requirements for the individual IT units to obtain permissions prior to specific actions, which may add unnecessary levels of approval and slow implementation.

Org-3: Organizational IT Capability Varies Across the State and is Often Weaker in Rural Areas

Statewide agencies typically have large geographic areas to manage and operate, which results in district or remote offices (or campuses). These district or remote offices maintain local staff or resources where needed and provide a more direct link to the local community. However, these smaller locations may focus on resources required for local infrastructure (construction and maintenance) and not contain all support services necessary to maintain the organization, such as IT staff, particularly when the offices are small and located in rural areas.

Like many administrative or business functions, IT services are often centralized in a headquarters location for various benefits. There are efficiencies to providing agency-wide support from a single, cohesive group that is co-located and managed. The shared resources can be deployed to district or remote offices as needed, and costs can be balanced between locations that may not require full-time staff. The larger headquarters group, or an IT group in a larger district office located in an urban area, typically benefit from a larger talent pool as well, which increases the number of quality candidates.

The challenges associated with a centralized headquarter IT group are largely associated with availability. IT staff are not readily available due to travel requirements and competing locations, and there may be significant delays involved in obtaining onsite service (though remote logins are possible for diagnosis and software-related issues). Rotating staff assigned from a centralized group may also result in unfamiliarity of the local technology systems, particularly if there is any customization.

As an example, the research identified one agency in which the larger metropolitan offices were able to benefit from greater talent pools and maintain stronger staff capabilities in the technology fields. However, the more rural offices lacked candidates with similar skills, and the technology capabilities in those locations were reduced and uneven.

Org-4: Approval Processes are Challenging for the Unique Environment and Function of TSMO

To minimize risk, many agencies have implemented formal approval processes to ensure that technology purchases and implementation follow standard protocols and have been vetted by IT leadership. To maintain control over the processes, there is often little delegation of authority, meaning a few individuals can delay the entire process due to availability or other issues.

Technologies used for TSMO functions are typically not standard enterprise IT solutions and therefore may involve further evaluation and justification. Depending on the level of review specified and how rigid the internal policies are set, this additional step may involve significant effort. Strict adherence to the same procedures used to evaluate enterprise applications may prevent or significantly delay adopting newer solutions specific to TSMO.

As an example, the research identified one agency in which the IT department processes were very rule-driven and lacked a high level of delegation, which created significant delays to budget approvals and project delivery because TSMO did not fall into traditional IT definitions. For this TSMO group to obtain internal IT approvals, project champions were assigned to provide the IT department with specific justifications to gain exemptions and approval.

Org-5: Meeting Expectations and Needs Can Be Challenging When Operating Key Business (IT) Resources is the Responsibility of a Separate Organizational Unit

The division of responsibilities within an agency’s organizational structure is often necessary to efficiently manage resources. It is common for TSMO and IT functions to be in different areas within an organizational structure due to their differing focus and skillsets. A dependent, working relationship is created when ownership (or management) of a technology system within TSMO is assigned to the IT unit.

Staff at all levels influence the relationship between separate divisions, groups, or units. Poorly identifying and communicating expectations and needs can affect business operations and performance at all levels. A lack of planning and coordination by frontline staff may result in increased pressure for resources.

This situation can also present itself in shared environments, where two or more external governments share the responsibility of common or adjacent infrastructure or operations. For instance, an owner-agency responsible for freeway operations may transfer oversight of the interchange traffic signals to the local agency who is vested in arterial operations across the freeway. These transfers of responsibilities are often formally documented through intergovernmental agreements but could also be nonbinding through more informal arrangements.

A.1.4. Policy

This subsection discusses common challenges related to policy. Policy includes legislation and regulations, executive-level directives, departmental policy, or requirements that are not directly targeted to TSMO or are outside the departments or divisions of either IT or TSMO.

Policy-1: TSMO Design, Procurement or Operations are Affected by Policy

Transportation agencies must operate within legislative and policy frameworks of their respective jurisdictions. Some legislation is directed specifically toward transportation agencies and is conceived of and developed with specific transportation policy issues in mind. However, sometimes more general legislation affects transportation agencies in more indirect ways. Centralizing IT resources in a single department can affect transportation agencies even though the legislation is not specifically directed at transportation agencies.

Centralizing IT resources can present challenges because the centralized IT staff may not understand not only the unique environment, requirements, systems, and devices that are needed for TSMO strategies, but also the transportation business at a higher level.

In general, TSMO staff may encounter legislation that makes their normal methods of doing business challenging and may even make conforming to the legislation challenging while meeting budgets, schedules, and other commitments.

Policy-2: TSMO Procurement Processes Can Be Affected by Jurisdiction Level IT Procurement Rules

Many devices, services, and systems needed to implement TSMO-related strategies are considered IT elements that may be required to follow jurisdiction (for example State, county, or city) IT procurement rules. These rules are generally based on the legislation mentioned in challenge Policy-1 above. These rules are often promulgated to implement legislation. Some of the rules have been in place for many years. In many cases, the rules were developed considering traditional enterprise IT devices, services, and systems. The environment for the systems and devices is usually inside a building. The functions that were the focus of the rules tend to include financial management, human resource, data management and reporting, and agency-level communication systems (both voice and data). The procurement rules may not have been developed considering transportation field devices or transportation management or maintenance systems. With more technology integrated in transportation devices, the expansion of sophisticated software systems, and the increased amount of data that needs to be managed, TSMO-related procurement increasingly falls under IT procurement rules.

Historically, transportation agency procurement rules for field devices and systems that communicate with and control them have developed to be in line with procurement of other transportation services and items, most notably construction, equipment (such as vehicles, signs, or material), and professional services. The requirements and processes for traditional transportation procurement and those for IT procurement differ and may not be compatible. The challenge for TSMO staff is to navigate the two sets of procurement rules to determine which take precedence and how best to utilize the most appropriate rules for each procurement. (FHWA developed a memo clarifying the distinction between operational improvements and ITS construction as it relates to procurement options: Procurement and Authorization of Federal-Aid Operational Improvements (Non-Construction Projects.)

A.2. Business and Technical Challenges

Implementing technology within an agency can be affected by the business and technical processes associated with developing systems, including planning, procurement, cybersecurity, and data management. Challenges can arise when processes that worked well before the proliferation of IT systems and devices are not suited to addressing the particular needs of TSMO or IT organizations. In other situations, processes may have been developed in isolation within either the TSMO or IT organizations and worked well if the organizations remained isolated. Common business and technical challenges could be encountered at any stage of project activity or development.

A.2.1. Strategic Planning

This subsection discusses common challenges related to strategic planning. Strategic planning includes the vision, mission, and objectives of the different organizations and functions and the integrating each in overall agency planning and resource allocation.

Planning-1: Strategic Plans and Governing Mission, Vision, and Goals Can Differ Between TSMO and IT

Agencies, and departments or divisions within the agencies, typically develop long-range strategic plans that guide policy, practice, and funding decisions. These strategic plans are usually prepared internal to the groups, and while they may consider outside needs and services, they may not be coordinated or integrated at a high level.

TSMO strategic plans are generally centered around external factors related to traffic operations and safety. The mission, vision, and goals are based on external performance measures (mobility, reliability, crashes) and supporting resources (staff, training, infrastructure). While a TSMO strategic plan may include supporting IT resources, these goals or needs are not communicated to or incorporated into the IT group’s strategic planning efforts.

The gap in coordinated strategic plans can ultimately create gaps in funding and programming efforts when goals shift to actions.

Planning-2: Programming and Development of TSMO IT Projects Does Not Align with Traditional Construction or IT Projects

Several IT-based projects or efforts that support TSMO are unique and do not align with traditional agency programs. These types of projects do not fit within the most common project development model centered around infrastructure construction—design (consultant) followed by build (contractor). They also may not fit within traditional IT efforts, which may focus on procurement of stand-alone products or services—equipment, software packages, network services.

TSMO-related IT needs are often unique within a public agency and may be a lone application of the system or services. As such, there may not be a standard process or even familiarity with the products, services, or providers. For example, data sources such as probe traffic data or weather monitoring services are readily available but may not fit in operating budgets. Many agencies that do not have enough funding in operations or maintenance budgets choose to program capital improvement funds to purchase such data. In addition, IT support may be needed for custom analytics or integration into existing systems.

The establishment or upgrade of a traffic management center is a unique hub of field data, media (e.g., video), and interagency networking that is not common for an agency to routinely build and may require a unique approach.

As an example, one State DOT identified an IT-based project that would develop a geographic information systems-based tool to assist in maintenance management. State legislation required the project to be led by the Statewide IT department because the tool would have IT components. The State DOT prepared the initial Request for Proposals based on the TSMO business needs, however the assigned IT department project manager followed standard enterprise IT contracting language. Because some of the TSMO-specific requirements were omitted from the standard procurement language, the contract took three years to complete instead of the original estimate of one year.

After completion, DOT staff identified two possible alternatives to the selected approach. First, identify if vendor-hosted services could be used to reduce internal IT burdens. Second, break the project into smaller contracts, each of which would be easier and less complex to manage.

Planning-3: Business Metrics and Performance Measures for TSMO Are Not Integrated in IT Unit’s Management

Business metrics and performance measures are important aspects to track an organization’s progress, identify trends, and communicate vital information to leadership. The development of business metrics and performance measures will be depend on the function of each business unit but can support the goals and objectives of the larger organization. Primary TSMO metrics typically focus on traffic operations measures, such as safety, efficiency, and reliability. Agencies often choose to have performance metrics include the IT components that TSMO relies on. For example, TSMO relies on many IT assets to monitor and operate roadway infrastructure as well as perform data analytics. These various IT-related steps need to be reliable and functioning to deliver on the primary TSMO business metrics. Performance measures surrounding field device “uptime” or reliability may be part of a TSMO operating unit but not included in an IT unit’s performance measures because they do not have similar field devices in their normal IT operation. Without similar performance measures within the IT unit, the tracking of needs and support resources may not be visible to appropriate IT leadership.

A.2.2. Procurement

This subsection discusses common challenges related to procurement. Procurement consists of the processes and procedures associated with buying IT-intensive products or services.

Procure-1: Statewide Procurement Policies and Procedures Often Do Not Align With TSMO Initiatives

Procurement policies are established to provide a structure to purchase and acquire goods and services. These policies help suppliers and vendors understand the requirements and navigate the process and protect the agency against harmful actions, whether willful or unintentional. Often, these policies are developed to apply to the widest range of services or departments to minimize cost and administrative burden on the agency.

Many IT goods and services are procured through large bulk orders which allow for efficiencies through volume orders, repeat orders, or open contracts. Some TSMO-related IT needs fit well under this arrangement, such as traffic detection or closed-circuit television (CCTV) cameras, electronic signs, or radios. Other TSMO-related IT goods or services often fit in a niche market and are provided by a narrow field of competitors that may not lend themselves to large bulk procurements. Traffic signal controllers and ramp meter controllers are examples of these types of niche TSMO purchases. Working through procurement with IT staff who are unfamiliar with transportation-specific devices often leads to significant adjustments to typical IT procurement practices, adding time and expense to the effort.

Procure-2: Purchasing Activities Require Multiple Levels of Approval Across Groups

A primary purpose of procurement offices is to regulate purchasing activities through policy and administration. This typically involves removing the advertising and selection process from the end user to preserve a fair and transparent award. To maintain this order, there is limited delegation of authority to individual divisions or groups.

Most agencies’ TSMO-related procurement processes follow the established approval process within the procurement organization, which may include multiple levels of authorization and administration. If the contracts are atypical or of a certain magnitude, these approvals are concentrated to a select few procurement officers who have the appropriate authority. The limited capacity of these approvers, combined with requests from multiple agencies, may increase the time required to process contracts particularly if managed on a sequential basis.

As an example, the research identified one agency in which the TSMO group is required to obtain several levels of approvals for procurement, including approval from the Statewide IT agency. The various approval levels often result in a time-consuming procurement process.

Procure-3: Business Needs for TSMO Require Both Legacy and Emerging Systems

The management of traffic operations has been part of many DOT programs for decades. Early efforts centered on traffic detection, video surveillance, and dynamic message signs. Many of these systems are still in operation, and agencies rely on continuing the existing IT platforms that store, analyze, and communicate the data.

As agencies transition to emerging systems for increased breadth and depth of information, the legacy systems are often still required. Procurement activities involving emerging technology may focus only on the requirements of the new system and not integrating and continuously operating the existing systems. Often the amalgamation of the various systems requires specialized services to transfer or translation between the systems or a new software platform that utilizes both systems.

Legacy platforms utilized in TSMO systems may also be considered outdated or to pose security vulnerabilities by enterprise IT groups or centralized procurement offices. In these situations, Statewide administrators may attempt to eliminate procurement contracts if they are not regularly used or supported by larger IT efforts. While eliminating older contacts may increase efficiencies for procurement offices, it may reduce options for TSMO efforts in the future.

Procure-4: Lack of or Limited Pre-Approved TSMO-Related IT Equipment/Services Can Slow Efforts

Agencies often use pre-approved product or vendor lists to reduce the time required to obtain equipment or services. These lists are developed through the early evaluation of products and services against specifications or requirements but without specific authorization for purchase. As needs arise, agency groups can utilize the pre-approved lists to acquire products or services.

Pre-approved product lists are commonplace for certain groups within DOTs that have well-defined specifications or product requirements that vendors can be evaluated against. Examples of traffic items include pavement marking paint, traffic-control cabinets, roadway luminaires, and breakaway bases, all of which are standard in design and application.

TSMO related IT products are not easily included in pre-approved lists due to the lack of design standards or specifications and the rapidly evolving requirements. The time required to develop specifications, solicit vendors, evaluate equipment, and obtain approval is often prohibitive in an environment like TSMO, where needs change and procurement activity is neither high volume nor standardized. Pre-approved products or contracts for specific equipment and devices are crucial in maintaining and replacing spare parts as devices need to be replaced.

A.2.3. Systems and Technology

This subsection discusses common challenges related to systems and technology. Systems and technology encompass all hardware and software components of data acquisition, management, and utilization technologies.

Tech-1: Challenges Maintaining Legacy Systems and Deploying Emerging Technology

Many TSMO agencies have well-documented procedures and staffing guidelines for the existing technologies on which they rely. However, these legacy systems may not be compatible with or may not complement newer technology. As agencies implement emerging technology, they may want to develop new approaches to support both legacy equipment and the new technology.

Many TSMO systems and devices are expected to last 10 to 20 years, and funding cycles to upgrade or replace these systems and devices reflect this lifecycle. As new technology is implemented—either as older legacy systems come to the end of their lifecycle or as new systems are deployed—agencies face the challenge of operating and maintaining new technologies while continuing to maintain legacy equipment and systems. Agencies have found that both maintenance and operations staff need to be proficient in both new and legacy systems. Agencies have also found that spare parts and replacements need to be maintained for all the technologies implemented. These challenges are faced in virtually every type of TSMO system, from traffic signal, to environmental sensor stations, to dynamic message signs, to cameras and detectors.

As an example, the research identified one agency that operated several different systems that all worked individually, but not together in an integrated or interoperable way. The growth of these systems had been ad hoc and not coordinated, thereby reducing interoperability and creating a need to integrate them afterward with no clear pathway for that integration.

Common types of systems that are implemented in an ad hoc manner and later integrated over time—incorporating both legacy systems and new technology—include communication systems and devices, detectors and data collection systems, and control systems for individual types of devices (like dynamic message signs and camera systems) that are later integrated in a single management system.

Tech-2: Insufficient Data Communication System Bandwidth and Redundancy for TSMO Services

Communication lines are critical to IT-TSMO efforts and allow data exchange between offices, field equipment, and other agencies. These can be fiber-optic cable, wireless cellular, radio, or other modes, including older copper lines. Each mode (and path) has a maximum bandwidth that determines the capacity of the data transfer and the types and amount of data that can be transmitted.

Functionality between locations is limited by the weakest link in the communication network. If lower bandwidth communication lines are part of a network path, the data will be constrained. This is particularly important because newer technologies generate more data and require larger bandwidths than those required by legacy systems. Challenges with bandwidth have surfaced with higher definition camera systems and connected-vehicle systems that generate a significantly greater amount data than older systems did.

Reliability in communication networks depends on mode and network structure. Networks that are linear can create vulnerabilities for TSMO operation; the lack of multiple paths may restrict routing options and limit redundancy in the case of a localized transmission outage. This can cause significant disruption to operations that run 24/7, because repairs on underground lines may require extensive work. Traditional transportation professionals may not be proficient at designing communication networks and specifying network equipment that will be optimized for the range of equipment and pathways needed. IT staff may have more experience with and knowledge of communication networks that will support these needs.

Tech-3: Complications with Interoperability Between Agencies and Partners

TSMO coordination across agencies can provide beneficial results to the traveling public. Integrating data, technology platforms, and communication networks can create synergies and expand functionality and range. Interoperability of data, software, and standards is possible but involves joint planning, and execution can be complicated due to the various TSMO and IT groups involved.

More often, different agencies have differing systems that are not directly compatible. Security issues (including firewall access and user credentials) often limit interactions to one-way exchanges or manual transactions. At the most basic level, even the exchange of data may require translation between differing formats and protocols.

As an example, the research identified one agency that had separate voice and data systems for the State DOT and public safety agencies, which did not allow for voice communication between field staff of the two agencies and did not allow for the electronic exchange of information, specifically for exchanging data between the public safety computer-aided dispatch system and the State DOT traffic management system. This lack of interoperability impacts the exchange of information during joint operations and impacts staffing and the ability to support multiple systems with limited resources.

Tech-4: Inability to Fully Utilize Third-Party Data and Crowdsourcing Data and Services

The availability of third-party data and crowdsourcing data can improve an agency’s ability to evaluate and respond to conditions. The datasets vary widely but can provide real-time, granular information across an extensive geographic area. To access the data, agencies will overcome various obstacles including but not limited to justification, funding, data management, and risk assessment.

External data sources (such as third party and crowdsourcing) often require a subscription and continued investment to receive a constant feed. Benefits of these data services are not always easy to identify and quantify against the cost, which may pose a challenge for agencies seeking to justify the additional costs.

Internal agency policies may prevent use of certain external data. For example, risk management groups may not allow the use or distribution of unverifiable data or data that is generated from unknown sources. Procurement rules may prohibit acquiring data that may be proprietary or sole-sourced.

From a technology or process standpoint, third-party data generally provides access to new information, but it is not customized. Additional effort is often needed to develop tools, automated tasks, performance measures, or other output that utilizes the third-party data for agency goals.

Tech-5: Emerging Technology in Connected and Autonomous Vehicles and Smart Cities Requires Significant IT Resources.

Connected and autonomous vehicle (CAV) and Smart Cities are evolving through various pilot projects, and the scope of each is still being refined. The IT needs for these efforts are still unknown but will likely be significant. Each of the two initiatives relies heavily on technology, particularly communication and data. These are skill sets that most TSMO groups depend on IT for support.

CAV and Smart Cities initiatives are not typical applications for an agency; therefore, IT staff may not be familiar with either. IT groups may not be fully engaged in the upfront planning or scoping of emerging technologies, and technical resources may not be clearly identified. Extensive partnership between multiple agencies, private service providers, manufacturers, utilities, and other public and private organizations can further complicate identifying IT resources and who is responsible for each function.

Tech-6: TSMO Initiatives are Impeded Due to Lack of IT Flexibility in Enterprise Operations

Centralized IT groups are designed to maximize efficiencies in technology infrastructure, software platforms, services, and staffing across an entire agency or across an entire State. Through deliberate planning, IT groups focus on integrating and replicating technology to simplify staffing, reduce costs, and improve internal reliability. TSMO strategies may rely on specialized IT approaches and platforms that may not coincide with other groups within an agency and may run counter to central IT objectives.

State IT policies and standards have been developed over time and may pre-date TSMO needs. The rigidity of these policies may impede TSMO efforts if central IT leadership is not receptive of the requests and does not provide resources.

As an example, the research identified one agency in which the IT department and the transportation agency had overlapping definitions of “engineering” and “data processing.” Both agencies believed they were responsible for traffic management systems. The IT department viewed these systems as “data processing” systems and therefore under their oversight. The transportation agency viewed them as essential systems to deliver transportation “engineering” solutions, and therefore completely within their authority. IT oversight involved processes and technologies that did not result in traffic management systems that met the transportation agency’s business needs. The lack of flexibility in the definition of “data processing” versus “engineering” led to added time and expense in the procurement and implementation process, and in procuring systems that did not meet the transportation needs as well as they could have.

Tech-7: Designing Complex Intelligent Transportation Systems Require a Systems Engineering Approach With Which IT Staff May Not Be Familiar

The complexity of TSMO systems may call for a systems engineering approach to successfully deliver a reliable project. The systems engineering approach starts in the planning stage and continues through design, construction, implementation, and operation with checkpoints and reviews at each stage. The benefit of the process is to reduce risk, improve reliability, and ensure all needs are considered.

Despite the wide use of the systems engineering process, agencies may underestimate the need on TSMO projects due to a perceived “off-the-shelf” simplification of the technology. In efforts to speed programming, TSMO projects may proceed through planning and scoping without the full benefit of the process. (See 23 CFR 940.11, sets requirements regarding the systems engineering process, standards, and interoperability for ITS projects that use Federal funds.)

When used on TSMO projects, the process is typically managed internally and coordinated with other stakeholders. While most efforts include neighboring agencies, law enforcement, maintenance, operators, and other key resources, IT staff may be overlooked in the initial steps. The lack of initial input from IT at the planning and scoping stages can increase risk and scope changes later in the process.

A.2.4. Risk/Security

This subsection discusses common challenges related to risk and security. Risk and security include those challenges associated with network security, data sharing, third-party applications, hosted and cloud platforms, and automation.

Risk-1: Cybersecurity Vulnerabilities

Public agencies face an increasing number of cyber threats as technology systems become more ubiquitous and integrated in operating physical assets. Technology-based applications that are more passive (such as surveillance or detection) are not as critical or as targeted as those that are more active and control-based. Systems that use technology to control, communicate, or operate physical assets (whether traffic signals or transit vehicles) can be high-value targets for cyber-attacks. Attacks on these systems can cause significant immediate operational and safety impacts, as well as longer-term societal impacts as trust in the systems are eroded. Longer replacement cycles for TSMO and ITS devices, compared to that for IT assets, increase cybersecurity risk, because the older devices may not have security patches available.

Vulnerabilities in cybersecurity are varied and can occur in different parts of the agency system. Traditional malware can access networks if not identified or filtered by security software or firewall, or when users fail to patch software updates. External users can also access networks through weak password control (e.g., user accounts with no password protection, common “admin” accounts, or written passwords attached to equipment).

The emergence of connected vehicles adds another attractive target for cybersecurity threats. Intrusions into connected-vehicle control systems could provide direct threats to the safety of the traveling public as well as reduce confidence in these systems and in transportation agencies that support them.

As more devices are deployed and web-based service or cloud computing is utilized, cybersecurity becomes more important as the location and type of threats multiplies. The effective network within the emerging Internet of Things becomes exponentially larger and the complexity of multiple platforms can introduce new vulnerabilities.

A relatively common example of cyber-attacks that agencies experience is “hacking” dynamic message signs. While not presenting immediate danger, rogue posting of messages can misdirect drivers, offend the public, and reduce confidence in the agency.

Risk-2: Lack of Robust Data Governance Rules

Many TSMO efforts rely on data to monitor and evaluate operations. Historical data sources were often agency-owned and limited in scope to vehicle count and speed data, which were manageable in terms of size (data fields), quantity, and sensitivity (personal identifiers). Newer data sources provide significantly more information but call for a new approach to data management.

Advances in technology have created many new data sources ranging from Wi-Fi/Bluetooth sensors, license plate readers, third-party probe data, and lidar mapping. Each source has varying degrees of personally identifiable information, various formats, varying size, and other factors. The third-party data is often a combination of many data sources and often references the “four V’s” of big data: volume, variety, velocity, and veracity.

Agencies have experienced challenges in managing the various data sources due to the lack of robust data governance regarding ownership, storage, sharing, and other factors. The absence of these rules creates security concerns for IT and data professionals and impacts the design of systems and platforms to analyze and archive the data.

As an example, the research identified one agency in which there was a general lack of planning—particularly in systems engineering—for data use and archiving. In this case, the issue was compounded by differing data protocols between agencies. The lack of a data management plan or a Concept of Operations for data use and archiving resulted in each headquarters or local office developing their own data standards. In fact, the data protocols could differ on a county-by-county basis. This inconsistency hampers interoperability and reduces the ability to host a common data archiving system or prepare combined analyses.

Risk-3: Increased Sharing of Information With a Third Party Can Increase Security Risks

The exchange of data between DOTs and third parties has increased due to the overlapping of services and potential benefits in the sharing of information. Often, DOT-owned data can provide third-party data providers with ground truth and fill gaps in their data sources while third-party data can provide DOTs with a broader coverage than traditional field collection.

Many third-party data providers are requesting data directly from the DOT, either as a stand-alone request or as an ongoing exchange of data. Many of these requests may not identify the end use or ultimate consumer, because the data can be combined with other sources and repackaged. There are no standards regarding the business need and level of access required, because many State DOTs and third-party data providers are still evaluating the utility and value of the data. Specifics on how to deal with sensitive information, liability, and public “open access” rights are typically not available or developed on a case-by-case basis.

Risk-4: Location of TSMO Network has Impacts to Security, Support, and Availability

Many aspects of TSMO have a heavy reliance on IT systems for field connectivity and software platforms. These systems and external connections, when installed on central networks, may be limited in terms of access due to strict IT administrative rules. This has led some TSMO groups to locate these systems on separate networks to maintain separation and allow more flexibility and autonomy. The level of network security, access, and IT support may differ depending on the network location.

TSMO groups with systems installed on central networks have encountered issues with being able to share data with organizations outside the agency’s security firewalls, incompatible system updates, and limited system and data access without IT approval. These factors are designed to maintain consistent network security across an agency but may have unintended impacts on TSMO functions (such as limited remote access and barriers to data sharing and sharing control with partner agencies).

TSMO groups with systems installed on separate networks have more flexibility in terms of administrative rights, software updates, and access. However, IT groups may have limited knowledge of these networks, which limits the support IT groups can provide. If TSMO networks are not connected to the primary IT network, IT staff would be unable to regularly monitor the TSMO networks, and the TSMO networks may be considered external to the agency’s primary network.

As an example, one TSMO group that participated in the agency outreach effort of this project had lost their ability to view some video feeds on computers tied to the agency network. The issue originated from new security policies that were implemented by the enterprise IT group, which did not allow the types of video used by the transportation agency’s CCTV system. The IT staff was not aware that any agencies on the network used this type of video. As an immediate resolution, the computers were moved onto a separate network to bypass the newly created conflict.

Risk-5: Summarized Data Sets and Analyses Can Mask Accuracy and Reliability Issues

The availability of third-party services, whether data or analytics, can improve TSMO operations by providing expanded datasets and/or quicker analysis. Introducing these services can reduce workload on staff and potentially automate complicated data processes. However, the lack of access to the raw data or lack of understanding of the underlying processes can mask accuracy or reliability issues.

External datasets (such as probe data) are a compilation of numerous individual datasets that are combined, assessed, and validated prior to being delivered to agencies. The processes involved are typically proprietary, and the end product is not transparent in terms of source or quality. Due to the lack of source data or validation methods, agencies generally either take the data at face value or perform limited checks against other available data.

Transportation professionals may not be aware of these issues and may not have the knowledge to incorporate ways to overcome these challenges in contract language. IT staff may be available who would be able to support TSMO staff in proposing contract language that include accuracy and reliability measures that would be acceptable to the data vendors.

External analytics are another service that is not easily deciphered. The end product can be customized to meet the agency’s goals, but the internal mechanics and processes are typically hidden within proprietary software. Some systems also include internal datasets that are not transparent. Rigorously checking the validity of the output may be beyond the capability of internal TSMO staff. However, IT staff may be available who would be able to assist TSMO staff in procurement documents or contract language to provide confidence in the analytics and an understanding of the analytical process while being acceptable to the vendor.

Risk-6: Field Equipment is Susceptible to Physical Damages

Many IT systems deployed by TSMO are either installed within the public rights-of-way or rely on equipment installed in the rights-of-way. Examples include fiber-optic cable, network switches, signal controllers, traffic sensors, and power sources. Each device or connection installed in the field is susceptible to physical damage, whether accidental or intentional.

Accidental damage to IT equipment can occur due to a variety of sources. Common occurrences include vehicular impacts, power surges, lightning, water, flooding, construction work, and extreme temperatures. Intentional damage is less common but has occurred due to copper theft, gun shots, vandalism, or curiosity.

TSMO staff may not be alerted to damaged field equipment unless it is reported by police, maintenance staff, or the public. Often, stand-alone equipment may not trigger an automated alert that it is inoperable, and there may be a delay in identifying the lack of utility. Damage to connected IT infrastructure may be identified more quickly because the impact may be greater than the single location. For example, broken fiber-optic cable would impact an entire network of devices and the outage would be quickly noted.