Systemic Safety Review Methodologies

By: Jike Wang, Nelson Costa, Ben Kos, Soroush Salek

Abstract

In the past few years, the Systemic Road Safety (SRS) process was introduced by the Federal Highway Administration (FHWA) to address some of the limitations of network screening based on the collision history of specific sites in a road network. The SRS process also considers the site's collision risk factors. Risk factors are those characteristics common to locations experiencing specific types of collisions.

In this article, two alternative SRS methodologies (i.e., the core and enhanced methodologies) are presented and used in analyses of the road network in York Region, Ontario. Their analysis results are compared, as well as their complementary nature to the results of network screening (based on site-specific collision history).

Acknowledgments

This study has been completed as part of the Traveller Safety Plan for the York Region. The authors thank the York Region project team for their input throughout the process and their permission to publish the outcomes of the conducted systemic safety analysis.

Introduction

The traditional Road Safety Management (RSM) process incorporated in the Highway Safety Manual (HSM) is widely used by different agencies in North America, including Canada, to improve road safety networkwide. The network screening component of the RSM typically identifies collision-prone locations following methodologies such as the Empirical Bayes (EB) approach using Safety Performance Functions (SPFs) calibrated by local historical collisions, traffic volumes, and road characteristics.

Despite significant success in objectively identifying and prioritizing collision "hot spots", the EB network screening process is subject to some restricting factors, including (1) poor estimation of AADTs for different facilities as the measure of exposure due to outdated count data, 2) the latency in updating fatal and serious injury collisions in agencies' road safety database, due to lengthy collision investigation effort involved for such collisions, 3) scatteredness of specific collision types (e.g., pedestrian and cyclist collisions) across the network preventing agencies from developing reliable SPFs for such collision types, and 4) the reactive nature of network screening process which is heavily dependent on historical collisions rather than environmental collision risk factors.

In the past few years, the Systemic Road Safety (SRS) process was introduced by the Federal Highway Administration (FHWA) to address some of the limitations of network screening [1]. The SRS process not only focuses on the prior collision history of facilities but also considers their collision risk factors. Risk factors are those characteristics common to locations experiencing specific types of collisions. These factors may include site-specific collision history, when available, in addition to geometric and operational characteristics. The SRS process is capable of identifying sites with specific risk factors, even in the absence of a history of target collisions. Consequently, these sites can be selected proactively for safety treatments based on their unique risk factors.

The core SRS methodology for the SRS process was initially outlined in the FHWA Systemic Safety Project Selection Tool in 2013 [2]. According to this methodology, a feature characteristic can be identified as a risk factor if the proportion of collisions at facilities with that characteristic is overrepresented compared to the proportion of those facilities within a jurisdiction. The magnitude of the collision overrepresentation can further be used to estimate the significance of the risk factor (for example, if 5% of rural unsignalized intersections are skewed, but more than 20% of the severe rural unsignalized collisions occurred at skewed intersections, rural skewed intersection is a risk factor and its significant level is 15%).

In a recent publication, the National Cooperative Highway Research Program (NCHRP) presented a more robust approach to identifying risk factors and estimating their significance level [3]. Based on this enhanced approach, candidate characteristics can be included as independent variables in an SPF for a collision type of interest. Characteristics exhibiting statistical significance can be adopted as risk factors, and their calibrated coefficients can be used as their relative significance levels.

The application of these two approaches in identifying risk factors for an SRS process is evaluated as part of the York Region Traveller Safety Plan (TSP). The outcome of the SRS process was further assisted in informing the priority locations for implementing specific countermeasures resulting from the TSP action plan.

This article summarizes two alternative SRS methodologies, compares their outcomes with each other and outcomes from network screening, and explores their advantages and disadvantages through the York TSP case study.

Methodology

The SRS process aims to proactively identify and address safety issues at facilities exhibiting high-collision potential not based on the collision history of the facility but on aggregate networkwide collision trends associated with risk factors present at multiple facilities (e.g., roadway geometry and cross-sectional design, roadside and area features, traffic control, and traffic volumes).

The SRS process consists of five steps, as shown in Figure 1. Since this study centres around comparing two alternative SRS risk analysis approaches (i.e., Core SRS methodology vs. Enhanced SRS methodology), this section elaborates on their methodological differences. The findings of the SRS process for the York TSP case study, including the comparison results, are presented later in the article, excluding the outcomes of the final step of the SRS process.

Figure 1: Systemic Road Safety Process Overview

Identification and Evaluation of Risk Factors – Core SRS Methodology

The core FHWA risk assessment methodology relies on the magnitude of collision over-representation to identify risk factors. The risk assessment is undertaken by comparing the proportion of facility types where the potential risk factors are present with the proportion of collisions experienced at those sites. Those potential risk factors with over-represented collisions are confirmed and selected as reliable risk factors. The observed over-representation values are then adopted to determine the weight of each risk factor.

A Systemic Safety Risk Index (SSRI) is calculated for each facility by adding the magnitude of the selected risk factors. However, to give these magnitudes an intuitive meaning, they are normalized so that a facility presenting all risk factors at its highest level has an SSRI of 100. The facilities are ultimately prioritized based on their SSRI values.

Identification and Evaluation of Risk Factors – Enhanced SRS Methodology

 

The enhanced risk assessment methodology takes a more sophisticated statistical approach to identify risk factors. This approach assesses the impact of potential risk factors by incorporating them as independent variables in a detailed SPF, utilizing an optimal generalized linear modelling structure, specifically employing negative binomial regression. The adopted model structure is shown in Equation 1:

Where,

• E(Y) is the average number of target collisions per year;
• X_i is the i^th risk factor; 
• α is the model intercept; and
• β_i is the weight of the i^th risk factor.

The performance of the model is assessed by its overall goodness of fit, model overdispersion, and the significance level and sign of model coefficients. Moreover, the average annual collisions estimated by the developed SPFs are further used as a performance metric to rank similar facility types.

Theoretically, the enhanced SRS methodology has several advantages over the core SRS methodology and is expected to result in more reliable outcomes:

• Accounts for the correlation between the risk factors and does not overcount their impact through the prioritization process;
• Results in a more intuitive and tangible performance metric; and
• Results in a continuous performance metric and allow for differentiation between locations with similar but not identical characteristics. The scoring process of the core SRS methodology has a discrete nature and may result in multiple ranking ties for sites with similar characteristics.

Required Data

Three types of data are used to conduct the SRS analysis: (1) physical characteristics, (2) traffic volume, and (3) collision. The required data for this assignment were extracted from the York Region road safety database and the associated GIS files, as well as from Google Earth / Google Maps satellite imagery. The details of the data compiled for the SRS analysis are discussed in the following sub-sections.

Physical Characteristics Data

The physical characteristics data extracted from the Region's road safety database and Google Maps satellite imagery are included in Table 1. The characteristics available from the Region's road safety database are adequate for conducting EB network screening but do not serve the purpose of the SRS analysis. We extracted additional physical characteristics through GIS coding and manually reviewing Google Earth / Google Maps satellite imagery. The additional physical characteristics correspond to potential risk factors suggested by FHWA [2], which were later reviewed, in this case study, to determine which ones (and to which extent) correlate to a higher proportion of collisions.

Table 1: Physical Characteristics Attributes for the SRS Analysis

Data Available from Region's Road Safety Database

Data Extracted Manually or by GIS Coding

Traffic Volume Data

Available AADT data were obtained through the Region's road safety database for all regional road segments and intersections. For the case of intersections, the AADT data were available in the form of total approach traffic volumes separated for the major and minor roads. The required traffic volumes were extracted for the study period between 2015 and 2019.

Collision Data

Collision data for the study period between 2015 and 2019 were also obtained from the Region's road safety database. Collision information at all regional facilities was coded into this database per the associated Motor Vehicle Accident (MVA) reports. Each collision site was geographically referenced to the associated regional facility using the existing Geo IDs. The collision data included all details such as collision impact type, collision severity, date and time, road conditions, environment conditions, direction of travel, driver actions, pedestrian actions, vehicle types, vehicle conditions, and vehicle maneuvers.

York Region Case Study - Results

As explained in the methodology section, the findings of the first four steps of the SRS process, including the two alternative risk analysis approaches, are presented in this article. The outcomes of the final step of the SRS process (i.e., Selection of countermeasures for priority locations) are not included in this article and are reported elsewhere.

Step 1 - Selection of Focus Collision Types

The TSP was developed following the Safe System philosophy to address the fatal and injury (FI) collisions within the Region. The vision for the TSP is to end all serious injury collisions in York Region. Moreover, the supporting short-term goal of the TSP is to reduce FI collisions by 15% between 2024 and 2028. On the other hand, collisions involving vulnerable road users (VRU) were among the most frequent collision types at regional road facilities (Figure 2) and were selected as one of the collision emphasis areas for the TSP, as per the findings of data analysis, public input, and stakeholder feedback. Therefore, the project team adopted FI and VRU collisions as the focus collision types for the SRS process.

Figure 2: Frequency of FI Collisions by Collision Type

Step 2 - Selection of Focus Facility Types

Intersection collisions were the most frequent collision type at regional road facilities (Figure 2) and were selected as one of the collision emphasis areas for the TSP. As shown in Figure 3, almost 90% of intersection collisions occurred at signalized intersections. Therefore, signalized intersections are selected as the focus facility type. Specifically, during the study period (i.e., 2015-2019), collisions at these facilities constituted the highest percentage of the selected focus collision types (i.e., 61.7% of fatal and injury collisions and 63.6% of VRU collisions).

Figure 3: Collision Tree – FI and VRU Collisions

Step 3 - Identification and Evaluation of Risk Factors

Core SRS Methodology

As explained in the methodology section above, the core risk assessment methodology relies on the magnitude of collision over-representation to identify risk factors. For each potential risk factor, the target facilities (i.e., signalized intersections) are divided into some sub-categories based on their characteristics related to the given risk factor.
The proportion of each sub-category is calculated in percentage and then compared to the proportion of collisions experienced at those sites. Figure 4 explains this methodology for the number of bus stops within 50 meters of signalized intersections as a potential risk factor. For both focus collision types, signalized intersections with three or more nearby bus stops presented collision proportions (35.5% for FI collisions and 40.5% for VRU collisions) higher than the proportion of facilities (23.6%). Based on these results, the presence of three or more bus stops within 50 metres of the signalized intersections is selected as a risk factor for both FI and VRU collisions and the weights of the risk factor for each focus collision type were calculated based on the observed collision over-representations (11.8% for FI collisions and 16.9% for VRU collisions).

 

Figure 4: Sample Risk Factor Identification – Core Risk Assessment Methodology

Enhanced SRS Methodology

As explained in the methodology section above, the enhanced SRS methodology takes a more sophisticated approach to identifying risk factors. A detailed SPF is developed for each focus collision type, and the statistical significance of potential risk factors is examined by including them as independent variables in the model. The average annual collisions estimated by the developed SPFs are further used as a performance metric to rank similar facility types.
It is anticipated that potential risk factors showing the strongest statistical correlation with focus collision types while simultaneously displaying minimal correlations with other risk factors will yield more accurate estimations for the target collisions. Therefore, potential risk factors underwent an initial screening based on their statistical correlation with the focus collision type and other potential risk factors. As per this initial screening, the optimal collision risk factors were shortlisted for the final modelling exercise. The statistical correlations of shortlisted risk factors are shown in Table 2.

Table 2: Correlation Matrix for Shortlisted Risk Factors

Subsequently, the statistical significance of the shortlisted risk factors was assessed using the negative binomial regression model outlined in the Methodology section above.
The calibrated model parameters for signalized intersections are shown in Table 3. According to this table, for FI collisions, total entering AADT, cross intersections, intersections with three or more nearby bus stops, and intersections with six or more lanes on their major approach are shown to be risk factors. Moreover, for VRU collisions, the same risk factors are selected, with one additional risk factor of urban areas. The weight of each risk factor is defined by its calibrated coefficient.

Table 3: Calibrated Model Parameters for Signalized Intersections – Enhanced SRS Methodology

Comparison of Effort Required

The effort required for SRS analysis was broadly comparable between the two methodologies. Initial analysis steps, such as data collection, selecting focus collision and facility types, and the risk assessment (i.e., scoring and ranking), remain consistent across both methodologies. The primary differences lie in identifying and evaluating risk factors, but both methodologies require each potential risk factor to be reviewed individually. Although the Enhanced SRS methodology encompasses additional computational steps and demands marginally more computing power, using statistical packages like Matlab, SAS or R.

Systemic Safety Risk Assessment

The defined performance metrics for each risk assessment methodology (i.e., SSRI for the core SRS methodology and estimated annual collisions for the enhanced SRS methodology) are used to assess regional facilities network-wide. Figure 6 and Figure 7 show the top 50 signalized intersections from the core and enhanced SRS methodologies for the FI and VRU collisions, respectively.
The following observations can be inferred from the comparison of the top priorities from the two methodologies:

• There is a substantial overlap in the top-ranking locations, and the overlaps form similar hotspots along corridors. This can be explained by the partial similarity in methodology between the two approaches.
  • FI collisions: 28 overlaps within the top 50, 68 overlaps within the top 100.
  • VRU collisions: 34 overlaps within the top 50, 69 overlaps within the top 100.
• Aside from the overlaps, the enhanced SRS methodology identifies more isolated signalized intersections.
• The enhanced SRS methodology accounts for the overlap among the impact of risk factors and introduces further improvement by providing a unique rank for each location. In the core SRS methodology, duplications in ranks are observed as the core approach relies on categorical factors and can potentially result in identical scores for locations with identical characteristics. The enhanced SRS methodology, in contrast, leverages Average Annual Daily Traffic (AADT) as a continuous variable to break the potential ties and does not result in multiple similar rankings among facilities. These nuances allow for a more granular ranking, eliminating duplicate ranks and providing a distinct ranking for each location. This enhanced precision in ranking and accounting for the overlap in the impact of risk factors demonstrates the superiority of the enhanced SRS approach, offering more nuanced and accurate results.

Comparison with Network Screening Results

While network screening focuses on identifying and addressing specific high-crash locations based on historical data, systemic safety review takes a broader and more proactive approach, aiming to identify underlying risk factors that contribute to the likelihood of collisions across the entire road network.
Figure 8 presents the overlaps among the top 50 locations between the Region's network screening results and the enhanced SRS results, where the following observations are noted:

• Network Screening vs. Enhance SRS: The immediate observation is that, as reflected by the small overlaps between the network screening results and the enhanced SRS results, many top-ranking locations from the network screening results have relatively lower ranks in terms of enhanced SRS results. This observation highlights the complementary nature of the two methodologies;
• Enhanced SRS FI vs. Enhanced SRS VRU: The enhanced SRS results between FI collisions and VRU collisions highly overlap, which is expected as most of the risk factors identified for FI collisions and VRU collisions are identical.

Figure 8: Comparison of Top 50 Locations between Network Screening Results and Systemic Safety Review Results

Conclusion

This study compared two alternative systemic road safety (SRS) methodologies to determine their advantages and disadvantages.
Although the two methodologies required the same data to conduct the analysis (i.e., collision data, traffic volume data, and infrastructure data), theoretically, the enhanced SRS methodology has several advantages over the core SRS methodology and is expected to result in more reliable outcomes. In this study, a case study was conducted as a part of the York Region Traveller Safety Plan (York TSP), where SRS analyses were conducted for fatal and injury (FI) collisions and vulnerable road user (VRU) collisions at signalized intersections using both the core SRS methodology as well as the enhanced SRS methodology.
The comparative analysis found that, despite a few different hotspots, there is a substantial overlap in the top-ranking locations. However, the enhanced SRS methodology introduces improvements by accounting for the overlap in the impacts of potential risk factors and providing a unique rank for each location, offering more nuanced and accurate results. In terms of the effort required to conduct the SRS analysis, the case study found that the effort required for SRS analysis was broadly comparable between the two methodologies.
The enhanced SRS results were also compared with the network screening results for signalized intersections within the York Region's road network. The comparison discovered a slight overlap between the two sets of results in terms of the top-ranking locations, which is expected as the SRS study and network screening study have different focuses. This observation confirms the need for both screening approaches as they complement each other and provide a better picture of priority locations needing safety improvements.

Bios

Jike Wang, B.SC., P.Eng., RSP₁, is a transportation engineer with more than four years of experience in road safety and traffic operations. He has extensive experience in road safety, which involves historical collision analysis, field investigation, selecting countermeasures, literature review, systemic safety review, and SPF development for multiple clients in Ontario. He is also experienced in conducting traffic operation analysis, such as traffic impact analysis, traffic signal coordination, and speed analysis.

Nelson Costa, Manager, Traffic Safety and Signal Operations, York Region, is a public sector leader in road safety engineering service delivery. Nelson has 23 years of experience contributing to York Region's safe road network.

Ben Kos, LET, C.E.T., RSP1, MITE, Program Manager, Traffic Safety, York Region, is a Road Safety Professional and Limited Licence holder with 16 years of experience in road safety engineering. Ben has extensive experience in public and private sectors leading and conducting road safety assessments and programs.

Soroush Salek, Ph.D., P.Eng., RSP₁, is a Director of Traffic Engineering with almost 20 years of academic and consulting experience in traffic engineering and road safety. His comprehensive experience spans the entire spectrum of safety management projects, including the development of Safety Performance Functions (SPFs), network screening exercises, formulation and assessment of safety countermeasures, and development of Vision Zero plans. His strengths include machine learning, statistical analysis, and prediction models, as well as developing new methodologies to collect, process, and analyze transportation data to improve client decision-making processes.

References

1. Highway Safety Manual, (2010), First Edition, American Association of State Highways and Transportation Officials.
2. Preston, H., Storm, R., Bennett, J.D., Wemple, B. (2013), Systemic Safety Project Selection Tool (Report No. FHWA-SA-13-019), Federal Highway Administration.
3. Thomas, L., Sandt, L., Zegeer, C., Kumfer, W., Lang, K., Lan, B. (2018), Systemic Pedestrian Safety Analysis (Report No. 893), National Cooperative Highway Research Program.