Data Simulation to Evaluate Safety Science Designs

The focus of outcome evaluations in safety science is typically the measurement of changes in an observed event, typically an injury or crash due to countermeasure implementation. Countermeasures in the road safety space that have been extensively evaluated include automated enforcement, road safety infrastructure programs and licensing initiatives such as graduated licensing for novice drivers.

Fully experimental designs, such as the Randomised Controlled Trial (RCT) are typically not achievable to evaluate road safety countermeasures, as generally sites or populations for treatment are not chosen randomly, but rather based on injury or crash history. A non-randomised approach to countermeasure evaluation simply compares crash rates before implementation to those after, with a variation adjusting the comparison explicitly for the effects of known confounders. Another non-randomised design is called a quasi-experimental design. It incorporates a control and compares the crash rates before and after program implementation in the ‘treated area’ (the ‘treatment’) to the before and after crash rates in the control area.

The issue of selection of the control area to adequately reflect the crash effects of all non-treatment factors at the treatment sites is an important one. Both non-randomised study designs described are potentially subject to the problems of selection bias, the most common of these being regression-to-the-mean (RTM), which can arise when treatment sites are not chosen at random. A method put forward to address RTM is Empirical-Bayes, however this method also has potential biases as well as significant limitations in its application due to required data sources not being available.

This project aims to use data simulation to evaluate safety science evaluation designs, examining their effectiveness in adequately accommodating RTM and controlling for other exogenous factors to the treatment on injury outcomes. Approaches such as propensity analysis, a statistical approach that attempts to reduce treatment-control selection bias will be examined.

Main supervisor: Professor Stuart Newstead

Associate supervisor: Dr Angelo D’Elia