Latrobe Safe Communities: Evaluation of a Local Level Injury Prevention Program - 1996-2000
Monash University Accident Research Centre - Report #190 - 2002
Full report in .pdf format [551KB]
Authors: L. Day, E. Cassell & J. Lough
The aim of this study was to evaluate the second phase of the La Trobe Safe Communities (LTSC) program, formerly the Latrobe Valley Better Health Injury Prevention Program, a community based injury prevention program in the Latrobe Valley, Victoria. The program began as a stand-alone project in 1992, and in July 1996 was incorporated into the City of La Trobe as an activity of local government.
The evaluation was an observational study covering January 1996 to December 2000 and including pre and post-intervention observations, some of which were also available for a comparison region. Process measures included key informant interviews with local organisation representatives. Impact evaluation relied on self-reported changes in injury risk and protective factors, gathered by a random telephone survey. Outcome evaluation was based on four years of emergency department injury surveillance data, and twelve years of hospitalisation data, for LTSC and a comparison region.
The program continued to build strategic partnerships and implemented promotional, educational and policy-changing activities across a range of injury types. There was a 22% increase in the proportion of households able to list home safety features (p<0.001). However, the proportion reporting purchase of a safety item in the previous 12 months decreased by 14% (p=0.02). The LTSC program region compared favourably with non-metropolitan Victoria, having a statistically significantly greater proportion of households with smoke detectors installed, and hand rails present in the bathroom, and a lower proportion with hot water capable of scalding. The age standardised emergency department presentation rate for unintentional injury increased by 2% per year on average (p=0.40), compared with a 7% increase in the comparison region (p<0.0001). The age standardised emergency department presentation rate for intentional injury decreased by 4.7% per year on average (p=0.54), compared with a 12% increase (p<0.0001) in the comparison region. When controlling for pre-intervention trends, statistical modelling indicated that the unintentional injury hospitalisation rate in the program region decreased significantly by 9.2% on average relative to the comparison region. In the case of intentional injury, the rate increased significantly by 13.1% on average relative to the comparison region.
The LTSC program appeared to have been successfully incorporated into the local government structure, and despite some methodological limitations with the selected comparison region, the injury rate trends provide some, perhaps limited, support for the program being associated with injury reductions.
The community based approach to all age all injury prevention has been applied increasingly in various parts of the world, following the first successful reports in Sweden during the early 1980's. The La Trobe Safe Communities (LTSC) program, formerly the Latrobe Valley Better Health Injury Prevention Program, is a community based program, modelled on the Swedish experience, in the Latrobe Valley, Victoria. The program began as a stand-alone project in 1992, initially with funding from the Victorian Health Promotion Foundation. In July 1996 the program was incorporated into La Trobe Shire and became an activity of local government.
An evaluation of the first four years (first phase evaluation) was completed in 1997. We now report on the evaluation of the second phase of the program, from June 1996. We have attempted to address the lack of comparison data apparent in the first phase evaluation. This was done by utilising comparable impact measures which have become available from the Australian Bureau of Statistics and the Victorian Department of Human Services, and through the availability of emergency department and hospital admission injury data for similar regions of Victoria.
The evaluation is an observational study which includes pre and post intervention observations, some of which were also available for a comparison region. Process, impact and outcome measures were included. The operational region for the program was defined by the former Victorian local government areas of the Cities of Moe, Morwell and Traralgon, and the Shire of Traralgon, and for the purposes of the evaluation the program region was defined on the basis of postcode (3825, 3840,3842, 3844, 3869, 3870). A comparison region was defined using the Accessibility/Remoteness Index of Australia (ARIA; Dept of Health and Aged Care, 1999) to identify areas in Victoria with the same index as the program region.
Data for process evaluation was obtained from reports to the Management Committee, an in-depth interview with the program officer, and through interviews with key informants from local organisations.
Data for impact evaluation (eg., changes to household safety features) was gathered using random household telephone surveys conducted in the program region in 1992, 1995 and 1999. In addition, two sources of comparison data for impact measures were identified. The first was a home safety survey which had been conducted by the Australian Bureau of Statistics, and from which some questions were included in the 1999 La Trobe survey. The second was the 1999 Victorian Population Health Survey which included one question which was also asked in the La Trobe surveys. Differences between responses in the pre and post-intervention surveys were tested using the chi-square function in Excel and SPSS 10.0. Proportional differences were tested using techniques outlined by Swinscow (1996).
Outcome evaluation included three sources of injury data: self-reported injury, emergency department presentations, and hospital admissions.
Self reported injury data was gathered in the telephone surveys. Injury rates for the two week period were calculated using the total population surveyed as the denominator.
Emergency Department injury presentation data for LTSC and the comparison region were obtained from the Victorian Emergency Minimum Dataset (VEMD) to which some 28 hospitals across Victoria provide data. Hospital admissions data for injury cases was obtained from the Victorian Admitted Episodes Database (VAED) which holds information relating to all Victorian public hospital (and more recently private hospital) admissions.
Consistent with the expansion of focus to all age all injury for the 1996-2000 period, analysis was not conducted by specific injury categories as was the case for the first phase evaluation. Injury data from the VEMD and VAED were used to calculate rates for the program and comparison regions. Population data for the denominators were obtained directly from the Australian Bureau of Statistics (ABS). Injury rates were standardised to the Victorian population by the direct method.
Trends in emergency department injury rates were determined using a log-linear regression model of rate data assuming a Poisson distribution of injuries. Trends in injury hospitalisation rates were examined with an offset log-linear regression model, with the injury frequencies included as a random variable and an age standardisation factor (derived from the age standardised rates) included as a fixed factor in the model.
Collaborative partnerships with key organisations had continued to develop, following some initial confusion about the roles and responsibilities of the Project Office and the Advisory Committee following incorporation of the program into the local government structures. Key factors facilitating working partnerships were the adoption of a team approach by Council based on good relationships between the organisations, and good two-way communication particularly between the Community Safety Officer and Advisory committee members. All external key informants indicated that their organisations were keen to maintain ongoing representation on the Advisory Committee and a partnership approach on community safety initiatives.
The household telephone surveys revealed that in 1999 respondents were more likely to be able to list their home's safety features (79.3%, 96.5%, P<0.001), and were less likely to be able to list ways of improving their home than in 1995 (47.3%, 35.0%, P<0.001). In 1999 respondents were also significantly less likely to have purchased safety items in the last 12 months (45.8%, 39.3%, P=0.02). The average number of safety features per household increased marginally (2.2, 2.6, P<0.001).
The LTSC program region compared favourably with non-metropolitan Victoria, having a statistically significantly greater proportion of households with smoke detectors installed (96.7%, 85.3%, P<0.001), and hand rails present in the bathroom (20.2%, 11.9%, P<0.001), and a lower proportion with hot water capable of scalding (70.9%, 75.7%, P<0.001). La Trobe households also reported a marginally higher safety item usage across all categories (excluding personal protective equipment, such as recreation or sports items) when contrasted with the comparison region.
Overall, the self-reported injury rate decreased by 4.6% from 1992 (62.7 per 1000 persons) to 1999 (59.8 per 1000 persons). The significant decrease in self-reported injury observed in the first phase evaluation was not maintained in the second phase. In fact, between 1995 and 1999 the rate increased, although this was not statistically significant (48.2 per 1000, 59.8 per 1000, P=0.15).
The emergency department presentation rate for unintentional injury in both the program and comparison regions increased between 1997 and 2000. The estimated annual percentage increase in the program region (2%) was less than the increase observed in the comparison region (7%). Further, the increased trend in the program region was not statistically significant (P=0.40), whereas the increased trend in the comparison region was statistically significant (P<0.0001).
The emergency department presentation rate for intentional injury decreased in the program region between 1997 and 2000, in contrast to an increase in the comparison region. The estimated annual percentage decrease in the program region was 4.7%, compared with a 12% increase in the comparison region. The decreased trend in the program region was not statistically significant (P=0.54), however, the increased trend in the comparison region was statistically significant (P<0.0001).
When controlling for the pre-intervention trends, it was estimated from the statistical model that the unintentional injury hospitalisation rate in the program region significantly decreased by 9.2% (P=0.0002) on average relative to the comparison region during the intervention period. In the case of intentional injury, the rate in the program region increased significantly by 13.1% (P=0.031) on average relative to the comparison region during the intervention period.
This current evaluation has a number of limitations in common with the first phase evaluation. These include that the community was self-selected, demographic differences were apparent in the samples obtained for the telephone surveys, and measurement of the outcome was at the level of the individual rather than at the community level, the level at which the intervention was delivered. Fewer systematic process measures were available, limiting the conclusions which can be drawn about implementation of some strategies such as development and delivery of relevant resources (such as safety audit tools, subsidy schemes), and changes to local government policies and practices.
A significant enhancement of the evaluation design was intended with the definition of a comparison region in rural Victoria, and the availability of some impact and outcome measures for this region. However, socio-economic differences between La Trobe and the comparison region, some unusual features of the hospitalisation data, and the differences in the pre-program trends between the La Trobe and comparison regions combined to weaken the validity of the comparison region and thereby complicate the interpretation of injury rate trends. However, it may have proven difficult to find a more appropriate comparison region in which emergency department surveillance data were also being captured.
Changes in injury rates were observed during the intervention period. Self-reported injury rates decreased marginally overall from 62.7 per 1000 persons in 1992 to 59.8 per 1000 persons in 1999. This should be interpreted with caution as age standardisation has not been possible. However, changes were also observed in emergency department presentation and hospitalisation rates that were age standardised. With respect to unintentional injury, emergency department presentation rates in the program region, although increasing, were not increasing to the same extent as observed in the comparison region. In addition, significant decreases in the hospitalisation rates for unintentional injury in the program region (relative to the comparison region) were observed. Decreases were also observed for emergency department presentation rates for intentional injury, although there was no accompanying significant decrease in hospitalisation rates for intentional injury.
Alternative explanations for these observed effects in injury rates were canvassed. These included changes to health care delivery in the program region, such as the introduction of new inpatient and outpatient services not captured by the surveillance system, systematic improvements in data capture by emergency departments in the comparison region above any such improvement in capture at Latrobe Regional Hospital, and the removal of previously operating injury prevention initiatives in the comparison region. It was not possible to fully accept or discount any of these explanations, although on balance there did not appear to be a convincing case for any of them.
Thus in the absence of plausible alternative explanations, the evidence points to some reduction in injury rates, or at least a moderating effect on injury rate increases, associated with the La Trobe Safe Communities Program.
The results from the impact evaluation (i.e. changes in the risk and protective factors) provide little explanation for the observed reductions in injury. The three household surveys conducted in the program region suggest that while knowledge about home safety features and about where to purchase safety items improved, there had been little change in the home environment. Further, while the program region performed better than the comparison region on a number of household safety items, the actual proportions of households reporting the presence of various safety items was still relatively low, with the exception of smoke detectors and fire protection equipment. It is questionable that these measured changes in injury risk and protective factors contributed substantially to the observed injury reductions. It was noted, however, that the injury reductions may have been achieved through strategies not specifically measured in the household surveys.
The La Trobe Safe Communities program appears to have been successfully incorporated into the local government structure, and the injury rate trends provide some, perhaps limited, support for the program having an effect on injury outcome.
The following recommendations were made:
- Exploration of possible alternative, more appropriately matched, comparison regions could be considered prior to ongoing monitoring of trends in emergency department presentation and injury hospitalisation rates in the program and comparison regions.
- Regular reporting of program activities and strategy implementation should be detailed enough to identify links between these and injury outcome. Such reports should include, if possible, the injury prevention activities delivered by partner organisations with whom the program has actively engaged.
- The feasibility of a cost effectiveness study for the La Trobe Safe Communities program should be established.
- Statistical approaches to addressing some of the limitations of the quasi-experimental design should be explored for their potential to improve the methodological rigour of community trials in injury prevention research
Sponsoring Organisations: the City of LaTrobe and the Victorian Health Promotion Foundation