Propensity score matching for selection of local areas as controls for evaluation of effects of alcohol policies in case series and quasi case-control designs

Objectives

Area-level public health interventions can be difficult to evaluate using natural experiments. We describe the use of propensity score matching (PSM) to select control local authority areas (LAU) to evaluate the public health impact of alcohol policies for (1) prospective evaluation of alcohol policies using area-level data, and (2) a novel two-stage quasi case–control design.

Study design

Ecological.

Methods

Alcohol-related indicator data (Local Alcohol Profiles for England, PHE Health Profiles and ONS data) were linked at LAU level. Six LAUs (Blackpool, Bradford, Bristol, Ipswich, Islington, and Newcastle-upon-Tyne) as sample intervention or case areas were matched to two control LAUs each using PSM. For the quasi case–control study a second stage was added aimed at obtaining maximum contrast in outcomes based on propensity scores. Matching was evaluated based on average standardized absolute mean differences (ASAM) and variable-specific P-values after matching.

Results

The six LAUs were matched to suitable control areas (with ASAM < 0.20, P-values >0.05 indicating good matching) for a prospective evaluation study that sought areas that were similar at baseline in order to assess whether a change in intervention exposure led to a change in the outcome (alcohol related harm). PSM also generated appropriate matches for a quasi case–control study – whereby the contrast in health outcomes between cases and control areas needed to be optimized in order to assess retrospectively whether differences in intervention exposure were associated with the outcome.

Conclusions

The use of PSM for area-level alcohol policy evaluation, but also for other public health interventions, will improve the value of these evaluations by objective and quantitative selection of the most appropriate control areas.