Understanding barriers to evaluating complex interventions


Project title: Barriers to, and strategies for, evaluating complex interventions

Project start date: January 2015

End date: June 2016

This project is finished.

What is the issue?

Primary prevention interventions are often highly complex. They frequently involve multiple interventions targeting numerous behaviours and are increasingly aimed at producing systems- or population-level changes. These interventions must be rigorously evaluated to determine their effects.

The randomised controlled trial (RCT), the traditional gold standard intervention design, may not be the most appropriate design for evaluating complex interventions. While the RCT is well suited to evaluating interventions targeting discrete change at an individual level, it has limitations in interventions that are multi-faceted and involve systems-level change.

A number of well-recognised intervention designs are able to evaluate complex interventions but organisations and industry groups do not always use these evaluation methods. The specific reasons for this remain relatively unknown and unaddressed.

If complex interventions are to be appropriately evaluated, we need to understand why organisations aren’t routinely conducting evaluations.

How did the project address the issue?

The project developed an understanding of the potential barriers to organisations conducting routine evaluation of complex interventions and developed strategies to overcome these barriers.

Steps in the project included:

  • Review of the literature about potential barriers to evaluating complex interventions
  • Using qualitative and quantitative methods to gather data from a range of national organisation and industry groups to explore these barriers
  • Conducting interviews and forums with experts in complex interventions, (including researchers, practitioners and policy makers) and survey a wider sample of research experts to develop strategies to overcome barriers.

What were the outcomes?

The series of papers resulting from this research argue that public health policy should be and can be evaluated robust manner. To facilitate discussion and debate regarding the imperative to increase the proportion of public health policy, the series of papers will be published in policy and public health journals that are accessed by researchers and public health policy makers.

Laureate Professor Rob Sanson-Fisher has presented the concepts contained in the papers to a range of academics and public health policy decision makers in an effort to advocate for public health policy evaluation and to encourage debate on this topic.

Relevance for practice

The project provided policy makers with evidence of the need to evaluate complex public health interventions. This can inform their planning by helping them to make better judgments about the effectiveness of their programs.

 

Project lead

  • Laureate Professor Rob Sanson-Fisher, University of Newcastle

Project team

This project was funded by the NHMRC, Australian Government Department of Health, NSW Ministry of Health, ACT Health and the HCF Research Foundation.

 

NHMRC, Australian Government Department of Health, NSW Government Health, HCF Research Foundation, ACT Government Health, hosted by Sax Institute

This project has now been completed. Several papers are under review.

2018