Skip to content
Home page - The Prevention Centre

Primary menu

  • About us
    • Our vision and mission
    • Our approach
    • Our partners
    • Our structure and governance
  • About prevention
    • What is prevention?
    • What are the benefits of prevention for health and wellbeing?
    • What is the burden of chronic disease?
    • What are the economic benefits of prevention?
    • What are the co-benefits of prevention?
  • Our people
  • Our work
    • Research projects
    • Knowledge syntheses
    • A systems thinking approach
    • Dynamic simulation modelling
    • Collaboration for Enhanced Research Impact (CERI)
  • Our impact
    • Research impact
  • Resource hub
  • Events
  • News
  • Engage with us
    • The Chronicle newsletter
    • Emerging Leaders Network
    • Communities of practice
    • Work or study with us
    • Contact us
Search
What are you looking for?

You can search across all our content by topic, methodology and content type.

    Home > Resource hub > The utility of natural experiments for improving health prevention evidence

The utility of natural experiments for improving health prevention evidence

Dr Mel Crane presented her review on the value of natural experiments to health prevention practice, with a focus on obesity prevention at the Public Health Association of Australia’s Preventive Health Conference 2020.

I’m Dr Mel Crane from the University of Sydney and The Australian Prevention Partnership Centre to talk to you about natural experiments

Slide 1

A natural experiment is an experiment – that is, exposure and outcomes are analysed using methods that attempt to make causal inference HOWEVER in this situation the researcher has had no control over exposure to the event but uses the opportunity that is presented. So as an example The current COVID19 crisis has provided the opportunity for many researchers to undertake natural experiment studies to assess a variety of issues

Why are these studies necessary – because planned, controlled experiments like RCTs are rarely appropriate or possible in many public health evaluation situations.  Also, research evidence may be required quickly  – like a policy decision so we need research evidence that can accommodate these opportunistic research needs.  -a RCT study takes time to implement, and the opportunity may be over before the research study is set up.

Slide 2

SO the question is – how can natural experiments help to build evidence for obesity prevention ? Issues like obesity are complex, driven by multiple interrelated factors including environmental, social and cultural factors, not just individual-level behaviour. Prevention requires interventions that target multilevel systemic factors – such as taxation, restrictions on unhealthy food and healthy food availability as well as changes to the built environment, but evidence of the effectiveness of such interventions is small – limited by a real lack of evaluation evidence.

Slide 3

We conducted a research study to investigate the use of natural experiments for obesity prevention and to understand how they were being utilised. We searched peer-reviewed literature for studies that self-described as natural experiments and aimed at improving nutrition, physical activity or obesity specifically. I won’t go into the methods or the characteristics of interventions and evaluation methodologies. For more information, the paper has just been published or you can email me, melanie.crane@sydney.edu.au

Slide 4

The kind of information we were looking for to determine the utility of natural experiments was to determine the need for population health research evidence, as well as things like the ability for researcher manipulation of exposure conditions for research; ethical constraints and practical considerations that may prevent a planned researcher controlled study…

Slide 5

The purpose of the studies were largely to evaluate the effectiveness of an intervention or impact of a policy, with fewer studies aimed at exploring health inequalities or environmental/social determinants:

  • A planned researcher driven research was not always feasible to answer the research question, however partial control over an intervention may be possible in some circumstances to collect baseline data but if a policy is enacted quickly then this would be restricted
  • We note that studies aimed at assessing inequalities in the population would have been unethical as a planned experiment where there was risk of potential harm from intentionally restricting access to medical care, economic support, or randomising groups to receive social benefits or the freedom to migrate
  • A number of practical considerations also necessitated the need for a natural experiment.

Slide 6

Natural experiments enable the evaluation of processes and outcomes of policies & interventions within the real-world complex social and political conditions they naturally operate. They offer opportunistic evidence where RCT methods may be impossible due to timing or exposure. They offer flexibility to use for a range of research questions, intervention settings and scale and evaluation methods – however, this increases the difficulty for making comparisons across studies.

They are often criticised for their inability to eliminate bias and bias due to confounding is a concern. We caution that deciding against natural experiments, as a result, ignores the contribution they can make to the overall evidence generation, particularly in regards to the complexity of real-world intervention evidence

But this can be reduced by ensuring good study design – such as having a control group comparator. We found less than one-third of the studies we assessed had a defined exposure and control group. Given these studies are often opportunistic, finding an unexposed group may be difficult. This is easier in environmental interventions where there is greater potential for researchers to be involved in establishing baseline data points. This is harder with a policy evaluation. Some studies have used modelling simulations to create a control group comparator, others employed step-wedge designs.

In response to the complexity of the obesity epidemic, obtaining policy and intervention impact evidence is critical and natural experiments may provide innovative translational research evidence on obesity policy interventions that have been missing.

 

Dr Crane argued that planned controlled experiments may be infeasible, inappropriate or inopportune in resolving public health research gaps after conducting a systematic literature search from January 1997 to December 2017. The project identified 46 population health studies that were assessed by examining intervention and evaluation characteristics and critically investigating the utility for evidence generation.

Natural experiment studies were predominantly conducted on policy interventions, such as changes to food labelling, food advertising or taxation, or built environment interventions such as the impact of built infrastructure on physical activity or healthy food access, rather than community or individual behaviour interventions. These were largely at the city-scale or national and state level. Research designs applied to natural experiment studies included quasi-experimental, but also pre-experimental and non-experimental methods.

Dr Crane concluded that natural experiments provide utility and versatility for generating evidence for complex health issues like obesity, particularly when unexpected opportunities arise to gather evidence.

Details

DATE 17 Jun 2020

TYPE Videos

Prevention experts

  • Associate Professor Melanie Crane

    The University of Sydney
    Headshot of Melanie Crane

Prevention methodologies

  • Implementation, evaluation and scale up ,
SHARE
TweetLinkedInFacebookEmail

More resources

  • Lessons from a decade of The Australian Prevention Partnership Centre

    Resource category: Videos Videos
    Date 02 May 2023
  • Multisectoral collaboration and its impact on health and wellbeing

    Resource category: Findings Brief Findings Brief
    Date 05 Apr 2023
  • Community of practice: Scaling the voice of older adults

    Resource category: Videos Videos
    Date 15 Mar 2023

Acknowledgement of Country

The Australian Prevention Partnership Centre acknowledges Aboriginal and Torres Strait Islander peoples as the First Australians and Traditional Custodians of the lands where we live, learn, and work.

Newsletter sign-up

Newsletter

Follow the Prevention Centre

Follow us

  • TwitterTwitter Icon
  • LinkedinLinkedin Icon
  • YoutubeYoutube Icon
  • PodcastPodcast Icon

Funding Partners

The Australian Prevention Partnership Centre is funded by the NHMRC, Australian Government Department of Health, ACT Health, Cancer Council Australia, NSW Ministry of Health, Wellbeing SA, Tasmanian Department of Health, and VicHealth. The Australian Government also contributed through the Medical Research Future Fund. Queensland Health became a financial contributor in 2022. The Prevention Centre is administered by the Sax Institute.

©2023 The Sax Institute.

Footer Navigation

  • Contact us
  • Privacy Policy
  • Disclaimer
  • Copyright
  • Terms of Use
  • Sitemap

BROWSER NOT SUPPORTED

It seems you’re using a browser that we no longer support, so this site might not function as expected.

Please use Chrome, Safari, Firefox or Edge to visit this website.

Share this ArticleLike this article? Email it to a friend!

Email sent!