Elsevier

Public Health

Volume 128, Issue 7, July 2014, Pages 619-627
Public Health

Original Research
The Cold Weather Plan evaluation: an example of pragmatic evidence-based policy making?

https://doi.org/10.1016/j.puhe.2014.05.006Get rights and content

Abstract

Objectives

An evaluation of the Cold Weather Plan (CWP) for England 2011–2012 was undertaken in April 2012 to generate the basis for further revisions. It is widely considered good practice to formulate and revise policy on the basis of the best available evidence. This paper examines whether the evaluation is an example of pragmatic evidence-based policy-making.

Study design

A process evaluation with a formative multimethods approach.

Methods

An electronic survey and national workshop were conducted alongside the production of a number of summary reports from the Health Protection Agency surveillance systems and Met Office meteorological data. The Department of Health and the Met Office were consulted on how the evaluation recommendations shaped the revised CWP and Met Office Cold Weather Alerting System respectively.

Results

The Cold Weather Plan survey had 442 responses, a majority from Local Authorities, and from all regions of England. Thematic analysis generated qualitative data, which along with feedback from the workshop were synthesized into six main recommendations. Reviewing the new CWP and the Met Office Cold Weather Alerting System revealed significant modifications on the basis of the evaluation.

Conclusions

The evaluation sets the context for cold weather and health during the 2011–2012 winter. This study shows that the CWP 2012–2013 was revised on the basis of the national evaluation recommendations and is an example of pragmatic evidence-based policy-making.

Introduction

An evaluation of the Cold Weather Plan (CWP) for England was undertaken in April 2012 to generate the basis for further revisions.1 The CWP, first published in November 2011, was developed to reduce avoidable winter deaths and illnesses by prompting actions to support people vulnerable to extreme cold weather and alleviate pressures on health and social care systems.2 The CWP was published with a detailed companion document, ‘Making the Case’ providing further background to the impact of cold on health as well as an Equality Assessment.3 On average over the past three years prior to the publication of the CWP, there have been approximately 29,320 excess deaths in England and Wales each winter (2008/09–2010/11) between November and March when compared to the rest of the year.4 Cold weather aggravates many chronic illnesses and increases the risk of acute illnesses and infections like strokes, heart attacks, thrombosis, bronchitis, pneumonia and asthma.5, 6 A significant proportion of excess mortality and morbidity is thought to be preventable.7, 8

It is widely considered good practice to formulate and revise policy on the basis of the best available evidence, but what constitutes such evidence and how this informs policy-making is often contested. Evidence-based policy-making is described as an approach that ‘helps people make well informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation’.9 The UK Cabinet Office attempted to define its understanding of evidence in its 1999 White Paper ‘Modernising Government’, according to which evidence is ‘expert knowledge; published research; existing research; stakeholder consultations; previous policy evaluations; the internet; outcomes from consultation; costings of policy options; outputs from economic and statistical modelling’.10 Therefore the breadth of what is considered evidence is wide and dynamic.11 Despite the broad definition, there is a distinct hierarchy of evidence with systematic reviews, meta-analyses and randomized controlled trials typically at the top. However there is increasing recognition that the hierarchy of evidence is a difficult construct to apply in evidence-based medicine and even more so in public health, and the promotion of typologiesg rather than hierarchies may be more useful.12, 13 Indeed the question of process of service delivery (i.e. does it work?), which is the area of interest in the CWP evaluation, is best answered through qualitative research and survey designs.10, 14

A multimethod evaluation of the CWP was undertaken to ascertain the extent of awareness of the plan, how successfully it had been disseminated, experiences of implementing the plan, generating the basis for future revisions. It was part of the ongoing process of engaging stakeholders and determining the potential needs of organisations that implemented and utilized the plan. It consists of six different strands, the first three establishing the context in which the plan was implemented and the last three generating recommendations for the next iteration of the plan.

Section snippets

Methods

As this was the first year of the implementation of the CWP, a formative ‘soft touch’ approach has been taken for employing multiple methods to generate the basis for future improvements, and to establish a background summary of cold weather related morbidity, mortality and weather conditions during the 2011–12 winter. A protocol was developed by the Health Protection Agency (HPA; the HPA became part of Public Health England (PHE) on 1 April 2013) in collaboration with the Department of Health

Results

The UK mean temperature for 2011–12 winter was 4.5 °C, much milder than the last three winters.1 This reflected in the syndromic surveillance data where there were no prolonged increases in cold weather related syndromic indicators during the winter.19 Weekly age-group and region specific all-cause excess mortality, which is calculated using the EuroMOMO (European Mortality Monitoring Project) algorithm correcting for reporting delays, was detected in weeks 51–52, 2011 and 7–8, 2012 in England

Discussion

The CWP evaluation generated many useful recommendations, which shaped and influenced the format, content and logistics of the revised plan, and helped modify the Met Office Cold Weather Alerting System. This was the first evaluation of the CWP since its publication and it clearly demonstrated high degrees of awareness and engagement among professionals involved in implementing and/or disseminating it. The survey and subsequent workshop revealed that while some actions in the plan were already

Acknowledgements

The authors would like to acknowledge the following individuals and teams for their input and on-going collaboration with the Cold Weather Plan and its evaluation:

  • The HPA Real-time Syndromic Surveillance Team, – Sally Harcourt, Helen Hughes, Paul Loveridge, Roger Morbey, Gillian Smith, Sue Smith;

  • The Respiratory Surveillance Team – Richard Pebody, Nick Andrews;

  • The Met Office – particularly Katie Russell, Patrick Sachon, Yolanda Clewlow and Caroline Prouse.

Ethical approval

Formal ethical approval was not required

References (24)

  • Triple S Project

    Assessment of syndromic surveillance in Europe

    Lancet

    (2011)
  • A. Ghosh et al.

    Evaluation report – Cold Weather Plan for England 2011–12. Extreme events and health protection

    (September 2012)
  • Department of Health

    Cold Weather Plan for England: protecting health and reducing harm from severe cold weather

    (2011)
  • Department of Health

    Cold Weather Plan for England: making the case: why cold weather planning is essential to health and well-being

    (2011)
  • Office for National Statistics

    Excess winter mortality in England and Wales, 2010/11 (provisional) and 2009/10 (final)

    (2011)
  • G.C. Donaldson et al.

    Early increases in ischaemic heart disease mortality dissociated from and later changes associated with respiratory mortality after cold weather in south east England

    J Epidemiol Community Health

    (1997)
  • Cold exposure and winter mortality from ischaemic heart disease, cerebrovascular disease, respiratory disease, and all causes in warm and cold regions of Europe

    Lancet

    (1997)
  • L. Donaldson

    Winter kills

    (2010)
  • S.C. Davies
    (2012)
  • P. Davies

    ‘Is evidence-based government possible?’ Jerry Lee Lecture, presented at the 4th annual Campbell collaboration Colloquium

    (2004)
  • Cabinet Office

    Modernising government

    (1999)
  • L. Shaxson

    ‘Is your evidence robust enough? Questions for policy makers and practitioners’ Evidence and Policy

    J Res Debate Pract

    (2005)
  • Cited by (0)

    View full text