Original ResearchThe Cold Weather Plan evaluation: an example of pragmatic evidence-based policy making?
Introduction
An evaluation of the Cold Weather Plan (CWP) for England was undertaken in April 2012 to generate the basis for further revisions.1 The CWP, first published in November 2011, was developed to reduce avoidable winter deaths and illnesses by prompting actions to support people vulnerable to extreme cold weather and alleviate pressures on health and social care systems.2 The CWP was published with a detailed companion document, ‘Making the Case’ providing further background to the impact of cold on health as well as an Equality Assessment.3 On average over the past three years prior to the publication of the CWP, there have been approximately 29,320 excess deaths in England and Wales each winter (2008/09–2010/11) between November and March when compared to the rest of the year.4 Cold weather aggravates many chronic illnesses and increases the risk of acute illnesses and infections like strokes, heart attacks, thrombosis, bronchitis, pneumonia and asthma.5, 6 A significant proportion of excess mortality and morbidity is thought to be preventable.7, 8
It is widely considered good practice to formulate and revise policy on the basis of the best available evidence, but what constitutes such evidence and how this informs policy-making is often contested. Evidence-based policy-making is described as an approach that ‘helps people make well informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation’.9 The UK Cabinet Office attempted to define its understanding of evidence in its 1999 White Paper ‘Modernising Government’, according to which evidence is ‘expert knowledge; published research; existing research; stakeholder consultations; previous policy evaluations; the internet; outcomes from consultation; costings of policy options; outputs from economic and statistical modelling’.10 Therefore the breadth of what is considered evidence is wide and dynamic.11 Despite the broad definition, there is a distinct hierarchy of evidence with systematic reviews, meta-analyses and randomized controlled trials typically at the top. However there is increasing recognition that the hierarchy of evidence is a difficult construct to apply in evidence-based medicine and even more so in public health, and the promotion of typologiesg rather than hierarchies may be more useful.12, 13 Indeed the question of process of service delivery (i.e. does it work?), which is the area of interest in the CWP evaluation, is best answered through qualitative research and survey designs.10, 14
A multimethod evaluation of the CWP was undertaken to ascertain the extent of awareness of the plan, how successfully it had been disseminated, experiences of implementing the plan, generating the basis for future revisions. It was part of the ongoing process of engaging stakeholders and determining the potential needs of organisations that implemented and utilized the plan. It consists of six different strands, the first three establishing the context in which the plan was implemented and the last three generating recommendations for the next iteration of the plan.
Section snippets
Methods
As this was the first year of the implementation of the CWP, a formative ‘soft touch’ approach has been taken for employing multiple methods to generate the basis for future improvements, and to establish a background summary of cold weather related morbidity, mortality and weather conditions during the 2011–12 winter. A protocol was developed by the Health Protection Agency (HPA; the HPA became part of Public Health England (PHE) on 1 April 2013) in collaboration with the Department of Health
Results
The UK mean temperature for 2011–12 winter was 4.5 °C, much milder than the last three winters.1 This reflected in the syndromic surveillance data where there were no prolonged increases in cold weather related syndromic indicators during the winter.19 Weekly age-group and region specific all-cause excess mortality, which is calculated using the EuroMOMO (European Mortality Monitoring Project) algorithm correcting for reporting delays, was detected in weeks 51–52, 2011 and 7–8, 2012 in England
Discussion
The CWP evaluation generated many useful recommendations, which shaped and influenced the format, content and logistics of the revised plan, and helped modify the Met Office Cold Weather Alerting System. This was the first evaluation of the CWP since its publication and it clearly demonstrated high degrees of awareness and engagement among professionals involved in implementing and/or disseminating it. The survey and subsequent workshop revealed that while some actions in the plan were already
Acknowledgements
The authors would like to acknowledge the following individuals and teams for their input and on-going collaboration with the Cold Weather Plan and its evaluation:
- •
The HPA Real-time Syndromic Surveillance Team, – Sally Harcourt, Helen Hughes, Paul Loveridge, Roger Morbey, Gillian Smith, Sue Smith;
- •
The Respiratory Surveillance Team – Richard Pebody, Nick Andrews;
- •
The Met Office – particularly Katie Russell, Patrick Sachon, Yolanda Clewlow and Caroline Prouse.
Ethical approval
Formal ethical approval was not required
References (24)
Assessment of syndromic surveillance in Europe
Lancet
(2011)- et al.
Evaluation report – Cold Weather Plan for England 2011–12. Extreme events and health protection
(September 2012) Cold Weather Plan for England: protecting health and reducing harm from severe cold weather
(2011)Cold Weather Plan for England: making the case: why cold weather planning is essential to health and well-being
(2011)Excess winter mortality in England and Wales, 2010/11 (provisional) and 2009/10 (final)
(2011)- et al.
Early increases in ischaemic heart disease mortality dissociated from and later changes associated with respiratory mortality after cold weather in south east England
J Epidemiol Community Health
(1997) Cold exposure and winter mortality from ischaemic heart disease, cerebrovascular disease, respiratory disease, and all causes in warm and cold regions of Europe
Lancet
(1997)Winter kills
(2010)- (2012)
‘Is evidence-based government possible?’ Jerry Lee Lecture, presented at the 4th annual Campbell collaboration Colloquium
(2004)