Elsevier

Public Health

Volume 122, Issue 1, January 2008, Pages 92-98
Public Health

Original Research
Critical appraisal for public health: A new checklist

https://doi.org/10.1016/j.puhe.2007.04.012Get rights and content

Summary

Objectives

There have been a number of attempts to develop critical appraisal tools, but few have had a public health focus. This paper describes a new checklist with public health aspects.

Study design

Review of previous appraisal instruments and pilot test of new checklist.

Methods

Criteria of particular reference to public health practice were added to well-established appraisal criteria. The checklist was piloted with 21 public health professionals, research staff or postgraduate students.

Results

The checklist is organized using the ‘ask’, ‘collect’, ‘understand’ and ‘use’ categories of the Population Health Evidence Cycle. Readers are asked to assess validity, completeness and transferability of the data as they relate to: the study question; key aspects of the methodology; possible public health implications of the key results; and the implications for implementation in their own public health practice. Of the 21 public health professionals that piloted the checklist, 20 said that they found the checklist useful and 18 would use it or recommend it in the future. Participants were prepared to commit to the majority of the questions, and there was good agreement with a consensus of ‘correct’ answers.

Conclusions

The public health critical appraisal checklist adds public health aspects that were missing from previous critical appraisal tools.

Introduction

Critical appraisal is a key skill required by health professionals and policy makers. There have been a number of attempts to develop critical appraisal tools, but few have had a public health focus. For example, neither the Users’ Guide to Evidence-based Practice, originally published as a series in the Journal of the American Medical Association,1 nor the UK Critical Appraisal Skills Programme,2 nor the book by Greenhalgh, originally published as a series of papers in the British Medical Journal,3 have guidelines that relate to public health interventions. A number of tools have been published on easily accessible websites with an emphasis on public health and health promotion interventions.4, 5, 6, 7 One5 includes a reference to the data-collection instrument used by the US Guide to Preventive Services,8 and another includes summary points for methodology quality for primary studies.6 However, much of the emphasis is on tools to assess reviews rather than individual studies, and only one includes public health issues.8 Most guidelines suggest that a different set of questions should be asked for each study type. However, the authors’ experience over many years with a single checklist to cover key aspects of study design is very positive,9 and a generic instrument may be attractive for those wishing to keep a structure for critical appraisal in their minds, for example, for the purposes of an examination.

Critical appraisal of papers relating to public health issues may require a different focus from papers relating purely to clinical topics. For example, there may be particular questions that will need to be answered in deciding whether to introduce public health interventions that relate to the context in which it is to be implemented, resource implications and appropriateness of the intervention. Public health practitioners appraising the policy relevance of studies exploring the link between risk factors and health outcomes need to take into account the potential population impact of preventive measures.10 A critical appraisal tool for public health practitioners needs to include these considerations.

Evidence-based medicine is often criticized for focusing on the assessment of efficacy and neglecting the importance of ‘implementation’,11 and a public health appraisal tool should place particular importance on how the results can be implemented. The tool also needs to be user friendly at all levels of experience to be of maximum use to public health teams, which include practitioners with varied levels of prior training.

This paper describes the development, characteristics and piloting with public health practitioners of a new framework for the appraisal of evidence of relevance to public health, through a critical appraisal checklist to help review papers and reports of public health research. The checklist has been designed to be used in conjunction with an evidence hierarchy that is relevant to public health rather than to clinical practice.10, 12

Section snippets

Development of the public health critical appraisal checklist

The appraisal method for study methodologies presented in this paper relates to well-established criteria of study methodology, as relevant to the type of study being reported. For this, previous critical appraisal criteria have been adapted. Various explorations of public health evidence were referred to whilst deciding which additional criteria are of particular reference to public health.

Petticrew and Roberts13 adapted a framework from Muir Gray, and suggested that one should first identify

The checklist

Readers are asked to appraise the validity, completeness and transferability of the data presented as they relate to:

  • the study question (‘ask’);

  • the study design, sampling, exposures, outcomes, confounders and other aspects of internal validity relevant to the study type (‘collect’);

  • the interpretation and population relevance of the results (‘understand’); and

  • the implications for implementation in their own population and public health practice (‘use’).

A glossary of terms is included at the end

Discussion

The public health critical appraisal checklist includes public health aspects that are missing from previous critical appraisal tools. As mentioned above, most critical appraisal tools are not designed to incorporate the critical appraisal of public health aspects of individual studies. This includes the majority of tools available for reviewing public health interventions. The major exception to this is the data-collection instrument used by the US Guide to Preventive Services,8 which is too

Conclusions

The checklist incorporates public health aspects into previous critical appraisal tools. It was piloted among people working in public health, who said that it was useful and would recommend its use. It is the authors’ hope that it may find a place as an addition to other critical appraisal checklists that do not include a public health perspective.

Ethical approval

Not applicable.

Funding

None.

Competing interests

None.

References (24)

  • S. Zaza et al.

    et al

    Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med

    (2000)
  • R.F. Heller et al.

    Putting the public back into public health. Part II. How can public health be accountable to the public? Public Health

    (2003)
  • Evidence Based Medicine Working Group. Users’ guide to evidence-based practice. Centre for Health Evidence; 2005....
  • Critical appraisal skills programme. Public Health Resource Unit; Oxford 2005. Available at:...
  • Greenhalgh T. How to read a paper: the basics of evidence based medicine. BMJ Books;...
  • Cochrane Health Promotion and Public Health Field, Victoria, Australia 2006. Available at:...
  • Community Guide Branch. Guide to community preventive services: methods. Atlanta: Centers for Disease Control and...
  • Health-evidence ca. Promoting evidence based decision making. Hamilton, Ontorio, 2003. Available at:...
  • Health-evidence ca. Effective Public Health Practice Project. City of Hamilton, 2006. Available at:...
  • P.J. Darzins et al.

    How to read a journal article

    Med J Aust

    (1992)
  • R.F. Heller

    Evidence for population health

    (2005)
  • R.F. Heller et al.

    A population perspective to evidence based medicine: “evidence for population health”

    J Epidemiol Community Health

    (2002)
  • Cited by (0)

    View full text