Original ResearchAppraisal of guidelines developed by the World Health Organization
Introduction
The World Health Organization (WHO) is the leading authority for global public health guidance and policy. One of its core functions is the development of evidence-based guidelines to help policymakers, healthcare practitioners and consumers make informed decisions regarding health interventions.1 WHO publishes nearly 200 guidelines each year on a broad range of topics including reproductive health, chronic and communicable diseases, nutrition and many others.2 WHO guidelines are used by most of the 194 United Nations member countries, thus, WHO recommendations can impact the health of millions of individuals. It is imperative that these guidelines are developed using rigorous, transparent processes, and that recommendations are based on an unbiased synthesis of the best available evidence.
In 2007, Oxman and colleagues criticized WHO guidelines for their infrequent use of systematic evidence reviews, inconsistent use of systematic guideline development processes, heavy reliance on expert opinion to develop the recommendations, and lack of transparency.3 In addition, WHO guidelines lacked implementation strategies and attention to resource implications, especially in low- and middle-income countries. This study also revealed that WHO had little to no internal support or resources devoted to guideline development.
In response to this criticism in 2007, the WHO Director General established the Guidelines Review Committee (GRC) to ensure that guidelines are developed using transparent, evidence-based processes and meet the highest international standards.4 The GRC revised the WHO Handbook for Guideline Development, instituting internationally recognized standards and methods for guideline development.5 These methods focused particularly on the use of high quality systematic evidence reviews upon which to base recommendations, disclosure and management of secondary interests, assembly of a diverse guideline development group, and the use of an explicit framework for assessing the quality of the evidence and for translating evidence into recommendations (Grading of Recommendations, Assessment, Development and Evaluation [GRADE]).6, 7
Several recent publications examined the quality of a small number of WHO guidelines and suggest that, although there may have been improvements since the inauguration of the GRC, concerns remain and further improvements are needed.8, 9, 10 Our study builds on these previous, small or topic-specific studies by appraising the quality of all WHO guidelines approved by the GRC. The results will help inform WHO of areas in the guideline development process and methods that need to be improved upon, and may help users determine how to prioritize resources across multiple guidelines and potential interventions.
Section snippets
Methods
The authors obtained a list of all published, GRC-approved guidelines from the GRC Secretariat on May 23, 2013. Guidelines are defined at WHO as ‘any document that contains a recommendation for the use of a health intervention, whether these are clinical, public health, or policy recommendations.’5 Because the GRC was implemented in 2007, only WHO guidelines published from that year forward are included in our cohort. Supplementary materials specifically referenced in each guideline were
Results
The list provided by the GRC Secretariat encompassed 381 guidelines in various stages of review and approval; 124 guidelines met inclusion criteria (Fig. 1). The WHO guidelines covered a wide variety of topics; HIV/AIDS and/or tuberculosis were the most frequent health topic (34% of all guidelines addressed one or both of these topics). The type of document also varied, from clinical practice guidelines to policy statements to model chapters for textbooks (Appendix Table 1).
The overall quality
Discussion
There are marked variations in the quality of guidelines developed by the WHO since the inauguration of the GRC, as well as variation within individual guidelines across the domains assessed by AGREE II. Delineation of guidelines' scope and purpose, and clarity of presentation were generally better done, however, stakeholder engagement, editorial independence, and applicability were often poorly reported and/or executed.
Our findings are similar to those reported in other studies examining
Acknowledgements
No funding was provided for this project. The authors would like to thank Susan L. Norris, M.D., M.Sc., M.P.H. for providing a list of GRC-approved guidelines and for providing her expertise on the WHO guideline development process; Michael Leo, Ph.D. at the Kaiser Permanente Center for Health Research for his advice on the data analysis; Sherril Gelmon, Dr.P.H. at Portland State University for identifying a graduate student to participate on the project; and Travis B. Carter for technical
References (150)
- et al.
Leading by example: a culture change at WHO
Lancet
(2007 Jun 2) - et al.
Use of evidence in WHO recommendations
Lancet
(2007) - et al.
GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
J Clin Epidemiol
(2011 Apr) - et al.
Use of systematic reviews in WHO recommendations
Lancet
(2011) - et al.
Quality varies across clinical practice guidelines for mammography screening in women aged 40–49 years as assessed by AGREE and AMSTAR instruments
J Clin Epidemiol
(2011 Sep) - et al.
An independent AGREE evaluation of the Occupational Medicine Practice Guidelines
Spine J
(2006 Jan–Feb) WHO guidelines approved by the Guidelines Review Committee
(2012)Guidelines Review Committee (GRC)
(2012)WHO handbook for guideline development
(2012)- et al.
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
BMJ
(2008 Apr 26)
Appraisal of WHO guidelines in maternal health using the AGREE II assessment tool
PLoS One
Guideline development at the World Health Organization: an evaluation
PLoS One
AGREE II: advancing guideline development, reporting and evaluation in health care
CMAJ
Intraclass correlations: uses in assessing rater reliability
Psychol Bull
Finding what works in health care: standards for systematic reviews
U.S. Preventive Services Task Force procedures manual. Washington, D.C.
Quality of clinical practice guideline for glycemic control in type 2 diabetes mellitus
PLoS One
Evaluation of the quality of guidelines for acute gastroenteritis in children with the AGREE instrument
J Pediatr Gastroenterol Nutr
A systematic review of recent clinical practice guidelines on the diagnosis, assessment and management of hypertension
PLoS One
Quality assessment of asthma clinical practice guidelines: a systematic appraisal
Chest
Applicability of the AGREE II instrument in evaluating the development process and quality of current National Academy of Clinical Biochemistry guidelines
Clin Chem
Clinical practice guidelines within the Southern African Development Community: a descriptive study of the quality of guideline development and concordance with best evidence for five priority diseases
Health Res Policy Syst
WHO recommendations for the prevention and treatment of pre-eclampsia and eclampsia
WHO recommendations for induction of labour
WHO guidelines for the management of postpartum hemorrhage and retained placenta
WHO recommendations for the prevention of postpartum hemorrhage
Cochrane handbook for systematic reviews of interventions. Version 5.1.0 ed.
Systematic reviews: CRDs guidance for undertaking reviews in health care United Kingdom
Acceptable medical reasons for breast milk substitutes
Age-friendly primary health care centres toolkit
Anthrax in humans and animals
Antiretroviral therapy for HIV infection in adults and adolescents: recommendations for a public health approach
Antiretroviral therapy for HIV infection in infants and children: towards universal access. Recommendations for a public health approach, 2010 revision
Antiretroviral drugs for treating pregnant women and prevention HIV infection in infants
Baby-friendly hospital initiative: revised, updated and expanded for integrated care
WHO best practices for injections and related procedures toolkit
Cancer control: knowledge into action: WHO guide for effective programs. Diagnosis and treatment
Clinical guidelines for withdrawal management and treatment of drug dependence in closed settings
Guidelines on basic newborn resuscitation
Clinical management of acute pesticide intoxication: prevention of suicidal behaviors
Communicable diseases and severe food shortage: WHO technical note
Community involvement in tuberculosis care and prevention: towards partnerships for health
Guideline: daily iron and folic acid supplementation in pregnant women
Dengue: guidelines for diagnosis, treatment, prevention and control
Fluorescent light-emitting diode (LED) microscopy for diagnosis of tuberculosis: policy statement
Global recommendations on physical activity for health
Part I: guidance document of characterizing and communicating uncertainty in exposure assessment
Guidance on couples HIV testing and counselling including antiretroviral therapy for treatment and prevention of serodiscordant couples: recommendations for a public health approach
Guidance on pre-exposure oral prophylaxis (PrEP) for serodiscordant couples, men and transgender women who have sex with men at high risk of HIV: recommendations for use in the context of demonstration projects
Guidance on prevention of viral hepatitis B and C among people who inject drugs
Cited by (30)
Assessing the World Health Organization: What does the academic debate reveal and is it democratic?
2022, Social Science and MedicineCitation Excerpt :For instance, one of these studies observed participants at the World Health Assembly and reviewed documents to explain how the format of this body changed over time and which ramifications this had for WHO's governance (Eckl, 2017). There were also two observational studies that analyzed its guidelines (Burda et al., 2014) and its mental health policy (Shen, 2014). One of the two reviews surveyed WHO reform proposals broadly (Cassels et al., 2014a), the other WHO reform proposals for better pandemic preparedness and control following the 2014 Ebola outbreak (Moon et al., 2017).
Identifying research questions for HIV, tuberculosis, tuberculosis-HIV, malaria, and neglected tropical diseases through the World Health Organization guideline development process: a retrospective analysis, 2008–2018
2020, Public HealthCitation Excerpt :In addition, guideline development groups should formulate research questions needed to address identified gaps in knowledge.6,7 There has been significant improvement within the WHO in developing public health guidelines.8,9 However, there has been little emphasis on the opportunity provided by the guideline development process to identify, formulate, and compile relevant research questions that address knowledge gaps.
CNGOF Guidelines for Clinical Practice: Revision of the Methodology
2020, Gynecologie Obstetrique Fertilite et SenologieImproving the quality of WHO guidelines over the last decade: progress and challenges
2017, The Lancet Global HealthImprovement evident but still necessary in clinical practice guideline quality: a systematic review
2017, Journal of Clinical EpidemiologyCitation Excerpt :The included reviews contained AGREE II domain scores from a total of 415 CPGs with publication dates ranging from 1992 to 2014. Each review of CPG quality included a mean of 18 AGREE II quality assessments [14–39], with 26% generating scores from two reviewers, 47% from three reviewers, and 57% from four or more reviewers. Only 44% (10 of 25) of reviews reported the interobserver agreement achieved while scoring CPGs; however, the mean reported interclass correlation coefficients (ICCs) were quite good (mean ICC, 0.857; range, 0.671–0.965).
Strength of recommendations in WHO guidelines using GRADE was associated with uptake in national policy
2015, Journal of Clinical EpidemiologyCitation Excerpt :WHO adopted the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) [2] system in January 2009 to support the formulation of evidence-based recommendations. Since that time, there has been a steady increase in the number of recommendations issued by WHO that are based on the GRADE approach [3–5]. According to GRADE, the strength of the recommendation is designated as either “strong” or “weak” (commonly termed “conditional” in WHO guidelines), with strong recommendations generally based on high-quality evidence from randomized controlled trials, except in certain circumstances [6].