Public health interventions and behaviour change: Reviewing the grey literature

Article Outline

• Summary
• Introduction
• Methods
  • Aim of the study
  • Sources and searches
  • Criteria for inclusion and exclusion
• Analysis
• Results and discussion
• Potential factors that can promote or inhibit successful interventions
• Conclusion
• Ethical approval
• Funding
• Competing interests
• References
• Copyright

Summary 

Objectives

This study identified and reviewed grey literature relating to factors facilitating and inhibiting effective interventions in three areas: the promotion of mental health and well-being, the improvement of food and nutrition, and interventions seeking to increase engagement in physical activity. Study design: Sourcing, reviewing and analysis of relevant grey literature.

Methods

Evidence was collected from a variety of non-traditional sources. Thirty-six pieces of documentary evidence across the three areas were selected for in-depth appraisal and review.

Results

A variety of approaches, often short-term, were used both as interventions and outcome measures. Interventions tended to have common outcomes, enabling the identification of themes. These included improvements in participant well-being as well as identification of barriers to, and promoters of, success. Most interventions demonstrated some positive impact, although some did not. This was particularly the case for more objective measures of change, such as physiological measurements, particularly when used to evaluate short-term interventions. Objective health measurement as part of an intervention may act as a catalyst for future behaviour change. Time is an important factor that could either promote or impede the success of interventions for both participants and facilitators. Likewise, the importance of involving all stakeholders, including participants, when planning health promoting interventions was established as an important indicator of success.

Discussion

Despite its limited scope, this review suggests that interventions can be more efficient and effective. For example, larger-scale, longer-term interventions could be more efficient, whilst outcomes relating to the implementation and beyond could provide a clearer picture of effectiveness. Additionally, interventions and evaluations must be flexible, evolve in partnership with local communities, and reflect local need and context.

Keywords: Grey literature, Public health interventions, Health promotion

Back to Article Outline

Introduction 

Grey literature represents information not found in mainstream sources. In line with the expanding worldwide web, the scope of this literature has grown and is now an accepted source of knowledge. Grey literature encompasses information that is, at best, semi-published; e.g. project reports and reports of small-scale, often localized evaluations that may not be widely available, and frequently include information that is methodologically weak. However, the value of grey literature is that it provides access to information which may not be found in sources that deal largely with studies of high methodological quality. Many public health interventions are complex and the outcomes may be influenced by any number of confounding variables. Thus, studies of these interventions do not always lend themselves to designs associated with the traditional hierarchy of evidence. Similarly, they are often studied through an evaluation design related to predetermined outcomes. The disadvantage of using grey literature is the methodological weakness of many of the studies. However, Ogilvie et al.¹ suggested that the evidence base for the effectiveness of public health interventions can be strengthened through location and appraisal of this ‘low-grade’ evidence.

Grey literature, without the restraining parameters of other published work (which quickly becomes out of date), can be timely, innovative and unique. Therefore, if one fails to identify good evidence from grey sources, there is a risk of missing crucial knowledge necessary to understand a phenomenon or process. This study sought to identify and review the grey literature concerning interventions aiming to change health behaviour in three key public health areas: promoting mental health and well-being, improving food and nutrition, and increasing engagement in physical activity.

In recent years, in the UK and globally, there has been a strong political drive to promote health in relation to the three areas of study.², ³, ⁴, ⁵, ⁶ The focus of this drive has been to support individuals and communities to be able to choose health, and there has been some acknowledgement that choices are made within a cultural and structural context which can serve to support or inhibit their sustainability. There is a clear need to identify interventions and strategies that can support people and communities to make positive changes (e.g. in relation to mental health and well-being, diet and physical activity), and more importantly to identify what factors then enable that change to be sustained and to become an integral part of health behaviour. Only when processes are understood can the adoption of quantifiable and sustainable interventions play an important role in addressing health inequalities. The National Institute of Health and Clinical Excellence (NICE) has published a number of documents related to public health interventions in the three areas under consideration, and also in relation to behaviour change in general.⁷, ⁸, ⁹, ¹⁰, ¹¹, ¹² Some of the common ideas that emerge from these documents include the need to plan and implement interventions in partnership with different communities, and to build on the skills and resources that already exist within them. In addition, it is recommended that interventions be personalized, taking account of individual or community characteristics, and that they should provide ongoing support. The need to evaluate all behaviour change interventions is also stressed. It is perhaps this latter recommendation that has suffered most from a lack of systematic and rigorous implementation, particularly in relation to the synthesis of intelligence from the many smaller-scale local interventions specifically aimed at addressing mental health and well-being, physical activity and food and nutrition, either individually or in combination. It has also suffered from the lack of a long-term view (i.e. existing ‘non-grey’ literature provides little evidence about the success of interventions over time). In general, public health prevention focused interventions do not readily lend themselves to measurement, and can be difficult to quantify. Consequently, they may not be reported in mainstream research literature, making it difficult to capture important outcome data. This study sought to identify evidence from within the grey literature, to identify factors influencing the success of interventions, and to understand how intervention strategies can be combined to support sustainability of behaviour change in different settings.

Back to Article Outline

Methods 

Aim of the study 

This study was commissioned by NHS Northwest, a UK Strategic Health Authority, to identify grey literature relating to factors facilitating and inhibiting effective interventions in three areas: the promotion of mental health and well-being, the improvement of food and nutrition, and interventions seeking to increase engagement in physical activity. A pragmatic approach was taken in terms of time and the scope to elicit information. This was important to enable the review to be contemporary and to reflect the local culture and norms where the interventions took place.

Sources and searches 

A reflexive proactive approach was taken that could be adapted to meet the needs of the key stakeholders (those funding the project) and the unfolding evidence. The breadth of the search (and, to a lesser extent, the depth of analysis) was shaped by the resources, timescale and aims of the project. The following sources of literature were used: professional (non-academic) journals and conference proceedings (partly available via bibliographic databases, i.e. CINAHL and MEDLINE); databases of grey literature [i.e. Health Management Information Consortium (HMIC) and the System for Information on Grey Literature in Europe Archive]; gateway services (i.e. INTUTE); and websites of key organizations and government departments [i.e. local primary care trusts, local authorities, Department of Health, NICE, other government departments, the Office of National Statistics and the Social Exclusion Unit (SEU)]. Finally, existing networks of the project team (and of the wider faculty of the University of Salford) were used to approach organizations in seeking information.

Initial searches in all cases were confined to the topics ‘mental health and well-being’, ‘food and nutrition’ and ‘physical activity’ in the context of health promotion/public health interventions in the UK.

Criteria for inclusion and exclusion 

Literature was deemed eligible if it included:

Table 1 outlines the inclusion and exclusion criteria applied to each piece of evidence. Each piece of evidence from the initial search was reviewed independently and face-to-face by two members of the project team. The selected evidence was then subjected to further review and analysis.

Table 1. Inclusion and exclusion criteria.

Back to Article Outline

Analysis 

In considering the evidence, the reviewers focused on cultural, systemic, organizational and individual facilitating and inhibiting factors, while also considering the motivation behind the intervention, and its sustainability (Table 2). Emergent themes, common across the three areas, were identified using a matrix and then collated by the research team.

Table 2. Data collection tool.

Back to Article Outline

Results and discussion 

In total, 36 items of evidence were included in the review (full information is available in the full report).¹³ This comprised 12 items for mental health and well-being interventions, 11 items concerned with food and nutrition, and 13 items detailing physical activity interventions.

The authors found that some grey literature was inaccessible, and the process of locating grey literature required effort and resources; this can be problematic when there are time and budgetary constraints. However, the mixed approach to locating information returned a comprehensive range of literature that offered a rich picture of knowledge, information and evidence. Interestingly, over 70% (n=26) of items of evidence included in the final analysis were obtained from the formal call for participation using existing local and national networks, while the rest were obtained from the searches performed on the different databases/websites (HMIC, n=6; CINAHL, n=3; SEU, n=1). This experience has been shared by other reviewers of health intervention evidence,¹ and may provide useful guidance for future researchers, particularly when reviewing grey literature.

The evidence covered a wide range of interventions. The interventions for all three areas broadly used four approaches to engage people and communities in behaviour change and improve their sense of well-being. These were: involvement in practical activities; provision of advice and information; improved access to resources; and support to make changes. Some of the interventions were national initiatives,¹⁴, ¹⁵ some linked with particular communities,¹⁶, ¹⁷ and some worked with individuals and families.¹⁸, ¹⁹, ²⁰ Guidance from NICE¹⁰ states that the effects of an intervention are rarely restricted to one level, irrespective of the original target group (individuals, communities or whole populations), and that population-level interventions have the greatest potential for sustainable change. In support of this, Rankin et al.²¹ discussed how individual-level interventions do not address the structural factors that may present real barriers to behaviour change or improvement in well-being.

NICE⁹ has stated that time and resources should be set aside for evaluation, and that their size and nature along with underpinning theory should determine the evaluation strategy. Campbell et al.²² and Rychetnik et al.²³ suggested that the theoretical basis of an intervention impacts on the credibility of outcome measures developed for evaluation. However, for the items in the present review, the theoretical basis was often not stated. Rychetnik et al.²³ discussed the range of outcomes that should be developed to ensure the availability of adequate evidence about the intervention. This includes outcomes that encompass stakeholder (including participant) needs, which include both anticipated and unanticipated effects, and that establish the cost-effectiveness of the intervention. Given the complexity of behaviour change and the number of different factors that can influence it, a variety of different evaluation methods may have been expected, collecting both quantitative and qualitative data and being concerned with all aspects of the intervention.²², ²³, ²⁴ The items reviewed generally gave clear overall aims for interventions, often accompanied by more specific objectives, and also identified metrics for evaluation of the intervention. This provided some indication of whether or not the intended aims were met and the factors that either facilitated or inhibited this. Interestingly, although evaluation of the process of public health interventions (i.e. the planning and delivery of the intervention) is recommended,¹⁰, ²³, ²⁴ this did not generally happen and the reports reviewed tended not to distinguish between longer- and shorter-term impact, or indicators of success.

The outcome measures used in the interventions were similar across all three intervention areas in that they routinely included data related to attendance and, where appropriate, details related to the referral process. Other ‘objective’ measures included: physiological measurements; measurement of health and well-being; changes to knowledge and attitudes (pre and post intervention); change or intended change in behaviour; and the cost of the intervention. Alongside these measures, many of the interventions also collected self-report data from participants, facilitators and other stakeholders with regard to increases in self-esteem, confidence, general well-being and social capital, as well as actual or intended behaviour change and other unplanned benefits. However, it was often not clear from the evidence reviewed whether these metrics were developed in consultation with intended participants, facilitators and stakeholders when planning the intervention, despite this being recognized as an important aspect of successful interventions.¹⁰, ²³, ²⁵

Within the items reviewed, the objective evidence, particularly when related to physiological measures, often indicated minor or no change over the time period of the intervention itself, and was rarely evaluated beyond this point.²⁶ In contrast, some of the self-report evidence located¹⁹ suggested that individuals had made changes, although more commonly this data related to intention to change or increased well-being (i.e. feeling generally better, having more social contact and/or having increased confidence and self-esteem). Improvement in well-being seemed to be a common short-term outcome that was captured largely through self-report measurement. This is a significant finding if this improvement acts as a precursor to change aspects of health behaviour. Therefore, when planning evaluations, it could be fruitful to measure improvement in well-being as one of the shorter-term outcomes. Equally, given the complexity of the process of behaviour change, it may be unrealistic to over rely on objective measures, particularly for short-term interventions.²⁶

In general, there was scant evidence of attempts being made to correlate objective and self-report findings, although in two reports,¹⁹, ²⁶ the lack of success with regard to physiological changes was contrasted with positive self-report data related to change in behaviour, intention to change, and increased self-esteem and confidence. The review suggests that it is important to collate a range of measures from individual interventions (as well as collectively between interventions) in order to make sense of conflicting findings and emerging themes. It was also clear from reviewing the evidence that evaluations which include clear process outcomes could help in understanding these issues more fully.

Finally, it appears to be important to match evaluation to the stage of the intervention, and to be realistic about what can be measured at a given time. For evidence about the effectiveness of an intervention to have credibility, it must relate realistically to achievements within a given timescale. In this review, it appeared that short-term projects (6 months or less) cannot be evaluated credibly using anything other than self-report data, while longer-term projects offer the opportunity to measure more objective outcomes of change and, where appropriate, incorporate physiological measurement.

Back to Article Outline

Potential factors that can promote or inhibit successful interventions 

Features across interventions for all three areas were identified in relation to factors that promoted or inhibited the success of different interventions (Table 3). A common barrier was time. This related to the time available to participants and practitioners/facilitators to engage in the intervention, and to the length of time over which the intervention ran. A number of issues could account for this. For example, the context in which interventions are carried out is recognized as having considerable impact on their success, and it should be acknowledged that people have competing demands on their energies and resources which impact on their engagement with the local community. Therefore, in the planning and evaluation of interventions, it may be useful to ensure that interventions are organized and delivered in an accessible way, making engagement more likely.

Table 3. Factors that can prevent or promote successful behaviour change interventions.

There is some recognition that behaviour change interventions (and their evaluation) should have some elements which are dynamic and responsive to needs that may arise as a result of the intervention.²⁷ Therefore, unless there is an attempt to recognize this in the initial aims of interventions and factor in appropriate time and resources, this will impact on the sense that some facilitators/practitioners have regarding a lack of time for effective interventions to support behaviour change. It should also be recognized that many community programmes take some time to get established, and it is not uncommon for both participants and facilitators to feel that work has ‘just got going’ as the funding runs out.²⁵, ²⁶ Other barriers identified in the review were lack of awareness (related to lack of publicity) of interventions, a lack of support or sustained commitment by health practitioners, and a lack of support (from families or communities) for individuals attempting to change their behaviour. All of these potential barriers to success need to be considered when designing an intervention.

Key factors identified in this review as promoting the success of an intervention included working in partnership with other community groups and organizations, and the potential for mainstreaming an intervention into existing services. Partnership working has the potential to contribute to the development of a stronger evidence base for the outcomes of behaviour change interventions. To achieve this, Boyce et al.²⁸ recommended that providers should consider establishing partnerships with local universities to develop robust and relevant evaluation tools. Other aspects of successful interventions were identified in the review. The use of appropriate strategies to raise awareness, the provision of support for engagement, and the flexibility to tailor activities to identified need were seen as important. Involvement in practical activities also appeared to impact on the overall success of interventions. Additionally, the provision of robust administrative support, supporting behaviour changes that offered immediate or tangible benefits to participants, using a straightforward and simple model of intervention, and using facilitators drawn from the local community were also identified as facilitators of successful interventions in this review.

Back to Article Outline

Conclusion 

This review elicited a number of factors that facilitate or inhibit change in relation to healthful behaviours. Given the complexity of behaviour change, interventions and their evaluations should be flexible and able to respond to changes in need and context. Consideration should also be given to how meaningful behaviour change can be achieved, and how some of the structural barriers can be removed.

It is clear that in order to maximize outcomes and find more efficient ways of promoting health, there is a need to move away from small-scale, short-term interventions towards large-scale interventions supported by theory, evidence and, crucially, the communities themselves.

Planning should include consideration of the best ways to evaluate, and should involve all stakeholders, including the ‘target’ population group/s, facilitators and those who commission the interventions. Furthermore, outcomes from interventions, including process outcomes, must relate to the timescale of an intervention and the stage of implementation. Additionally, a range of methods should be used to collect both qualitative and quantitative outcome data. There should be a coordinated approach to the evaluation of multiple small-scale projects, and it is important to measure interventions in terms of process outcomes (including short-, intermediate- and longer-term indicators of success) as well as a range of measures of cost-effectiveness. To achieve this, a range of methods should be used to collect qualitative and quantitative data, and both the intervention and evaluation will need to be flexible to respond to changing needs as they arise.

This review elicited factors related to the success of interventions that can make a real difference to the well-being and health of populations. It suggests that innovative small-scale projects are happening locally, and that reviewing the grey literature can illuminate the benefits of this work. However, small-scale interventions with variable evaluation and outcome measures are unlikely to have an impact on the health of the wider population or on health inequalities. In order to achieve meaningful reductions in morbidity and mortality resulting from poor lifestyle choices, and to reduce health inequalities, scarce resources will need to be used wisely, efficiently and in a more coordinated way. The recommendations arising from this review of the grey literature have an important contribution to make to the understanding of how this can be achieved.

Back to Article Outline

Ethical approval 

The study did not involve human or animal subjects, or personal identifying data. Therefore, it was not necessary to seek formal external ethical approval. The project team committed to following all local and university governance procedures.

Back to Article Outline

Funding 

This project was commissioned and funded by NHS Northwest.

Back to Article Outline

Competing interests 

None declared.

Back to Article Outline

References 

1. Ogilvie D, Hamilton V, Egan M, Petticrew P. Systematic reviews of health effects of social interventions: 1 finding the evidence: how far should you go?. J Epidemiol Community Health. 2005;59:804–808
   • View In Article
2. Department of Health . Saving lives. Our healthier nation. London: Department of Health; 1999;
   • View In Article
3. Department of Health . National service framework for mental health: modern standards and service models. London: Department of Health; 1999;
   • View In Article
4. Department of Health . Choosing health. Making healthier choices easier choices. London: Department of Health; 2004;
   • View In Article
5. Department of Health . Choosing a better diet: a food and health action plan. London: Department of Health; 2005;
   • View In Article
6. Department of Health . Choosing activity: a physical activity action plan. London: Department of Health; 2005;
   • View In Article
7. Tilford S, Delaney F, Vogels M. Effectiveness of mental health promotion interventions: a review. London: NICE; 1997;
   • View In Article
8. Roe L, Hunt P, Bradshaw H, Rayner M. Health promotion interventions to promote healthy eating in the general population – a review. London: NICE; 1997;
   • View In Article
9. National Institute for Health and Clinical Excellence . Four commonly used methods to increase physical activity. NICE public health intervention guidance 2. London: National Institute for Health and Clinical Excellence; 2006;
   • View In Article
10. National Institute for Health and Clinical Excellence . Behaviour change at population, community and individual levels. NICE public health intervention guidance 6. London: National Institute for Health and Clinical Excellence; 2007;
    • View In Article
11. National Institute for Health and Clinical Excellence . Mental well being and older people. NICE public health intervention guidance 16. London: National Institute for Health and Clinical Excellence; 2008;
    • View In Article
12. National Institute for Health and Clinical Excellence . Promoting physical activity in the workplace. NICE public health intervention guidance 13. London: National Institute for Health and Clinical Excellence; 2008;
    • View In Article
13. Hardiker N, McGrath M, McQuarrie C. A synthesis of grey literature around public health interventions and programmes. Available at: http://usir.salford.ac.uk/12137/1/Public_health_2009.pdf.
    • View In Article
14. Carneigie Research Institute . The national evaluation of LEAP: final report on the national evaluation of the local exercise action pilots. London: Department of Health; 2007;
    • View In Article
15. TNS Social . Evaluation of the 5 a day programme. Final report London: BIG Lottery Fund; 2006;
    • View In Article
16. McAllister J, Chourbaji L. Get alive, get active: healthy Gorton physical activity. Project report Manchester: Central Manchester Primary Care Trust; 2006;
    • View In Article
17. Towers A, Nicholson GPJ. Evaluation of the food cooperative groups established by the rural regeneration unit Cockermouth. West Cumbria: University of Central Lancashire/North West Food and Health Task Force; 2005;
    • View In Article
18. Dinan S, Lenihan P, Tenn T, Illiffe S. Is the promotion of physical activity in vulnerable older people feasible and effective in general practice?. Br J Gen Pract. 2006;56:791–793
    • View In Article
    • MEDLINE
19. Sharman K, Wickett J. Evaluation report of franchise of SHINE programme to Oldham community health services. Oldham: Oldham Community Health Services; 2008;
    • View In Article
20. Phillips D, Hagan T, Bodfiled E, Woodthorpe K, Grimsley M. Exploring the impact of group work and mentoring for multiple heritage children’s self esteem, well being and behaviour. Health Soc Care Commun. 2008;16:310–321
    • View In Article
21. Rankin D, Truman J, Backett-Millburn K, Platt S, Petticrew M. The contextual development of health living centre services: an examination of food-related initiatives. Health Place. 2006;12:644–655
    • View In Article
    • MEDLINE
    • CrossRef
22. Campbell M, Fitzpatrick R, Haines A, Kinmouth A, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321:694–696
    • View In Article
    • MEDLINE
    • CrossRef
23. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Commun Health. 2002;56:119–127
    • View In Article
24. Health Development Agency . Framework for action – supplement guidance on evaluation. London: National Institute for Health and Clinical Excellence; 2001;
    • View In Article
25. McGlone P, Dobson B, Dowler E, Nelson M. Food projects and how they work. York: Joseph Rowntree Foundation; 1999;
    • View In Article
26. Peerbhoy D, Majumdar A, Wrightman N, Brand V. Successes and challenges of a community healthy lifestyles intervention in Merseyside (UK) to target families at risk from coronary heart disease. Health Educ J. 2008;67:310–321
    • View In Article
27. Colman S, Emanuel J. The best of Zest: evaluation for health and sustainability. Manchester: NHS Manchester; 2008;
    • View In Article
28. Boyce T, Robertson R, Dixon A. Commissioning and behaviour change. Kicking bad habits. Final report London: The King’s Fund; 2008;
    • View In Article