International Journal of Health Policy and Management

International Journal of Health Policy and Management

Addressing Health Equity Through Action on the Social Determinants of Health: A Global Review of Policy Outcome Evaluation Methods

Document Type : Review Article

Authors
Janice Lee 1
Ashley Schram 1
Emily Riley 2
Patrick Harris 2
Fran Baum 3
Matt Fisher 3
Toby Freeman 3
Sharon Friel 1
1 School of Regulation and Global Governance (RegNet), College of Asia and the Pacific, Australian National University, Canberra, ACT, Australia
2 Menzies Centre for Health Policy, Sydney Medical School, The University of Sydney, Sydney, NSW, Australia
3 Southgate Institute of Health, Society and Equity, Flinders University, Adelaide, SA, Australia
10.15171/ijhpm.2018.04
Abstract
Background
Epidemiological evidence on the social determinants of health inequity is well-advanced, but considerably less attention has been given to evaluating the impact of public policies addressing those social determinants. Methodological challenges to produce evidence on policy outcomes present a significant barrier to mobilising policy actions for health equities. This review aims to examine methodological approaches to policy evaluation of health equity outcomes and identify promising approaches for future research.
 
Methods
We conducted a systematic narrative review of literature critically evaluating policy impact on health equity, synthesizing information on the methodological approaches used. We searched and screened records from five electronic databases, using pre-defined protocols resulting in a total of 50 studies included for review. We coded the studies according to (1) type of policy analysed; (2) research design; (3) analytical techniques; (4) health outcomes; and (5) equity dimensions evaluated.
 
Results
We found a growing number of a wide range of policies being evaluated for health equity outcomes using a variety of research designs. The majority of studies employed an observational research design, most of which were cross-sectional, however, other approaches included experimental designs, simulation modelling, and meta-analysis. Regression techniques dominated the analytical approaches, although a number of novel techniques were used which may offer advantages over traditional regression analysis for the study of distributional impacts of policy. Few studies made intra-national or cross-national comparisons or collected primary data. Despite longstanding challenges of attribution in policy outcome evaluation, the majority of the studies attributed change in physical or mental health outcomes to the policy being evaluated.
 
Conclusion
Our review provides an overview of methodological approaches to health equity policy outcome evaluation, demonstrating what is most commonplace and opportunities from novel approaches. We found the number of studies evaluating the impacts of public policies on health equity are on the rise, but this area of policy evaluation still requires more attention given growing inequities

Highlights

Supplementary File 1 (Download)

Keywords
Policy Evaluation
Methodology
Health Inequities
Social Determinants of Health
Review
Subjects
  1. Marmot M, Allen J, Bell R, et al. WHO European review of social determinants of health and the health divide. Lancet. 2012;380(9846):1011-1029. doi:10.1016/S0140-6736(12)61228-8
  2. Marmot M, Commission on Social Determinants of H. Achieving health equity: from root causes to fair outcomes. Lancet. 2007;370(9593):1153-1163. doi:10.1016/S0140-6736(07)61385-3
  3. Baum F, Friel S. Politics, policies and processes: a multidisciplinary and multi-methods research program on policies on the social determinants of health inequity in Australia. BMJ Open. 2017;7:e017772. doi:10.1136/bmjopen-2017-017772
  4. March JG, Olsen JP. The New Institutionalism - Organizational-Factors in Political Life. Am Polit Sci Rev. 1984;78(3):734-749. doi:10.2307/1961840
  5. Nilsen PS, Christian, Roback K, Cairney P. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implement Sci. 2013;8(1):63. doi:10.1186/1748-5908-8-63
  6. Hudson JR, Lowe SG. Understanding the Policy Process: Analysing Welfare Policy and Practice. 2nd ed. Bristol: The Policy Press; 2009.
  7. Haas PJ, Springer JF. Applied Policy Research: Concepts and Cases. New York: Garland; 1998.
  8. Blas E, Gilson L, Kelly MP, et al. Addressing social determinants of health inequities: what can the state and civil society do? Lancet.372(9650):1684-1689. doi:10.1016/S0140-6736(08)61693-1
  9. Leeuw F, Vaessen J. Impact evaluations and development: NONIE guidance on impact evaluation. Washington, DC: World Bank; 2009.
  10. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
  11. Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008;336(7656):1281.
  12. Lynch J. Reframing inequality? The health inequalities turn as a dangerous frame shift. J Public Health (Oxf). 2017:1-8. doi:10.1093/pubmed/fdw140
  13. Kelly MP, Morgan A, Bonnefoy J, Butt J, Bergman V. The social determinants of health: developing an evidence base for political action. Geneva: World Health Organization; 2007.
  14. Milton B, Moonan M, Taylor-Robinson D, Whitehead M. How can the health equity impact of universal policies be evaluated? Insights into approaches and next steps. Copenhagen: World Health Organisation - Regional Office for Europe; 2011.
  15. Commission on Social Determinants of Health WHO. Closing the gap in a generation: health equity through action on the social determinants of health: Commission on Social Determinants of Health final report. World Health Organization; 2008.
  16. Banks G. Evidence-based policy making: what is it? How do we get it? Paper presented at: ANZSOG/ANU Public Lecture Series; February 4, 2009; Canberra.
  17. Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101-117. doi:10.1016/S0899-3467(07)60142-6
  18. Collins JA, Fauser BC. Balancing the strengths of systematic and narrative reviews. ESHRE; 2005.
  19. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005;10(1 suppl):6-20.
  20. Solar O, Irwin A. A conceptual framework for action on the social determinants of health. Social determinants of health discussion paper 2 (Policy and Practice). Geneva: World Health Organization; 2010.
  21. Birkland TA. An Introduction to the Policy Process: Theories, Concepts and Models of Public Policy Making. 3rd ed. New York: Routledge; 2014.
  22. Cochran CL, Malone EF. Public Policy: Perspectives and choices. 5th ed. Boulder Lynne Rienner; 2014.
  23. Peters BG. American Public Policy: Promise and Performance. 9th ed. United States of America: Sage/CQ Press; 2012.
  24. Carson NA, ZahidHart W. Priority setting in public health and health services research. Aust Health Rev. 2000;23:46-57. doi:10.1071/AH000046
  25. World Health Organization. Global strategy on diet, physical activity and health: a framework to monitor and evaluate implementation. Geneva: WHO; 2006.
  26. Basu S, Vellakkal S, Agrawal S, Stuckler D, Popkin B, Ebrahim S. Averting obesity and type 2 diabetes in India through sugar-sweetened beverage taxation: an economic-epidemiologic modeling study. PLoS Med. 2014;11(1):e1001582.
  27. Blakely T, Cobiac LJ, Cleghorn CL, et al. Health, Health Inequality, and Cost Impacts of Annual Increases in Tobacco Tax: Multistate Life Table Modeling in New Zealand. PLoS Med. 2015;12(7):e1001856. doi:10.1371/journal.pmed.1001856
  28. Barr B, Bambra C, Whitehead M. The impact of NHS resource allocation policy on health inequalities in England 2001-11: longitudinal ecological study. BMJ. 2014;348:g3231.
  29. Stuckler D, Basu S, McKee M. Health care capacity and allocations among South Africa's provinces: infrastructure-inequality traps after the end of apartheid. Am J Public Health. 2011;101(1):165-172.
  30. Kloek GC, van Lenthe FJ, van Nierop PWM, Koelen MA, Mackenbach JP. Impact evaluation of a Dutch community intervention to improve health-related behaviour in deprived neighbourhoods. Health Place. 2006;12(4):665-677. doi:10.1016/j.healthplace.2005.09.002
  31. Burstrom B, Whitehead M, Clayton S, Fritzell S, Vannoni F, Costa G. Health inequalities between lone and couple mothers and policy under different welfare regimes–the example of Italy, Sweden and Britain. Soc Sci Med. 2010;70(6):912-920.
  32. Hatzenbuehler ML, Keyes KM. Inclusive Anti-bullying Policies and Reduced Risk of Suicide Attempts in Lesbian and Gay Youth. J Adolesc Health. 2013;53(1 Suppl):S21-S26. doi:10.1016/j.jadohealth.2012.08.010
  33. Palència L, Malmusi D, De Moortel D, et al. The influence of gender equality policies on gender inequalities in health in Europe. Soc Sci Med. 2014;117:25-33. doi:10.1016/j.socscimed.2014.07.018
  34. Funnell SC, Rogers PJ. Purposeful program theory: effective use of theories of change and logic models. San Francisco: Jossey-Bass/Wiley; 2011.
  35. Bonner L. Using theory-based evaluation to build evidence-based health and social care policy and practice. Crit Public Health. 2003;13(1):77-92. doi:10.1080/0958159031000100224
  36. Weiss CH. Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Connell JP, Kubisch AC, Schorr CH, Weiss CH, eds. New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, DC: Aspen Institute; 1995:65-92.
  37. Pawson R, Tilley N. Realistic Evaluation. Los Angeles: Sage Publications Ltd; 1997.
  38. Edwards P, Steinbach R, Green J, et al. Health impacts of free bus travel for young people: evaluation of a natural experiment in London. J Epidemiol Community Health. 2013;67(8):641-647.
  39. Edwards R, Thomson G, Wilson N, et al. After the smoke has cleared: evaluation of the impact of a new national smoke-free law in New Zealand. Tob Control. 2008;17(1):e2. doi:10.1136/tc.2007.020347
  40. McNamara C. Trade liberalization, social policies and health: an empirical case study. Globalization and Health. 2015;11:42. doi:10.1186/s12992-015-0126-8
  41. Reeves A, Karanikolos M, Mackenbach J, McKee M, Stuckler D. Do employment protection policies reduce the relative disadvantage in the labour market experienced by unhealthy people? A natural experiment created by the Great Recession in Europe. Soc Sci Med. 2014;121:98-108.
  42. Dalton AM, Jones A, Ogilvie D, Petticrew M, White M, Cummins S. Using spatial equity analysis in the process evaluation of environmental interventions to tackle obesity: the healthy towns programme in England. Int J Equity Health. 2013;12(1):43.
  43. Van de Gaer D, Vandenbossche J, Figueroa JL. Children's Health Opportunities and Project Evaluation: Mexico's Oportunidades Program. World Bank Econ Rev. 2014;28(2):282-310. doi:10.1093/wber/lhs032
  44. Ogilvie D, Fayter D, Petticrew M, et al. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Med Res Methodol. 2008;8:8. doi:10.1186/1471-2288-8-8
  45. European Social Survey. The European Social Survery.  http://www.europeansocialsurvey.org/.  Accessed November 8 , 2017.
  46. Mackenbach JP, McKee M. Government, politics and health policy: a quantitative analysis of 30 European countries. Health Policy. 2015;119(10):1298-1308.
  47. Kunst AE, Bos V, Mackenbach JP. Monitoring socio-economic inequalities in health in the European Union: guidelines and illustrations. The Netherlands: Department of Public Health, Erasmus University Rotterdam; 2001.
  48. White H. A contribution to current debates in impact evaluation. Evaluation. 2010;16(2):153-164.
  49. Council MR. A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council; 2000.
  50. Hu Y, van Lenthe FJ, Hoffmann R, van Hedel K, Mackenbach JP. Assessing the impact of natural policy experiments on socioeconomic inequalities in health: how to apply commonly used quantitative analytical methods? BMC Med Res Methodol. 2017;17(1):68. doi:10.1186/s12874-017-0317-5
  51. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012;66(12):1182-1186. doi:10.1136/jech-2011-200375
  52. Milligan V, Phibbs P, Gurran N, Fagan K. Approaches to Evaluation of Affordable Housing Initiatives in Australia. Melbourne, Victoria: Australian Housing and Urban Research Institute; 2007.
  53. Kunst AE. Evaluation of health equity impact of structural policies: overview of research methods used in the SOPHIE project. Int J Health Serv. 2017;47(3):432-239.
  54. Exworthy M, Hunter DJ. The challenge of joined-up government in tackling health inequalities. Int J Public Adm. 2011;34(4):201-212.
International Journal of Health Policy and Management
Volume 7, Issue 7
2018
Pages 581-592

  • Received Date 05 July 2017
  • Revised Date 25 November 2017
  • Accepted Date 13 January 2018
  • First Published Date 01 July 2018
  • Published Date 01 July 2018