Mapping the Qualitative Evidence Base on the Use of Research Evidence in Health Policy-Making: A Systematic Review

Document Type : Review Article

Authors

1 Centre for Evidence-Based Intervention, University of Oxford, Oxford, UK

2 Swiss Tropical and Public Health Institute, Basel, Switzerland

3 University of Basel, Basel, Switzerland

Abstract

Background
The use of research evidence in health policy-making is a popular line of inquiry for scholars of public health and policy studies, with qualitative methods constituting the dominant strategy in this area. Research on this subject has been criticized for, among other things, disproportionately focusing on high-income countries; overemphasizing ‘barriers and facilitators’ related to evidence use to the neglect of other, less descriptive concerns; relying on descriptive, rather than in-depth explanatory designs; and failing to draw on insights from political/policy studies theories and concepts. We aimed to comprehensively map the global, peer-reviewed qualitative literature on the use of research evidence in health policy-making and to provide a descriptive overview of the geographic, temporal, methodological, and theoretical characteristics of this body of literature.
 
Methods
We conducted a systematic review following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched nine electronic databases, hand-searched 11 health- and policy-related journals, and systematically scanned the reference lists of included sudies and previous reviews. No language, date or geographic limitations were imposed.
 
Results
The review identified 319 qualitative studies on a diverse array of topics related to the use of evidence in health policy-making, spanning 72 countries and published over a nearly 40 year period. A majority of these studies were conducted in high-income countries, but a growing proportion of the research output in this area is now coming from low- and middle-income countries, especially from sub-Saharan Africa. While over half of all studies did not use an identifiable theory or framework, and only one fifth of studies used a theory or conceptual framework drawn from policy studies or political science, we found some evidence that theory-driven and explanatory (eg, comparative case study) designs are becoming more common in this literature. Investigations of the barriers and facilitators related to evidence use constitute a large proportion but by no means a majority of the work in this area.
 
Conclusion
This review provides a bird’s eye mapping of the peer reviewed qualitative research on evidence-to-policy processes, and has identified key features of – and gaps within – this body of literature that will hopefully inform, and improve, research in this area moving forward.

Keywords


  1. Merton RK. The role of applied social science in the formation of policy: a research memorandum. Philos Sci. 1949;16(3):161-181.
  2. Weiss CH. Using Social Research In Public Policy Making. Lexington Books; 1977.
  3. Lindblom CE, Cohen DK. Usable Knowledge: Social Science and Social Problem Solving. Vol 21. Yale University Press; 1979.
  4. Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. British Medical Journal Publishing Group; 1996.
  5. Lavis JN. How can we support the use of systematic reviews in policymaking? PLoS Med. 2009;6(11):e1000141. doi:10.1371/journal.pmed.1000141
  6. Lavis JN, Posada FB, Haines A, Osei E. Use of research to inform public policymaking. Lancet. 2004;364(9445):1615-1621.
  7. Sutcliffe S, Court J. Evidence-Based Policymaking: What Is It? How Does It Work? What Relevance for Developing Countries? London: Overseas Development Institute;2005.
  8. Lavis JN, Ross SE, Hurley JE. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125-154.
  9. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1(1):2. doi:10.1186/1478-4505-1-2
  10. Weiss CH, Bucuvalas MJ. Social Science Research and Decision-Making. New York: Columbia University Press; 1980.
  11. Weiss CH. Research for policy's sake: The enlightenment function of social research. Policy Analysis. 1977;3(4):531-545.
  12. Boswell C. The political functions of expert knowledge: Knowledge and legitimation in European Union immigration policy. J Eur Public Policy. 2008;15(4):471-488. doi:10.1080/13501760801996634
  13. Weiss CH. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426-431.
  14. Pelz DC. Some expanded perspectives on use of social science in public policy. In: Yinger J, Cultler S, eds. Major Social Issues: A Multidisciplinary View. New York: The Free Press; 1978:346-357.
  15. Contandriopoulos D, Lemire M, Denis JL, Tremblay É. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 2010;88(4):444-483. doi:10.1111/j.1468-0009.2010.00608.x
  16. Oliver K, Innvær S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2. doi:10.1186/1472-6963-14-2
  17. Orton L, Lloyd-Williams F, Taylor-Robinson D, O'Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PloS one. 2011;6(7):e21704.
  18. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. a systematic review. PLoS One. 2013;8(10):e77404. doi:10.1371/journal.pone.0077404
  19. Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers' perceptions of their use of evidence: a systematic review. J Health Serv Res Policy.. 2002;7(4):239-244.
  20. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2005;10 Suppl 1:35-48. doi:10.1258/1355819054308549
  21. Masood S, Kothari A, Regan S. The use of research in public health policy: a systematic review. Evid Policy. 2019. doi:10.1332/174426418X15193814624487
  22. Nutley SM, Walter I, Davies HT. Using Evidence: How Research Can Inform Public Services. Policy press; 2007.
  23. Oliver K, Lorenc T, Innvær S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst.. 2014;12(1):34.
  24. Biesbroek R, Dupuis J, Jordan A, et al. Opening up the black box of adaptation decision-making. Nat Clim Chang. 2015;5(6):493.
  25. Checkland K, Harrison S, Marshall M. Is the metaphor of 'barriers to change' useful in understanding implementation? Evidence from general medical practice. J Health Serv Res Policy. 2007;12(2):95-100.
  26. Bach-Mortensen A, Verboom B. Barriers and facilitators systematic reviews in health: A methodological review and recommendations for reviewers. Res Syn Meth. 2020. doi:10.1002/jrsm.1447
  27. Cairney P. The Politics of Evidence-Based Policy Making.Springer; 2016.
  28. Parkhurst J. The Politics of Evidence: From Evidence-Based Policy to the Good Governance of Evidence. Abingdon, UK: Routledge; 2016.
  29. Greer SL, Bekker M, De Leeuw E, et al. Policy, politics and public health. Eur J Public Health. 2017;27(suppl_4):40-43.
  30. Fafard P, Cassola A. Public health and political science: challenges and opportunities for a productive partnership. Public Health. 2020;186:107-109.
  31. Hyder AA, Corluka A, Winch PJ, et al. National policy-makers speak out: are researchers giving them what they need? Health Policy Plan. 2011;26(1):73-82. doi:10.1093/heapol/czq020
  32. Mirzoev T, Das M, Ebenso B, et al. Contextual influences on the role of evidence in health policy development: what can we learn from six policies in India and Nigeria? Evid Policy. 2017;13(1):59-79. doi:10.1332/174426415x14454407579925
  33. Hawkes S, B KA, Jadeja N, et al. Strengthening capacity to apply health research evidence in policy making: experience from four countries. Health Policy Plan. 2016;31(2):161-170. doi:10.1093/heapol/czv032
  34. Nabyonga-Orem J, Mafigiri DK, Marchal B, Ssengooba F, Macq J, Criel B. Research, evidence and policymaking: the perspectives of policy actors on improving uptake of evidence in health policy development and implementation in Uganda. BMC Public Health. 2012;12(1):109.
  35. Young J. Research, policy and practice: why developing countries are different. Journal of International Development: The Journal of the Development Studies Association. 2005;17(6):727-734.
  36. Verboom B, Montgomery P, Bennett S. What factors affect evidence-informed policymaking in public health? Protocol for a systematic review of qualitative evidence using thematic synthesis. Syst Rev.. 2016;5(1):61. doi:10.1186/s13643-016-0240-6
  37. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264-269. doi:10.7326/0003-4819-151-4-200908180-00135
  38. Glenton C, Colvin CJ, Carlsen B, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013(10):CD010414. doi:10.1002/14651858.CD010414.pub2
  39. Noyes J, Popay J. Directly observed therapy and tuberculosis: how can a systematic review of qualitative research contribute to improving services? A qualitative meta-synthesis. J Adv Nurs. 2007;57(3):227-243.
  40. Munro SA, Lewin SA, Smith HJ, Engel ME, Fretheim A, Volmink J. Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med. 2007;4(7):e238.
  41. Noyes J, Booth A, Flemming K, et al. Cochrane Qualitative and Implementation Methods Group guidance paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018;97:49-58. . doi:10.1016/j.jclinepi.2017.06.020
  42. Garside R. Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how? Innovation (Abingdon).2014;27(1):67-79. doi:10.1080/13511610.2013.777270
  43. Carroll C, Booth A. Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Res Synth Methods. 2015;6(2):149-154. doi:10.1002/jrsm.1128
  44. Stewart E, Smith KE. 'Black magic'and'gold dust': the epistemic and political uses of evidence tools in public health policy making. Evid Policy. 2015;11(3):415-437. doi:10.1332/174426415x14381786400158
  45. Humphries S, Stafinski T, Mumtaz Z, Menon D. Barriers and facilitators to evidence-use in program management: a systematic review of the literature. BMC Health Serv Res. 2014;14(1):171. doi:10.1186/1472-6963-14-171
  46. Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet. 2008;372(9650):1661-1669.
  47. Lorenc T, Tyner EF, Petticrew M, et al. Cultures of evidence across policy sectors: systematic review of qualitative evidence. Eur J Public Health. 2014;24(6):1041-1047. doi:10.1093/eurpub/cku038
  48. DeJean D, Giacomini M, Simeonov D, Smith A. Finding qualitative research evidence for health technology assessment. Qual Health Res. 2016;26(10):1307-1317.
  49. Newson R, King L, Rychetnik L, Milat A, Bauman A. Looking both ways: a review of methods for assessing research impacts on policy and the policy utilisation of research. Health Res Policy Syst. 2018;16(1):54. doi:10.1186/s12961-018-0310-4
  50. Petticrew M, Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide. Oxford: John Wiley & Sons; 2008.
  51. Gough D, Oliver S, Thomas J. An Introduction to Systematic Reviews. Sage; 2012.
  52. Erasmus E, Orgill M, Schneider H, Gilson L. Mapping the existing body of health policy implementation research in lower income settings: what is covered and what are the gaps? Health Policy Plan. 2014;29(suppl_3):iii35-iii50.
  53. World Bank. World Bank Country and Lending Groups. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups.  Accessed July 1, 2019. Published 2019.
  54. MapChart.net website.  https://mapchart.net/world.html.   Accessed March 24, 2020.
  55. United Nations Population Division. World Population Prospects 2019, Online Edition. 2019; https://population.un.org/wpp/Download/Standard/Population/.  Accessed March 24, 2020.
  56. Sabatier PA, Weible CM. Theories of the Policy Process. 3rd ed. Boulder, CO: Westview Press; 2014.
  57. Gilson L. Qualitative research synthesis for health policy analysis: what does it entail and what does it offer? Health Policy Plan. 2014;29(suppl 3):iii1-iii5. doi:10.1093/heapol/czu121
  58. Buse K, Mays N, Walt G. Making Health Policy. McGraw-Hill International; 2012.
  59. Jones CO. An Introduction to the Study of Public Policy. Brooks/Cole Publishing Company; 1984.
  60. Becker LA, Loch MR, Reis RS. Obstáculos percibidos por los directores de servicios de salud para la toma de decisiones basada en la evidencia [Barriers perceived by health directors for evidenced-based decision-making]. Rev Panam Salud Publica. 2017;41:e147. doi:10.26633/RPSP.2017.147
  61. Mosquera J, Gomez OL, Mendez F. Uso de los resultados de las investigaciones en salud pública en una entidad territorial de salud en Colombia [Use of the results of public health research in a regional health unit in Colombia]. Colombia Medica. 2005;36(1):16-22.
  62. Ettelt S. The politics of evidence use in health policy making in Germany-the case of regulating hospital minimum volumes. J Health Polit Policy Law. 2017;42(3):513-538. doi:10.1215/03616878-3802965
  63. Weiss CH, Murphy-Graham E, Birkeland S. An alternate route to policy influence: how evaluations affect DARE. Am J Eval. 2005;26(1):12-30. doi:10.1177/1098214004273337
  64. Lancaster K. Performing the evidence-based drug policy paradigm. Contemp Drug Probl. 2016;43(2):142-153. doi:10.1177/0091450916633306
  65. Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs. 2005;50(2):204-211.

Articles in Press, Accepted Manuscript
Available Online from 01 November 2020