While many international organisations have independent evaluations, including the International Monetary Fund (IMF) and World Health organization (WHO), uniquely the World Bank in its 2015 World Development Report sought to ascertain the potential biases that influence how its staff interpret evidence and influence policy. Here, we describe the World Bank’s study design, including experiments to ascertain the impact on Bank staff’s judgements of complexity, confirmation bias, sunk cost bias, and an understanding of the wishes of those whom they seek to help. We then review the Bank’s proposed mechanisms to minimise the impact of the biases they identified. We argue that this approach, that we refer to as ‘reflective practice,’ deserves to be adopted more widely among institutions that seek to use evidence from research to inform policy and practice.
Commentary Published on this Paper
Mitigating Evidentiary Bias in Planning and Policy-Making; Comment on “Reflective Practice: How the World Bank Explored Its Own Biases?”
Panel of independent experts. Report of the Ebola Interim Assessment Panel - July 2015. Geneva: WHO; 2015.
Bank W. World Development Report 2015: Mind, Society, and Behavior. Washington DC: World Bank; 2014.
Weiss CH. Research for policy's sake: the enlightenment function of social research. Policy Analysis. 1977;3(4):531-545.
Nutley S, Walter I, Davies HT. From knowing to doing a framework for understanding the evidence-into-practice agenda. Evaluation. 2003;9(2):125-148. Doi:10.1177/1356389003009002002
Collins H. Tacit and explicit knowledge. Chicago: University of Chicago Press; 2010.
Amara N, Ouimet M, Landry R. New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication. 2004;26(1):75-106. Doi:10.1177/1075547004267491
McKee M, Stuckler D. How cognitive biases affect our interpretation of political messages. Bmj. 2010;340:c2276. doi:10.1136/bmj.c2276
Nickerson RS. Confirmation bias: A ubiquitous phenomenon in many guises. Rev Gen Psychol. 1998;2(2):175. Doi:10.1037//1089-2618.104.22.168
Babcock L, Wang X, Loewenstein G. Choosing the wrong pond: Social comparisons in negotiations that reflect a self-serving bias. Q J Econ. 1996;111(1):1-19. Doi:10.2307/2946655
Leighton JP, Sternberg RJ. The nature of reasoning. Cambridge: Cambridge University Press; 2004.
Druckman JN. The implications of framing effects for citizen competence. Political Behavior. 2001;23(3):225-256.
Parkhurst JO. Framing, ideology and evidence: Uganda's HIV success and the development of PEPFAR's' ABC'policy for HIV prevention. Evidence & Policy: A Journal of Research, Debate and Practice. 2012;8(1):17-36. Doi:10.1332/174426412x620119
Sabatier PA. An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences. 1988;21(2-3):129-168. Doi:10.1007/bf00136406
Mullainathan S, Shafir E. Scarcity: Why having too little means so much. Macmillan; 2013.
Nyhan B, Reifler J. When corrections fail: The persistence of political misperceptions. Political Behavior. 2010;32(2):303-330. Doi:10.1007/s11109-010-9112-2
Yang A, Abbass H, Sarker R. Characterizing warfare in red teaming. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on. 2006;36(2):268-285. Doi:10.1109/tsmcb.2005.855569
Risso-Gill I, Balabanova D, Majid F, et al. Understanding the modifiable health systems barriers to hypertension management in Malaysia: a multi-method health systems appraisal approach. BMC Health Serv Res. 2015;15:254. doi:10.1186/s12913-015-0916-y
Kuhlbrandt C, Balabanova D, Chikovani I, et al. In search of patient-centred care in middle income countries: the experience of diabetes care in the former Soviet Union. Health Policy. 2014;118(2):193-200. doi:10.1016/j.healthpol.2014.08.009