The Dutch Citizen Forum on Public Reimbursement of Healthcare: A Qualitative Analysis of Opinion Change

Document Type : Original Article


Department for Health Evidence, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands


A deliberative Citizen Forum ‘Choices in healthcare’ was held in the Netherlands to obtain insight into the criteria informed citizens would propose for the public reimbursement of healthcare. During 3 weekends, 24 citizens participated in evidence-informed deliberation on the basis of 8 case studies. The aim of this study was to assess how the opinions of 8 participants in the deliberative Citizens Forum changed and if so, why participants themselves believe their opinions have changed, whether participation influenced their perceived reasonableness of other participants in the forum and whether it influenced their opinions about involvement of citizens in decision-making.

Semi-structured interviews were held with 8 participants before and after their participation in the Citizen Forum. Using the method of reconstructing interpretive frames opinions about the public reimbursement of healthcare were reconstructed.

Participants’ opinions changed over time; they became more aware of the complexity of decision-making and came to accept that there are limits to the available resources and accept cost as a criterion for reimbursement decisionmaking. Participants report that exchanging arguments and personal experiences with other participants made them change their initial opinions. Participants ascribed increases in the perceived reasonableness of other participants’ opinions to feelings of group-bonding and becoming more familiar with each other’s personal circumstances. Participants further believe that citizens represent an additional opinion to that of other stakeholders and believe their opinions should be considered in relation to those of other stakeholders, given they are provided with opportunities for critical discussion.
Organized deliberation should allow for the exchange of arguments and the sharing of personal experiences which is linked to learning. On the one hand this is reflected in the uptake of new arguments and on the other hand in the revision, specification or expansion of personal argumentation. Providing opportunities for critical deliberation is key to prevent citizens from adhering to initial emotional reactions that remain unchallenged and which may no longer be supported after deliberation.



Supplementary File 1 (Download)

Supplementary File 2 (Download)



  1. Zorginstituut Nederland. Package Advice in Practice – Deliberations for Arriving at A Fair Package. Accessed February 20, 2019. Published 2017.
  2. Jeurissen P, Maarse H, Tanke M. Betaalbare zorg: contouren van een ‘tijdloos’ thema. In: Jeurissen P, Maarse H, Tanke M, eds. Betaalbare zorg. Nijmegen: Radboud Institute for Health Sciences, Celsus academie voor betaalbare zorg; 2018.
  3. Degeling C, Carter SM, Rychetnik L. Which public and why deliberate?--a scoping review of public deliberation in public health and health policy research. Soc Sci Med. 2015;131:114-121. doi:10.1016/j.socscimed.2015.03.009
  4. Abelson J, Eyles J, McLeod CB, Collins P, McMullan C, Forest PG. Does deliberation make a difference? results from a citizens panel study of health goals priority setting. Health Policy. 2003;66(1):95-106. doi:10.1016/s0168-8510(03)00048-4
  5. Solomon S, Abelson J. Why and when should we use public deliberation? Hastings Cent Rep. 2012;42(2):17-20. doi:10.1002/hast.27
  6. Street J, Duszynski K, Krawczyk S, Braunack-Mayer A. The use of citizens' juries in health policy decision-making: a systematic review. Soc Sci Med. 2014;109:1-9. doi:10.1016/j.socscimed.2014.03.005
  7. Menon D, Stafinski T. Role of patient and public participation in health technology assessment and coverage decisions. Expert Rev Pharmacoecon Outcomes Res. 2011;11(1):75-89. doi:10.1586/erp.10.82
  8. Abelson J, Wagner F, DeJean D, et al. Public and patient involvement in health technology assessment: a framework for action. Int J Technol Assess Health Care. 2016;32(4):256-264. doi:10.1017/s0266462316000362
  9. Abelson J, Warren ME, Forest PG. The future of public deliberation on health issues. Hastings Cent Rep. 2012;42(2):27-29. doi:10.1002/hast.30
  10. Bolsewicz Alderman K, Hipgrave D, Jimenez-Soto E. Public engagement in health priority setting in low- and middle-income countries: current trends and considerations for policy. PLoS Med. 2013;10(8):e1001495. doi:10.1371/journal.pmed.1001495
  11. Mitton C, Smith N, Peacock S, Evoy B, Abelson J. Public participation in health care priority setting: a scoping review. Health Policy. 2009;91(3):219-228. doi:10.1016/j.healthpol.2009.01.005
  12. Carman KL, Heeringa JW, Heil SKR, et al. The Use of Public Deliberation in Eliciting Public Input: Findings from a Literature Review (Prepared by the American Institutes for Research Under Contract No. 290-02-0009). AHRQ Publication No. 13-EHC070-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2013.
  13. Stafinski T, Menon D, Yasui Y. Assessing the impact of deliberative processes on the views of participants: is it 'in one ear and out the other?' Health Expect. 2014;17(2):278-290. doi:10.1111/j.1369-7625.2011.00749.x
  14. Maxwell J, Rosell S, Forest PG. Giving citizens a voice in healthcare policy in Canada. BMJ. 2003;326(7397):1031-1033. doi:10.1136/bmj.326.7397.1031
  15. Dolan P, Cookson R, Ferguson B. Effect of discussion and deliberation on the public's views of priority setting in health care: focus group study. BMJ. 1999;318(7188):916-919. doi:10.1136/bmj.318.7188.916
  16. Carman KL, Mallery C, Maurer M, et al. Effectiveness of public deliberation methods for gathering input on issues in healthcare: results from a randomized trial. Soc Sci Med. 2015;133:11-20. doi:10.1016/j.socscimed.2015.03.024
  17. Farmakas A, Theodorou M, Galanis P, et al. Public engagement in setting healthcare priorities: a ranking exercise in Cyprus. Cost Eff Resour Alloc. 2017;15:16. doi:10.1186/s12962-017-0078-3
  18. Theodorou M, Farmakas A. Public participation in priority setting and health policy: the case of Greece. In: Papazisi SK, ed. Democracy, Citizens and Health Police: Participation in Decision Making, Interest Groups and Patients Organizations. Athens: Papazizsi; 2014:185-209.
  19. Danis M, Goold SD, Parise C, Ginsburg M. Enhancing employee capacity to prioritize health insurance benefits. Health Expect. 2007;10(3):236-247. doi:10.1111/j.1369-7625.2007.00442.x
  20. Hurst SA, Schindler M, Goold SD, Danis M. Swiss-CHAT: citizens discuss priorities for swiss health insurance coverage. Int J Health Policy Manag. 2018;7(8):746-754. doi:10.15171/ijhpm.2018.15
  21. Rosén P. Public dialogue on healthcare prioritisation. Health Policy. 2006;79(1):107-116. doi:10.1016/j.healthpol.2005.11.015
  22. Gold MR, Franks P, Siegelberg T, Sofaer S. Does providing cost-effectiveness information change coverage priorities for citizens acting as social decision makers? Health Policy. 2007;83(1):65-72. doi:10.1016/j.healthpol.2006.11.012
  23. Luskin RC, Fishkin JS, Jowell R. Considered opinions: deliberative polling in Britain. Br J Polit Sci. 2002;32(3):455-487. doi:10.1017/s0007123402000194
  24. Gastil J, Black L, Moscovitz K. Ideology, attitude change, and deliberation in small face-to-face groups. Polit Commun. 2008;25(1):23-46. doi:10.1080/10584600701807836
  25. Himmelroos S, Christensen HS. Deliberation and opinion change: evidence from a deliberative mini-public in Finland. Scan Polit Stud. 2014;37(1):41-60. doi:10.1111/1467-9477.12013
  26. Hansen KM, Andersen VN. Deliberative democracy and the deliberative poll on the Euro. Scan Polit Stud. 2004;27(3):261-286. doi:10.1111/j.1467-9477.2004.00106.x
  27. Barabas J. How deliberation affects policy opinions. Am Polit Sci Rev. 2004;98(4):687-701. doi:10.1017/s0003055404041425
  28. Merkle DM. The polls--review: the national issues convention deliberative poll. Public Opin Q. 1996;60(4):588-619.
  29. Fishkin JS. The Voice of the People: Public Opinion and Democracy. New Haven, CT: Yale University Press; 1995.
  30. Fishkin JS. Democracy and Deliberation: New Directions for Democratic Reform. New Haven, CT: Yale University Press; 1991.
  31. Blacksher E, Diebel A, Forest PG, Goold SD, Abelson J. What is public deliberation? Hastings Cent Rep. 2012;42(2):14-17. doi:10.1002/hast.26
  32. Radboudumc. Draagvlak voor lastige keuzes – Eindrapport van het Burgerforum ‘Keuzes in de zorg.’     Accessed June 10, 2019. Published 2018.
  33. Bijlmakers L, Jansen M, Boer B, et al. Increasing the legitimacy of tough choices in healthcare reimbursement: approach and results of a citizen forum in the Netherlands. Value Health. 2020;23(1):32-38. doi:10.1016/j.jval.2019.07.015
  34. Kahane D, Loptson K, Herriman J, Hardy M. Stakeholder and citizen roles in public deliberation. J Public Deliberation. 2013;9(2):2.
  35. Motivaction. Basismodel Mentality.  Accessed June 10, 2019.
  36. Grin J, van de Graaf H, Hoppe R. Interactive Technology Assessment: Een eerste gids voor wie het wagen wil. Den Haag: Rathenau Instituut; 1997.
  37. Reckers-Droog V, Jansen M, Bijlmakers L, Baltussen R, Brouwer W, van Exel J. How does participating in a deliberative citizens panel on healthcare priority setting influence the views of participants? Health Policy. 2020;124(2):143-151. doi:10.1016/j.healthpol.2019.11.011
  38. Ginsburg M, Goold SD, Danis M. (De)constructing 'basic' benefits: citizens define the limits of coverage. Health Aff (Millwood). 2006;25(6):1648-1655. doi:10.1377/hlthaff.25.6.1648
  39. Kahneman D. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux; 2011.
Volume 11, Issue 2
February 2022
Pages 118-127
  • Receive Date: 31 July 2019
  • Revise Date: 13 April 2020
  • Accept Date: 27 May 2020
  • First Publish Date: 01 February 2022