Evidence-Informed Deliberative Processes for HTA Around the Globe: Exploring the Next Frontiers of HTA and Best Practices; Comment on “Use of Evidence-informed Deliberative Processes by Health Technology Assessment Agencies Around the Globe”

Document Type : Commentary


1 Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway

2 Department of Community Medicine and Global Health, Institute of Health and Society, University of Oslo, Oslo, Norway

3 Global Health Development Group, Imperial College London School of Public Health, London, UK

4 Center for Global Development Europe, London, UK


This comment reflects on an article by Oortwijn, Jansen, and Baltussen about the use and features of ‘evidence-informed deliberative processes’ (EDPs) among health technology assessment (HTA) agencies around the world and the need for more guidance. First, we highlight procedural aspects that are relevant across key steps of EDP, focusing on conflict of interest, the different roles of stakeholders throughout a HTA and public justification of decisions. Second, we discuss new knowledge and models needed to maximize the value of deliberative processes at the expanding frontiers of HTA, paying special attention to when HTA is applied in primary care, employed for public health interventions, and is produced through international collaboration.


  1. World Health Organization (WHO). Making Fair Choices on the Path to Universal Health Coverage: Final report of the WHO Consultative Group on Equity and Universal Health Coverage. WHO; 2014.
  2. Norheim OF. Ethical priority setting for universal health coverage: challenges in deciding upon fair distribution of health services. BMC Med. 2016;14:75. doi:10.1186/s12916-016-0624-4
  3. Daniels N, Sabin JE. Setting Limits Fairly: Learning to Share Resources for Health. Oxford: Oxford University Press; 2008.
  4. Daniels N. Accountability for reasonableness. BMJ. 2000;321(7272):1300-1301. doi:10.1136/bmj.321.7272.1300
  5. Culyer AJ. NICE's use of cost effectiveness as an exemplar of a deliberative process. Health Econ Policy Law. 2006;1(Pt 3):299-318. doi:10.1017/s1744133106004026
  6. Byskov J, Marchal B, Maluka S, et al. The accountability for reasonableness approach to guide priority setting in health systems within limited resources--findings from action research at district level in Kenya, Tanzania, and Zambia. Health Res Policy Syst. 2014;12:49. doi:10.1186/1478-4505-12-49
  7. Oortwijn W, Jansen M, Baltussen R. Use of evidence-informed deliberative processes by health technology assessment agencies around the globe. Int J Health Policy Manag. 2020;9(1):27-33. doi:10.15171/ijhpm.2019.72
  8. Oortwijn W, Jansen M, Baltussen R. Evidence-Informed Deliberative Processes: A Practical Guide for HTA Agencies to Enhance Legitimate Decision-Making. Version 1.0. Nijmegen: Radboud University Medical Centre, Radboud Institute for Health Sciences; 2019.
  9. Making sense of non-financial competing interests. PLoS Med. 2008;5(9):e199. doi:10.1371/journal.pmed.0050199
  10. Frybourg S, Remuzat C, Kornfeld Å, Toumi M. Conflict of interest in Health Technology Assessment decisions: case law in France and impact on reimbursement decisions. J Mark Access Health Policy. 2015;3(1):25682. doi:10.3402/jmahp.v3.25682
  11. Mandeville KL, Barker R, Packham A, Sowerby C, Yarrow K, Patrick H. Financial interests of patient organisations contributing to technology assessment at England's National Institute for Health and Care Excellence: policy review. BMJ. 2019;364:k5300. doi:10.1136/bmj.k5300
  12. Bond K. Deliberative Processes in Health Technology Assessment: Prospects, Problems, and Policy Proposals. Global Policy Forum; 2020.
  13. Panteli D, Nolting A, Eckhardt H, Kulig M, Busse R. Published and unpublished evidence in coverage decision-making for pharmaceuticals in Europe: existing approaches and way forward. Health Res Policy Syst. 2016;14:6. doi:10.1186/s12961-016-0080-9
  14. Bullement A, Taylor M, McMordie ST, Waters E, Hatswell AJ. NICE, in confidence: an assessment of redaction to obscure confidential information in single technology appraisals by the National Institute for Health and Care Excellence. Pharmacoeconomics. 2019;37(11):1383-1390. doi:10.1007/s40273-019-00818-0
  15. Allen N, Pichler F, Wang T, Patel S, Salek S. Development of archetypes for non-ranking classification and comparison of European National Health Technology Assessment systems. Health Policy. 2013;113(3):305-312. doi:10.1016/j.healthpol.2013.09.007
  16. Kanpirom K, Luz ACG, Chalkidou K, Teerawattananon Y. How should global fund use value-for-money information to sustain its investments in graduating countries? Int J Health Policy Manag. 2017;6(9):529-533. doi:10.15171/ijhpm.2017.25
  17. Teerawattananon Y, McQueston K, Glassman A, Yothasamut J, Myint CY. Health technology assessments as a mechanism for increased value for money: recommendations to the Global Fund. Global Health. 2013;9:35. doi:10.1186/1744-8603-9-35
  18. Culyer AJ. Use of evidence-informed deliberative processes - learning by doing comment on " use of evidence-informed deliberative processes by health technology assessment agencies around the globe." Int J Health Policy Manag. 2020;9(6):263-265. doi:10.15171/ijhpm.2019.116
  19. Augustovski F, Alcaraz A, Caporale J, García Martí S, Pichon Riviere A. Institutionalizing health technology assessment for priority setting and health policy in Latin America: from regional endeavors to national experiences. Expert Rev Pharmacoecon Outcomes Res. 2015;15(1):9-12. doi:10.1586/14737167.2014.963560
  20. Hollingworth S, Gyansa-Lutterodt M, Dsane-Selby L, et al. Implementing health technology assessment in Ghana to support universal health coverage: building relationships that focus on people, policy, and process. Int J Technol Assess Health Care. 2020;36(1):8-11. doi:10.1017/s0266462319000795
  21. Mohara A, Youngkong S, Velasco RP, et al. Using health technology assessment for informing coverage decisions in Thailand. J Comp Eff Res. 2012;1(2):137-146. doi:10.2217/cer.12.10
  22. Sharma M, Teerawattananon Y, Luz A, Li R, Rattanavipapong W, Dabak S. Institutionalizing evidence-informed priority setting for universal health coverage: lessons from Indonesia. Inquiry. 2020;57:46958020924920. doi:10.1177/0046958020924920
  23. Tantivess S, Chalkidou K, Tritasavit N, Teerawattananon Y. Health Technology Assessment capacity development in low- and middle-income countries: experiences from the international units of HITAP and NICE. F1000Res. 2017;6:2119. doi:10.12688/f1000research.13180.1
  24. World Health Organization, Regional Office for the Western Pacific. Factors Conducive to the Development of Health Technology Assessment in Asia: Impacts and Policy Options. Manila: WHO Regional Office for the Western Pacific; 2015.
  25. Li R, Ruiz F, Culyer AJ, Chalkidou K, Hofman KJ. Evidence-informed capacity building for setting health priorities in low- and middle-income countries: a framework and recommendations for further research. F1000Res. 2017;6:231. doi:10.12688/f1000research.10966.1
  26. NOU 2018: 16. Det viktigste først — Prinsipper for prioritering i den kommunale helse- og omsorgstjenesten og for offentlig finansierte tannhelsetjenester. Helse- og omsorgsdepartementet; 2018. Available from: https://www.regjeringen.no/no/dokumenter/nou-2018-16/id2622153/. Published December 13, 2018.
  27. Bergen Kommune. Pilot for kunnskapsoppsummeringer med kommunene. Bergen Kommune website. https://www.bergen.kommune.no/hvaskjer/tema/kunnskapskommunen/kunnskapsoppsummeringer. Published 2019.
  28. NIHR. Public Health Intervention Responsive Studies Teams (PHIRST) - call for Local Authority Initiatives specification document.  https://www.nihr.ac.uk/documents/public-health-intervention-responsive-studies-teams-phirst-call-for-local-authority-initiatives-specification-document/24345. Published 2020.
  29. Polus S, Mathes T, Klingler C, et al. Health technology assessment of public health interventions published 2012 to 2016: an analysis of characteristics and comparison of methods. Int J Technol Assess Health Care. 2019;35(4):280-290. doi:10.1017/s0266462319000515
  30. Jain V, Baker P, Chalkidou K. Harnessing the power of health taxes. BMJ. 2020;369:m1436. doi:10.1136/bmj.m1436
  31. Kriegner S, Ottersen T, Røttingen JA, Gopinathan U. Promoting intersectoral collaboration through the evaluations of public health interventions: insights from key informants in 6 European countries. Int J Health Policy Manag. 2020. doi:10.34172/ijhpm.2020.19
  32. Atkins L, Kelly MP, Littleford C, Leng G, Michie S. From the National Health Service to local government: perceptions of public health transition in England. Public Health. 2019;174:11-17. doi:10.1016/j.puhe.2019.05.016
  33. Lepage-Nefkens I, Douw K, Mantjes G, de Graaf G, Leroy R, Cleemput I. Horizon Scanning for Pharmaceuticals: Proposal for the BeNeLuxA Collaboration. https://beneluxa.org/sites/beneluxa.org/files/2017-07/Horizon%20scanning_ScientificReport_full.pdf. Accessed May 15, 2020.
  34. Erdös J, Ettinger S, Mayer-Ferbas J, de Villiers C, Wild C. European Collaboration in Health Technology Assessment (HTA): goals, methods and outcomes with specific focus on medical devices. Wien Med Wochenschr. 2019;169(11-12):284-292. doi:10.1007/s10354-019-0684-0
  35. HTAi Asia Policy Forum 2019. Background Paper: Sustainability of Healthcare Systems in Asia: Exploring the Roles of Horizon Scanning and Reassessment in the HTA Landscape. https://htai.org/wp-content/uploads/2020/03/HTAi_APF-2019-Background-Paper.pdf. Published 2019.