A Process for Evaluating Quality Decision-Making Practices During the Development, Review and Reimbursement of Medicines

Document Type : Original Article


1 Department of Clinical and Pharmaceutical Sciences, School of Life & Medical Sciences, University of Hertfordshire, Hatfield, UK

2 Centre for Innovation in Regulatory Science (CIRS), London, UK


The development of a medicine is not only underpinned by good science but also by Quality DecisionMaking Practices (QDMPs). Indeed, it is important to ensure that all organisations involved in the lifecycle of medicines are aligning their practices in decision-making to the QDMPs to ensure quality, transparent and consistent decisionmaking processes.

The aim of this study was to evaluate the practicality of QoDoS (Quality of Decision-Making Orientation Scheme) in assessing the incorporation of ten QDMPs during the development, review and reimbursement of medicines, illustrated by case studies with a pharmaceutical company, a regulatory authority and a health technology assessment (HTA) agency. Individuals from each organisation completed the 47-item QoDoS questionnaire.

The results demonstrate the applicability of QoDoS in identifying favourable and unfavourable practices and in assessing the consistency and transparency of the QDMPs within each organisation, as well as across the different stakeholders. Furthermore, the study established the value of the methodology in raising awareness of the biases and best practices in decision-making, as well as having a basis for discussion for differences within and across stakeholders to promote consistency and alignment in decision-making. Finally, the QoDoS demonstrated the need for improvement across a number of decision-making practices for the 3 organisations such as the evaluation of alternatives and of the decision impact.

The QoDoS can be used to benchmark organisations’ decision-making practices to provide a basis for discussion to ultimately encourage a level of trust across and within organisations and helping to identify areas for improvement.



Supplementary File 1 (Download)

Supplementary File 2 (Download)



  1. Hammond JS, Keeney RL, Raiffa H. Smart Choices: A Practical Guide to Making Better Decisions. New York: Harvard Business School; 1999.
  2. Thaler RH, Sunstein CR. Nudge: Improving Decisions About Health, Wealth and Happiness. New Haven: Yale University Press; 2009.
  3. Kahneman D. Thinking Fast and Slow. New York: Farrar, Straus and Giroux; 2011.
  4. Spetzler C, Winter H, Meyer J. Decision Quality: Value Creation from Better Business Decisions. New York: Wiley; 2016.
  5. Wang T, McAuslane N, Liberti L, Leufkens H, Hövels A. Building synergy between regulatory and HTA agencies beyond processes and procedures-can we effectively align the evidentiary requirements? a survey of stakeholder perceptions. Value Health. 2018;21(6):707-714. doi:10.1016/j.jval.2017.11.003
  6. McAuslane N, Liberti L, Connelly P. The confluence of accelerated regulatory and health technology assessment access pathways. Clin Pharmacol Ther. 2019;105(4):935-942. doi:10.1002/cpt.1315
  7. Donelan R, Walker S, Salek S. The development and validation of a generic instrument, QoDoS, for assessing the quality of decision making. Front Pharmacol. 2016;7:180. doi:10.3389/fphar.2016.00180
  8. Donelan R, Walker S, Salek S. Factors influencing quality decision-making: regulatory and pharmaceutical industry perspectives. Pharmacoepidemiol Drug Saf. 2015;24(3):319-328. doi:10.1002/pds.3752
  9. Bujar M, McAuslane N, Walker SR, Salek S. Evaluating quality of decision-making processes in medicines' development, regulatory review, and health technology assessment: a systematic review of the literature. Front Pharmacol. 2017;8:189. doi:10.3389/fphar.2017.00189
  10. Bujar M, Donelan R, McAuslane N, Walker S, Salek S. Assessing the quality of decision making in the development and regulatory review of medicines: identifying biases and best practices. Ther Innov Regul Sci. 2017;51(2):250-256. doi:10.1177/2168479016662681
  11. Bujar M, McAuslane N, Walker S, Salek S. The reliability and relevance of a quality of decision making instrument, Quality of Decision-Making Orientation Scheme (QoDoS), for use during the lifecycle of medicines. Front Pharmacol. 2019;10:17. doi:10.3389/fphar.2019.00017
  12. Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. 5th ed. Oxford: Oxford University Press; 2015.
  13. Centre for Innovation in Regulatory Science (CIRS). 2019 Agenda. http://www.cirsci.org/wp-content/uploads/2019/07/CIRS-2019-8-Page-agenda_8July_high-res.pdf.  Accessed September 2, 2019.
  14. Pharmaceutical Executive (PharmExec). Pharm Excec’s Pharma 50 companies 2017.  http://www.pharmexec.com/pharm-execs-top-50-companies-2017.   Accessed May, 7 2018. Published 2017.
  15. World Health Organisation (WHO). Clarification with Respect to a Stringent Regulatory Organisation as Applicable to the Stringent Regulatory Authority (SRA) Guideline. https://extranet.who.int/prequal/sites/default/files/documents/75%20SRA%20clarification_February2017_0.pdf.   Accessed May 7, 2018. Published 2017.
  16. International Network of Agencies for Health Technology Assessment (INAHTA). Members List; 2018. http://www.inahta.org/members/members_list/.   Accessed May 7, 2018.
  17. European Network for Health Technology Assessment (EUnetHTA). Our Organisation; 2018. https://www.eunethta.eu/about-eunethta/organisation/.  Accessed May 7, 2018. Published 2018.
  18. Walker S, McAuslane N, Liberti L, Connelly P. Centre for Innovation in Regulatory Science Workshop report: Building the benefit-risk toolbox: Are there enough common elements across the different methodologies to enable a consensus on a scientifically acceptable framework for making benefit-risk decisions?. http://www.cirsci.org/sites/default/files/CIRS%20June%202012%20Workshop%20Report%20low-res.pdf.   Accessed August 27, 2019. Published 2012.
  19. Walker S, McAuslane N, Bujar M, Connelly P, Liberti L. Quality decision-making practices: Their application and impact in the development, review and reimbursement of medicines. Netherlands: Off Page; 2018.
  20. Leong J, McAuslane N, Walker S, Salek S. Is there a need for a universal benefit-risk assessment framework for medicines? regulatory and industry perspectives. Pharmacoepidemiol Drug Saf. 2013;22(9):1004-1012. doi:10.1002/pds.3464
  21. European Network for Health Technology Assessment (EUnetHTA).  HTA Core Model; 2016. https://meka.thl.fi/htacore/ViewHandbook.aspx.  Accessed May 7, 2018.
  22. McAuslane N, Patel P, Liberti L, Connelly P. Centre for Innovation in Regulatory Science Workshop report: Evolving the Regulatory Review Process: What are the features that enable a transparent, timely, predictable and good-quality review? http://cirsci.org/publications/CIRS_December_2011_WS_Report.pdf.      Accessed May 7, 2019.  Published 2011.
  23. Smietana K, Ekstrom L, Jeffery B, Møller M. Improving R&D productivity. Nat Rev Drug Discov. 2015;14(7):455-456. doi:10.1038/nrd4650
Volume 11, Issue 2
February 2022
Pages 128-137
  • Receive Date: 02 January 2020
  • Revise Date: 30 May 2020
  • Accept Date: 30 May 2020
  • First Publish Date: 01 February 2022