Benchmarking Drug Regulatory Systems for Capacity Building: An Integrative Review of Tools, Practice, and Recommendations

Document Type : Review Article

Authors

1 State Key Laboratory of Quality Research in Chinese Medicine, Institute of Chinese Medical Sciences, University of Macau, Macao, China

2 Centre for Pharmaceutical Regulatory Sciences, University of Macau, Macao, China

3 Department of Public Health and Medicinal Administration, Faculty of Health Sciences, University of Macau, Macao, China

Abstract

Background 
Benchmarking has been increasingly used on drug regulatory systems to achieve sustainable pharmaceutical system strengthening. This study aimed to identify the scope, tools and benefits of benchmarking regulatory capacities and the most recent development in such phenomenon.

Methods 
This study employed an integrative and critical review of the literature and documents on benchmarking drug regulatory capacities identified from 6 databases and 5 websites of related organizations and government agencies in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines.

Results 
Forty-three studies and 6 documents about regulatory benchmarking published between 2005 and 2022 were included in this review. Five benchmarking assessment tools or programmes recommended or adopted by international organizations or government agencies had been identified, which collectively covered 12 major regulatory functions (4 at system level and 8 at operational level) involving 9 indicator categories and 382 sub-indicators. Benchmarking drug regulatory systems was reportedly employed at national, regional and international levels for either internal assessment (mostly on regulatory system establishment, drug review process and post marketing surveillance) or external evaluation (mostly on regulatory standards, drug review process and pharmacovigilance systems) to assess current status, monitor performance, determine major challenges, and inform actions for capacity building. Priority of actions in areas such as regulatory process, resources allocation, cooperation and communication, and stakeholder engagement have been suggested for strengthening drug regulatory systems. Nevertheless, the evidence about benchmarking in optimizing regulatory capacities remained underreported.

Conclusion 
This integrative review depicted a framework for decision-makers about why and how benchmarking drug regulatory systems should be undertaken. For effective benchmarking, well-informed decisions about the goals, the scope, the choice of reference points, and benchmarking tools are essential to guide the implementation strategies. Further studies about the positive effects of regulatory benchmarking are warranted to engage continuous commitment to the practice.

Keywords

References

  1. Khadem Broojerdi A, Baran Sillo H, Ostad Ali Dehaghi R, Ward M, Refaat M, Parry J. The World Health Organization global benchmarking tool an instrument to strengthen medical products regulation and promote universal health coverage. Front Med (Lausanne). 2020;7:457. doi:3389/fmed.2020.00457
  2. World Health Organization (WHO). Everybody's Business--Strengthening Health Systems to Improve Health Outcomes: WHO's Framework for Action. WHO; 2007.
  3. Lodge M. Regulatory capacity. In: Lodge M, Wegrich K, eds. The Problem-Solving Capacity of the Modern State: Governance Challenges and Administrative Capacities. Oxford University Press; 2014;63-85.
  4. Hafner T, Walkowiak H, Lee D, Aboagye-Nyame F. Defining pharmaceutical systems strengthening: concepts to enable measurement. Health Policy Plan. 2017;32(4):572-584. doi:1093/heapol/czw153
  5. Haldane V, De Foo C, Abdalla SM, et al. Health systems resilience in managing the COVID-19 pandemic: lessons from 28 countries. Nat Med. 2021;27(6):964-980. doi:1038/s41591-021-01381-y
  6. Zairi M. Benchmarking: the best tool for measuring competitiveness. Benchmarking for Quality Management & Technology. 1994;1(1):11-24. doi:1108/14635779410056859
  7. Ahmed PK, Rafiq M. Integrated benchmarking: a holistic examination of select techniques for benchmarking analysis. Benchmarking for Quality Management & Technology. 1998;5(3):225-242. doi:1108/14635779810234802
  8. Tillema S. Public sector benchmarking and performance improvement: what is the link and can it be improved? Public Money Manag. 2010;30(1):69-75. doi:1080/09540960903492414
  9. Triantafillou P. Benchmarking in the public sector: a critical conceptual framework. Public Adm. 2007;85(3):829-846. doi:1111/j.1467-9299.2007.00669.x
  10. Dorsch JJ, Yasin MM. A framework for benchmarking in the public sector. Int J Public Sect Manag. 1998;11(2/3):91-115. doi:1108/09513559810216410
  11. Guzman J, O'Connell E, Kikule K, Hafner T. The WHO Global Benchmarking Tool: a game changer for strengthening national regulatory capacity. BMJ Glob Health. 2020;5(8):e003181. doi:1136/bmjgh-2020-003181
  12. Owusu Sekyere S, Škrnjug-Yudov I, Ateba Ngoa U, et al. Leveraging WHO's Global Benchmarking Tool to strengthen capacity in clinical trials oversight for public health emergencies: the GHPP VaccTrain model. Global Health. 2022;18(1):63. doi:1186/s12992-022-00854-0
  13. Khadem Broojerdi A, Alfonso C, Ostad Ali Dehaghi R, Refaat M, Sillo HB. Worldwide assessment of low- and middle-income countries' regulatory preparedness to approve medical products during public health emergencies. Front Med (Lausanne). 2021;8:722872. doi:3389/fmed.2021.722872
  14. Lavery C, Emmott J, Jeck-Thole S, et al. An industry survey on managing the pharmacovigilance system master file in a global environment: the need for a pragmatic approach. Pharmaceut Med. 2022;36(4):233-245. doi:1007/s40290-022-00422-2
  15. Barry A, Olsson S, Minzi O, et al. Comparative assessment of the national pharmacovigilance systems in East Africa: Ethiopia, Kenya, Rwanda and Tanzania. Drug Saf. 2020;43(4):339-350. doi:1007/s40264-019-00898-z
  16. World Health Organization (WHO). WHO Global Benchmarking Tool (GBT) for Evaluation of National Regulatory Systems of Medical Products: Revision VI. WHO; 2021.
  17. Centre for Innovation in Regulatory Science (CIRS). Measuring Process and Performance in Regulatory Agencies: The OpERA Programme. CIRS; 2020. https://www.cirsci.org/wp-content/uploads/2020/02/CIRS-RD-Briefing-74-OpERA-programme.pdf. Accessed October 2, 2020.
  18. Shabani JBB, Kayitare E, Nyirimigabo E, et al. The capacity of young national medicine regulatory authorities to ensure the quality of medicines: case of Rwanda. J Pharm Policy Pract. 2022;15(1):90. doi:1186/s40545-022-00492-2
  19. Sani NM, McAuslane N, Kasbon SH, Ahmad R, Yusof FAM, Patel P. An evaluation of Malaysian regulatory process for new active substances approved in 2017 using the OpERA methodology. Ther Innov Regul Sci. 2020;54(5):1215-1224. doi:1007/s43441-020-00140-4
  20. Rahalkar H, Sheppard A, Salek S. Comparison of BRICS-TM countries' biosimilar regulatory frameworks with Australia, Canada and Switzerland: benchmarking best practices. Front Pharmacol. 2021;12:711361. doi:3389/fphar.2021.711361
  21. World Health Organization (WHO). WHO Benchmarks for International Health Regulations (‎IHR)‎ Capacities. WHO; 2019. https://apps.who.int/iris/handle/10665/311158. Accessed December 1, 2022.
  22. World Health Organization (WHO). WHO Essential Medicines and Health Products: Annual Report 2017: Towards Access 2030. WHO; 2018. https://apps.who.int/iris/handle/10665/272972. Accessed December 1, 2022.
  23. World Health Organization (WHO). Regulatory System Strengthening for Medical Products. Proceedings of the 67th World Health Assembly; 2014:28.
  24. Subirana M, Solá I, Garcia JM, Gich I, Urrútia G. A nursing qualitative systematic review required MEDLINE and CINAHL for study identification. J Clin Epidemiol. 2005;58(1):20-25. doi:1016/j.jclinepi.2004.06.001
  25. Torraco RJ. Writing integrative literature reviews: guidelines and examples. Hum Resour Dev Rev. 2005;4(3):356-367. doi:1177/1534484305278283
  26. Bujar M, McAuslane N, Walker S, Salek S. A process for evaluating quality decision-making practices during the development, review and reimbursement of medicines. Int J Health Policy Manag. 2022;11(2):128-137. doi:34172/ijhpm.2020.86
  27. Chaw EE, Suntornsuk L. Regulatory landscape analysis of Myanmar Food and Drug Administration based on the World Health Organization Global Benchmarking Tool. Pharm Sci Asia. 2022;49(5):498-505. doi:29090/psa.2022.05.22.174
  28. Garashi HY, Steinke DT, Schafheutle EI. a systematic review of pharmacovigilance systems in developing countries using the WHO pharmacovigilance indicators. Ther Innov Regul Sci. 2022;56(5):717-743. doi:1007/s43441-022-00415-y
  29. Keyter A, Salek S, Banoo S, Walker S. A proposed regulatory review model to support the South African Health Products Regulatory Authority to become a more efficient and effective agency. Int J Health Policy Manag. 2022;11(6):795-809. doi:34172/ijhpm.2020.213
  30. Mahdavi M, Sajjadi Khasraghi J, Sajadi HS, et al. Developing framework and strategies for capacity building to apply evidence-informed health policy-making in Iran: mixed methods study of SAHSHA Project. Int J Health Policy Manag. 2022;11(10):2236-2247. doi:34172/ijhpm.2021.142
  31. Sithole T, Salek S, Mahlangu G, Walker S. Comparison of the registration process of the medicines control authority of Zimbabwe with Australia, Canada, Singapore, and Switzerland: benchmarking best practices. Expert Rev Clin Pharmacol. 2022;15(1):109-119. doi:1080/17512433.2022.1987883
  32. Xing LY, Lin XN, Zhou HX, Mo JL, Chen M. Research on the construction of vaccine supervision quality management of drug regulatory authorities in compliance with WHO Global Benchmarking Tool (GBT). Chin Pharm. 2022;36(4):391-398. [Chinese].
  33. Zhang Q, Xie Y, Sheng LL, et al. Third-party evaluation methods and systems for drug regulatory measures. Chin food & Drug Administration Magazine. 2022;4:20-29. [Chinese].
  34. Li F. Study on ability evaluation model for drug review authority. Chin J New Drugs. 2021;30(24):2242-2248. [Chinese].
  35. Rodier C, Bujar M, McAuslane N, Patel P, Liberti L. Use of the Certificate for Pharmaceutical Products (CPP) in 18 maturing pharmaceutical markets: comparing agency guidelines with company practice. Ther Innov Regul Sci. 2021;55(1):118-128. doi:1007/s43441-020-00196-2
  36. Russom M, Bahta I, Debesai M. Eritrean pharmacovigilance system: key strategies, success stories, challenges and lessons learned. Drug Saf. 2021;44(10):1021-1032. doi:1007/s40264-021-01102-x
  37. Sithole T, Mahlangu G, Salek S, Walker S. Evaluation of the regulatory review process in Zimbabwe: challenges and opportunities. Ther Innov Regul Sci. 2021;55(3):474-489. doi:1007/s43441-020-00242-z
  38. Hartmann K, Pagliusi S, Precioso A. Landscape analysis of pharmacovigilance and related practices among 34 vaccine manufacturers' from emerging countries. Vaccine. 2020;38(34):5490-5497. doi:1016/j.vaccine.2020.06.016
  39. Keyter A, Salek S, McAuslane N, Banoo S, Azatyan S, Walker S. Implementation of a framework for an abridged review using good reliance practices: optimising the medicine regulatory review process in South Africa. Ther Innov Regul Sci. 2020;54(5):1199-1207. doi:1007/s43441-020-00144-0
  40. Liberti L, Extavour RM, Patel P, McAuslane N. An evaluation of the Caribbean regulatory system centralized assessment process for medicines submitted 2017-2018 using the OpERA methodology. J Pharm Policy Pract. 2020;13:56. doi:1186/s40545-020-00261-z
  41. Patel P, Cerqueira DM, Santos GML, et al. A baseline analysis of regulatory review timelines for ANVISA: 2013-2016. Ther Innov Regul Sci. 2020;54(6):1428-1435. doi:1007/s43441-020-00169-5
  42. Preston C, Freitas Dias M, Peña J, Pombo ML, Porrás A. Addressing the challenges of regulatory systems strengthening in small states. BMJ Glob Health. 2020;5(2):e001912. doi:1136/bmjgh-2019-001912
  43. Saaristo V, Hakamäki P, Koskinen H, Wiss K, Ståhl T. The comparative and objective measurement of health promotion capacity-building: from conceptual framework to operationalization. Glob Health Promot. 2020;27(1):24-32. doi:1177/1757975918769608
  44. Keyter A, Salek S, Banoo S, Walker S. The South African medicines control council: comparison of its registration process with Australia, Canada, Singapore, and Switzerland. Front Pharmacol. 2019;10:228. doi:3389/fphar.2019.00228
  45. Chong SSF, Lim JCW, Tominaga T. Developing key performance indicators to measure the progress of regional regulatory convergence and cooperation in Asia-Pacific Economic Cooperation (APEC). AAPS Open. 2018;4(1):4. doi:1186/s41120-018-0024-2
  46. Mashaki Ceyhan E, Gürsöz H, Alkan A, Coşkun H, Koyuncu O, Walker S. The Turkish medicines and medical devices agency: comparison of its registration process with Australia, Canada, Saudi Arabia, and Singapore. Front Pharmacol. 2018;9:9. doi:3389/fphar.2018.00009
  47. Tang WY, Zhang H, Jin DZ. Study on construction of knowledge management system for drug evaluation and inspection. Shanghai Med Pharm J. 2018;39(13):56-57. [Chinese].
  48. Li R, Ruiz F, Culyer AJ, Chalkidou K, Hofman KJ. Evidence-informed capacity building for setting health priorities in low- and middle-income countries: a framework and recommendations for further research. F1000Res. 2017;6:231. doi:12688/f1000research.10966.1
  49. Mery G, Dobrow MJ, Baker GR, Im J, Brown A. Evaluating investment in quality improvement capacity building: a systematic review. BMJ Open. 2017;7(2):e012431. doi:1136/bmjopen-2016-012431
  50. Liu P, Liu ZP. Measurement and Assessment of Regulatory Capability of Social Regulatory Agencies -Based on a comparative analysis of three regulatory authorities. Wuhan Univer J. 2016;69(4):32-40. [Chinese].
  51. Zhang M, Guan X, Li J, Gao H, Liu D, Shi L. Establishment of drug safety performance evaluation indicator system for work quality evaluation of drug administration department. Chin Pharm. 2016;27(1):1-4. [Chinese].
  52. Chen YC, Wang SZ, Jiang JF. Research on evaluation system of regulatory standards of circulation safety about essential medicines in rural in Jiangxi province. Chin Health Service Management. 2015;5:373-377. [Chinese].
  53. Yang S, Yang DS, Zheng XY, Huang G, Cong LL, Jiang B. Introduction to drug regulatory capabilities: a theory and framework. J Chin Pharm Sci. 2015;24(4):257-262. [Chinese].
  54. Yao JC, Zeng LG, Li YB, Hu L, Du J, Rao J. Research on the key indicators of drug regulation. Chin Pharmaceut. 2014,23(12):3-5. [Chinese].
  55. Zhang X, Lin ZJ, Dong L. Questionaire designing of evaluation on drug safety supervision capacities based upon a 360 degree performance model. Chin Pharmaceut. 2014;28(7):700-705. [Chinese].
  56. Liu LL, McAuslane N, Tzou MC, et al. Characterizing good review practices: a survey report among agencies of APEC member economies. Ther Innov Regul Sci. 2013;47(6):678-683. doi:1177/2168479013494394
  57. Yang S, Wang GP, Huang G, Jiang B. Research on evaluation index system of China's drug regulatory capability. Chin Pharmceut J. 2012;47(14):1175-1178. [Chinese].
  58. McAuslane N, Cone M, Collins J, Walker S. Emerging markets and emerging agencies: a comparative study of how key regulatory agencies in Asia, Latin America, the Middle East, and Africa are developing regulatory processes and review models for new medicinal products. Drug Inf J. 2009;43(3):349-359. doi:1177/009286150904300314
  59. Hirako M, McAuslane N, Salek S, Anderson C, Walker S. A comparison of the drug review process at five international regulatory agencies. Ther Innov Regul Sci. 2007;41(3):291-308. doi:1177/009286150704100302
  60. Cooke J. A framework to evaluate research capacity building in health care. BMC Fam Pract. 2005;6:44. doi:1186/1471-2296-6-44
  61. Arndt C, Baker AC, Querbach T, Schultz R. 2015 Indicators of Regulatory Policy and Governance: Design, Methodology and Key Results. In: OECD Regulatory Policy Working Papers. Paris: OECD Publishing; 2015. doi:1787/5jrnwqm3zp43-en
  62. Heads of Medicines Agencies (HMA). Benchmarking of European Medicines Agencies (BEMA). Final Report on the Fourth BEMA Cycle. HMA; 2019. https://www.hma.eu/fileadmin/dateien/Human_Medicines/01-About_HMA/Working_Groups/BEMA/2019_10_BEMA_IV_Final_Report.pdf. Accessed December 1, 2022.
  63. Government Accountability Office (GAO). FDA Workforce: Agency-Wide Workforce Planning Needed to Ensure Medical Product Staff Meet Current and Future Needs. GAO; 2022. https://www.gao.gov/products/gao-22-104791. Accessed December 1, 2022.
  64. Government Accountability Office (GAO). Scientific Integrity: HHS Agencies Need to Develop Procedures and Train Staff on Reporting and Addressing Political Interference. GAO; 2022. https://www.gao.gov/assets/gao-22-104613.pdf. Accessed December 1, 2022.
  65. Bowen S, Zwi AB. Pathways to "evidence-informed" policy and practice: a framework for action. PLoS Med. 2005;2(7):e166. doi:1371/journal.pmed.0020166
  66. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst. 2009;7(Suppl 1):S1. doi:1186/1478-4505-7-s1-s1
  67. World Health Organization (WHO). Republic of Korea Achieves the Highest Who Level for Regulation of Medicines and Vaccines. WHO; 2022. https://www.who.int/news/item/29-11-2022-republic-of-korea-achieves-the-highest-who-level--for-regulation-of-medicines-and-vaccines. Accessed December 1, 2022.
  68. World Health Organization (WHO). Manual for Benchmarking of the National Regulatory System of Medical Products and Formulation of Institutional Development Plans. WHO; 2021.
  69. Krutten J. Benchmarking in the pharmaceutical industry. Mark Health Serv. 1999;19(3):14-22.
  70. Zairi M, Ahmed PK. Benchmarking maturity as we approach the millennium? Total Qual Manag. 1999;10(4-5):810-816. doi:1080/0954412997848
  71. Peters DH. The application of systems thinking in health: why use systems thinking? Health Res Policy Syst. 2014;12:51. doi:1186/1478-4505-12-51
  72. Naaldenberg J, Vaandrager L, Koelen M, Wagemakers AM, Saan H, de Hoog K. Elaborating on systems thinking in health promotion practice. Glob Health Promot. 2009;16(1):39-47. doi:1177/1757975908100749
  73. Seaton TL. Dissemination and implementation sciences in pharmacy: a call to action for professional organizations. Res Social Adm Pharm. 2017;13(5):902-904. doi:1016/j.sapharm.2017.05.021

Files

History

  • Receive Date: 07 May 2023
  • Revise Date: 22 August 2023
  • Accept Date: 30 September 2023
  • First Publish Date: 04 October 2023

Share

How to cite

Statistics