The Challenges of Regulating Artificial Intelligence in Healthcare; Comment on “Clinical Decision Support and New Regulatory Frameworks for Medical Devices: Are We Ready for It? - A Viewpoint Paper”

Document Type : Commentary

Authors

1 Faculty of Public Health and Policy, London School of Hygiene & Tropical Medicine, London, UK

2 Department of Health Policy, London School of Economics, London, UK

Abstract

Regulation of health technologies must be rigorous, instilling trust among both healthcare providers and patients. This is especially important for the control and supervision of the growing use of artificial intelligence in healthcare. In this commentary on the accompanying piece by Van Laere and colleagues, we set out the scope for applying artificial intelligence in the healthcare sector and outline five key challenges that regulators face in dealing with these modernday technologies. Addressing these challenges will not be easy. While artificial intelligence applications in healthcare have already made rapid progress and benefitted patients, these applications clearly hold even more potential for future developments. Yet it is vital that the regulatory environment keep up with this fast-evolving space of healthcare in order to anticipate and, to the extent possible, prevent the risks that may arise.

Keywords

References

  1. Eichler HG, Abadie E, Baker M, Rasi G. Fifty years after thalidomide; what role for drug regulators? Br J Clin Pharmacol. 2012;74(5):731-733. doi:1111/j.1365-2125.2012.04255.x
  2. Van Laere S, Muylle KM, Cornu P. Clinical decision support and new regulatory frameworks for medical devices: are we ready for it?-A viewpoint paper. Int J Health Policy Manag. 2021. doi:34172/ijhpm.2021.144
  3. Kirchhoff CF, Wang XM, Conlon HD, Anderson S, Ryan AM, Bose A. Biosimilars: Key regulatory considerations and similarity assessment tools. Biotechnol Bioeng. 2017;114(12):2696-2705. doi:1002/bit.26438
  4. Tudur Smith C, Williamson PR, Beresford MW. Methodology of clinical trials for rare diseases. Best Pract Res Clin Rheumatol. 2014;28(2):247-262. doi:1016/j.berh.2014.03.004
  5. Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med. 2022;28(1):31-38. doi:1038/s41591-021-01614-0
  6. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402-2410. doi:1001/jama.2016.17216
  7. Porter P, Abeyratne U, Swarnkar V, et al. A prospective multicentre study testing the diagnostic accuracy of an automated cough sound centred analytic system for the identification of common respiratory disorders in children. Respir Res. 2019;20(1):81. doi:1186/s12931-019-1046-6
  8. Kehl KL, Elmarakeby H, Nishino M, et al. Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports. JAMA Oncol. 2019;5(10):1421-1429. doi:1001/jamaoncol.2019.1800
  9. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi:1038/nature21056
  10. Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci. 2017;5(3):457-469. doi:1177/2167702617691560
  11. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. doi:1038/s41591-018-0300-7
  12. Heaven WD. Hundreds of AI tools have been built to catch covid. None of them helped. 2021. https://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/. Accessed May 20, 2022.
  13. Weiner JP, Kfuri T, Chan K, Fowles JB. "e-Iatrogenesis": the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc. 2007;14(3):387-388. doi:1197/jamia.M2338
  14. Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA. 2017;318(6):517-518. doi:1001/jama.2017.7797
  15. Burrell J. How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 2016;3(1):2053951715622512. doi:1177/2053951715622512
  16. Grote T, Berens P. On the ethics of algorithmic decision-making in healthcare. J Med Ethics. 2020;46(3):205-211. doi:1136/medethics-2019-105586
  17. Maharao N, Antontsev V, Wright M, Varshney J. Entering the era of computationally driven drug development. Drug Metab Rev. 2020;52(2):283-298. doi:1080/03602532.2020.1726944
  18. Bhattacharyya RP, Bandyopadhyay N, Ma P, et al. Simultaneous detection of genotype and phenotype enables rapid and accurate antibiotic susceptibility determination. Nat Med. 2019;25(12):1858-1864. doi:1038/s41591-019-0650-9
  19. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453. doi:1126/science.aax2342
  20. Food and Drug Administration. Artificial Intelligence and Machine Learning in Software as a Medical Device. 2021. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device. Accessed March 17, 2022.
  21. van Kolfschooten H. EU regulation of artificial intelligence: challenges for patients’ rights. Common Mark Law Rev. 2022;59(1):81-112. doi:54648/cola2022005
  22. World Health Organization (WHO). Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Geneva: WHO; 2021.
  23. Eng DK, Khandwala NB, Long J, et al. Artificial intelligence algorithm improves radiologist performance in skeletal age assessment: a prospective multicenter randomized controlled trial. Radiology. 2021;301(3):692-699. doi:1148/radiol.2021204021
  24. Gichoya JW, Banerjee I, Bhimireddy AR, et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health. 2022;4(6):e406-e414. doi:1016/s2589-7500(22)00063-2
  25. Urbina F, Lentzos F, Invernizzi C, Ekins S. Dual use of artificial-intelligence-powered drug discovery. Nat Mach Intell. 2022;4(3):189-191. doi:1038/s42256-022-00465-9
  26. Maresova P. Impact of regulatory changes on innovations in the medical device industry comment on "clinical decision support and new regulatory frameworks for medical devices: are we ready for it?-A viewpoint paper". Int J Health Policy Manag. 2022. doi:34172/ijhpm.2022.7262
  27. Heikkilä M. A quick guide to the most important AI law you’ve never heard of. 2022. https://www.technologyreview.com/2022/05/13/1052223/guide-ai-act-europe/. Accessed August 21, 2022.
  28. Hwang TJ, Kesselheim AS, Vokinger KN. Lifecycle regulation of artificial intelligence- and machine learning-based software devices in medicine. JAMA. 2019;322(23):2285-2286. doi:1001/jama.2019.16842
  29. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy. 1999;4(2):112-121. doi:1177/135581969900400210
  30. Vayena E, Blasimme A, Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. 2018;15(11):e1002689. doi:1371/journal.pmed.1002689

Files

History

  • Receive Date: 18 March 2022
  • Revise Date: 21 August 2022
  • Accept Date: 07 September 2022
  • First Publish Date: 10 September 2022

Share

How to cite

Statistics