Model Choice for Quantitative Health Impact Assessment and Modelling: An Expert Consultation and Narrative Literature Review

Document Type : Review Article

Authors

1 ISGlobal, Barcelona, Spain

2 Universitat Pompeu Fabra (UPF), Barcelona, Spain

3 CIBER Epidemiología y Salud Pública (CIBERESP), Madrid, Spain

4 Institute of Collective Health (ISC), Federal University of Bahia (UFBA), Salvador, Brazil

5 School of Applied Mathematics, Getulio Vargas Foundation, Rio de Janeiro, Brazil

6 Intelligent Data Science and Artificial Intelligence Research Center, Universitat Politècnica de Catalunya (IDEAI-UPC), Barcelona, Spain

7 Hospital Clínic—Universitat de Barcelona, Barcelona, Spain

Abstract

Background 
Health impact assessment (HIA) is a widely used process that aims to identify the health impacts, positive or negative, of a policy or intervention that is not necessarily placed in the health sector. Most HIAs are done prospectively and aim to forecast expected health impacts under assumed policy implementation. HIAs may quantitatively and/or qualitatively assess health impacts, with this study focusing on the former. A variety of quantitative modelling methods exist that are used for forecasting health impacts, however, they differ in application area, data requirements, assumptions, risk modelling, complexities, limitations, strengths, and comprehensibility. We reviewed relevant models, so as to provide public health researchers with considerations for HIA model choice.

Methods 
Based on an HIA expert consultation, combined with a narrative literature review, we identified the most relevant models that can be used for health impact forecasting. We narratively and comparatively reviewed the models, according to their fields of application, their configuration and purposes, counterfactual scenarios, underlying assumptions, health risk modelling, limitations and strengths.

Results 
Seven relevant models for health impacts forecasting were identified, consisting of (i) comparative risk assessment (CRA), (ii) time series analysis (TSA), (iii) compartmental models (CMs), (iv) structural models (SMs), (v) agentbased models (ABMs), (vi) microsimulations (MS), and (vii) artificial intelligence (AI)/machine learning (ML). These models represent a variety in approaches and vary in the fields of HIA application, complexity and comprehensibility. We provide a set of criteria for HIA model choice. Researchers must consider that model input assumptions match the available data and parameter structures, the available resources, and that model outputs match the research question, meet expectations and are comprehensible to end-users.

Conclusion 
The reviewed models have specific characteristics, related to available data and parameter structures, computational implementation, interpretation and comprehensibility, which the researcher should critically consider before HIA model choice.

Keywords

References

  1. Glymour MM, Spiegelman D. Evaluating public health interventions: 5. Causal inference in public health research-do sex, race, and biological factors cause health outcomes? Am J Public Health. 2017;107(1):81-85. doi:2105/ajph.2016.303539
  2. World Health Organization (WHO). Health Impact Assessment: Main Concepts and Suggested Approach. Gothenburg Consensus Paper. Gothenburg: WHO; 1999.
  3. Joffe M, Mindell J. Health impact assessment. Occup Environ Med. 2005;62(12):907-912. doi:1136/oem.2004.014969
  4. Wismar M, Blau J, Ernst K, Figueras J. The Effectiveness of Health Impact Assessment: Scope and Limitations of Supporting Decision-Making in Europe. Copenhagen: WHO Regional Office for Europe; 2007. http://www.euro.who.int/__data/assets/pdf_file/0003/98283/E90794.pdf. Accessed October 2, 2016.
  5. Khandker SR, Koolwal GB, Samad HA. Handbook on Impact Evaluation: Quantitative Methods and Practices. Vol 1. Washington, DC: World Bank Publications; 2010.
  6. Jacobs LR, Shapiro RY. Questioning the conventional wisdom on public opinion toward health reform. PS Polit Sci Polit. 1994;27(2):208-14. doi:2307/420272
  7. Burstein P. Bringing the public back in: should sociologists consider the impact of public opinion on public policy? Soc Forces. 1998;77(1):27-62. doi:1093/sf/77.1.27
  8. Nieuwenhuijsen MJ, Khreis H, Verlinghieri E, Mueller N, Rojas-Rueda D. Participatory quantitative health impact assessment of urban and transport planning in cities: a review and research needs. Environ Int. 2017;103:61-72. doi:1016/j.envint.2017.03.022
  9. Mindell J, Ison E, Joffe M. A glossary for health impact assessment. J Epidemiol Community Health. 2003;57(9):647-651. doi:1136/jech.57.9.647
  10. Nieuwenhuijsen MJ, Ristovska G, Dadvand P. WHO environmental noise guidelines for the European region: a systematic review on environmental noise and adverse birth outcomes. Int J Environ Res Public Health. 2017;14(10):1252. doi:3390/ijerph14101252
  11. Mumpower JL. Selecting and evaluating tools and methods for public participation. Int J Technol Policy Manag. 2001;1(1):66-77. doi:1504/ijtpm.2001.001745
  12. van de Kerkhof M. Making a difference: on the constraints of consensus building and the relevance of deliberation in stakeholder dialogues. Policy Sci. 2006;39(3):279-299. doi:1007/s11077-006-9024-5
  13. Murray C, Ezzati M, Lopez A, Rodgers A, Vander Hoorn S. Comparative quantification of health risks: conceptual framework and methodological issues. In: Ezzati M, Lopez A, Rogers A, Murray C, eds. Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors. Geneva: World Health Organization; 2004:1-38.
  14. Vander Hoorn S, Ezzati M, Rodgers A, Lopez AD, Murray CJ. Estimating attributable burden of disease from exposure and hazard data. In: Ezzati M, Lopez A, Rodgers A, Murray C, eds. Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors. Geneva: World Health Organization; 2004:2129-2140.
  15. Veerman JL, Barendregt JJ, Mackenbach JP. Quantitative health impact assessment: current practice and future directions. J Epidemiol Community Health. 2005;59(5):361-370. doi:1136/jech.2004.026039
  16. Mueller N, Nieuwenhuijsen MJ, Rojas-Rueda D. Quantitative health impact and burden of disease assessment of traffic-related air pollution. In: Khreis H, Nieuwenhuijsen M, Zietsman J, Ramani T, eds. Traffic-Related Air Pollution. Elsevier; 2020:339-359. doi:1016/b978-0-12-818122-5.00013-2
  17. Mueller N, Rojas-Rueda D, Basagaña X, et al. Urban and transport planning related exposures and mortality: a health impact assessment for cities. Environ Health Perspect. 2017;125(1):89-96. doi:1289/ehp220
  18. Mytton OT, Tainio M, Ogilvie D, Panter J, Cobiac L, Woodcock J. The modelled impact of increases in physical activity: the effect of both increased survival and reduced incidence of disease. Eur J Epidemiol. 2017;32(3):235-250. doi:1007/s10654-017-0235-1
  19. Mueller N, Rojas-Rueda D, Cole-Hunter T, et al. Health impact assessment of active transportation: a systematic review. Prev Med. 2015;76:103-114. doi:1016/j.ypmed.2015.04.010
  20. Murray C, Ezzati M, Lopez A, Rodgers A, Vander Hoorn S. Comparative Quantification of Health Risks: Conceptual Framework and Methodological Issues. Vol 1. Geneva: World Health Organization; 2004.
  21. Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2095-2128. doi:1016/s0140-6736(12)61728-0
  22. Forouzanfar MH, Alexander L, Anderson HR, et al. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386(10010):2287-2323. doi:1016/s0140-6736(15)00128-2
  23. Mueller N, Rojas-Rueda D, Khreis H, et al. Changing the urban design of cities for health: the superblock model. Environ Int. 2020;134:105132. doi:1016/j.envint.2019.105132
  24. Mueller N, Rojas-Rueda D, Salmon M, et al. Health impact assessment of cycling network expansions in European cities. Prev Med. 2018;109:62-70. doi:1016/j.ypmed.2017.12.011
  25. Woodcock J, Tainio M, Cheshire J, O'Brien O, Goodman A. Health effects of the London bicycle sharing system: health impact modelling study. BMJ. 2014;348:g425. doi:1136/bmj.g425
  26. Hamilton JD. Time Series Analysis. Princeton, New Jersey: Princeton University Press; 1994.
  27. Brodersen KH, Gallusser F, Koehler J, Remy N, Scott SL. Inferring causal impact using Bayesian structural time-series models. Ann Appl Stat. 2015;9(1):247-274. doi:1214/14-aoas788
  28. Hyndman R, Athanasopoulos G. Dynamic regression models. In: Forecasting: Principles and Practice. 2nd ed. Monash University; 2018. https://otexts.com/fpp2/dynamic.html.
  29. Schaffer AL, Dobbins TA, Pearson SA. Interrupted time series analysis using autoregressive integrated moving average (ARIMA) models: a guide for evaluating large-scale health interventions. BMC Med Res Methodol. 2021;21(1):58. doi:1186/s12874-021-01235-8
  30. Mason TG, Chan KP, Schooling CM, et al. Air quality changes after Hong Kong shipping emission policy: an accountability study. Chemosphere. 2019;226:616-624. doi:1016/j.chemosphere.2019.03.173
  31. Siettos CI, Russo L. Mathematical modeling of infectious disease dynamics. Virulence. 2013;4(4):295-306. doi:4161/viru.24041
  32. A Conceptual Framework for Budget Allocation in the RIVM Chronic Disease Model. A Case Study of Diabetes Mellitus. 2005. https://www.rivm.nl/bibliotheek/rapporten/260706001.pdf.
  33. Al Mamun A. Multistate life tables in public health. In: Life History of Cardiovascular Disease and Its Risk Factors: Multistate Life Table Approach and Application to the Framingham Heart Study (Population Studies). Purdue University Press; 2003.
  34. Briggs AD, Wolstenholme J, Blakely T, Scarborough P. Choosing an epidemiological model structure for the economic evaluation of non-communicable disease public health interventions. Popul Health Metr. 2016;14:17. doi:1186/s12963-016-0085-1
  35. Ojal J, Griffiths U, Hammitt LL, et al. Sustaining pneumococcal vaccination after transitioning from Gavi support: a modelling and cost-effectiveness study in Kenya. Lancet Glob Health. 2019;7(5):e644-e654. doi:1016/s2214-109x(18)30562-x
  36. Xiang Y, Jia Y, Chen L, Guo L, Shu B, Long E. COVID-19 epidemic prediction and the impact of public health interventions: a review of COVID-19 epidemic models. Infect Dis Model. 2021;6:324-342. doi:1016/j.idm.2021.01.001
  37. Williams MJ. External validity and policy adaptation: from impact evaluation to policy design. World Bank Res Obs. 2019;35(2):158-191. doi:1093/wbro/lky010
  38. Carroll CD. The method of endogenous gridpoints for solving dynamic stochastic optimization problems. Econ Lett. 2006;91(3):312-320. doi:1016/j.econlet.2005.09.013
  39. McCarthy M, Biddulph JP, Utley M, Ferguson J, Gallivan S. A health impact assessment model for environmental changes attributable to development projects. J Epidemiol Community Health. 2002;56(8):611-616. doi:1136/jech.56.8.611
  40. Low H, Meghir C. The use of structural models in econometrics. J Econ Perspect. 2017;31(2):33-58. doi:1257/jep.31.2.33
  41. Castro MC, Massuda A, Almeida G, et al. Brazil's unified health system: the first 30 years and prospects for the future. Lancet. 2019;394(10195):345-356. doi:1016/s0140-6736(19)31243-7
  42. Hedstrom P. Dissecting the Social: On the Principles of Analytical Sociology. Cambridge: Cambridge University Press; 2005. doi:1017/cbo9780511488801
  43. Hammond RA. Considerations and best practices in agent-based modeling to inform policy. In: Wallace R, Geller A, Ogawa VA, eds. Assessing the Use of Agent-Based Models for Tobacco Regulation. Washington, DC: The National Academy of Sciences; 2015:161-193. https://www.ncbi.nlm.nih.gov/books/NBK305917/.
  44. Burch TK. Computer simulation and statistical modeling: rivals or complements? In: Burch TK, ed. Model-Based Demography. Cham: Springer; 2018:67-77. doi:1007/978-3-319-65433-1_4
  45. Müller B, Bohn F, Dreßler G, et al. Describing human decisions in agent-based models–ODD + D, an extension of the ODD protocol. Environ Model Softw. 2013;48:37-48. doi:1016/j.envsoft.2013.06.003
  46. Chao D, Hashimoto H, Kondo N. Dynamic impact of social stratification and social influence on smoking prevalence by gender: an agent-based model. Soc Sci Med. 2015;147:280-287. doi:1016/j.socscimed.2015.08.041
  47. Auchincloss AH, Riolo RL, Brown DG, Cook J, Diez Roux AV. An agent-based model of income inequalities in diet in the context of residential segregation. Am J Prev Med. 2011;40(3):303-311. doi:1016/j.amepre.2010.10.033
  48. Stevens H. Why Outbreaks Like Coronavirus Spread Exponentially, And How to “Flatten the Curve.” The Washington Post; 2020. https://www.washingtonpost.com/graphics/2020/world/corona-simulator/. Published April 14, 2020.
  49. Abraham JM. Using microsimulation models to inform U.S. health policy making. Health Serv Res. 2013;48(2 Pt 2):686-695. doi:1111/1475-6773.12052
  50. Rasella D, Basu S, Hone T, Paes-Sousa R, Ocké-Reis CO, Millett C. Child morbidity and mortality associated with alternative policy responses to the economic crisis in Brazil: a nationwide microsimulation study. PLoS Med. 2018;15(5):e1002570. doi:1371/journal.pmed.1002570
  51. Rasella D, Hone T, de Souza LE, Tasca R, Basu S, Millett C. Mortality associated with alternative primary healthcare policies: a nationwide microsimulation modelling study in Brazil. BMC Med. 2019;17(1):82. doi:1186/s12916-019-1316-7
  52. Krijkamp EM, Alarid-Escudero F, Enns EA, Jalal HJ, Hunink MGM, Pechlivanoglou P. Microsimulation modeling for health decision sciences using R: a tutorial. Med Decis Making. 2018;38(3):400-422. doi:1177/0272989x18754513
  53. Roberts M, Russell LB, Paltiel AD, Chambers M, McEwan P, Krahn M. Conceptualizing a model: a report of the ISPOR-SMDM modeling good research practices task force-2. Med Decis Making. 2012;32(5):678-689. doi:1177/0272989x12454941
  54. Rutter CM, Zaslavsky AM, Feuer EJ. Dynamic microsimulation models for health outcomes: a review. Med Decis Making. 2011;31(1):10-18. doi:1177/0272989x10369005
  55. Richiardi MG, Richardson RE. JAS-mine: a new platform for microsimulation and agent-based modelling. Int J Microsimul. 2017;10(1):106-134. doi:34196/ijm.00151
  56. UK Health Forum. UK Health Forum microhealth simulation model. https://ukhealthforum.org.uk/our-work/. Published 2022. Accessed July 18, 2022.
  57. Organization for Economic Cooperation and Development (OECD). OECD’s SPHeP Models. A Tool to Inform Strategic Planning in Public Health. https://www.oecd.org/health/OECD-SPHEP-Models-Brochure-2020.pdf. Published 2020. Accessed July 18, 2022.
  58. Basu S, Vellakkal S, Agrawal S, Stuckler D, Popkin B, Ebrahim S. Averting obesity and type 2 diabetes in India through sugar-sweetened beverage taxation: an economic-epidemiologic modeling study. PLoS Med. 2014;11(1):e1001582. doi:1371/journal.pmed.1001582
  59. Richardson E, Fenton L, Parkinson J, et al. The effect of income-based policies on mortality inequalities in Scotland: a modelling study. Lancet Public Health. 2020;5(3):e150-e156. doi:1016/s2468-2667(20)30011-6
  60. Reddy KP, Shebl FM, Foote JHA, et al. Cost-effectiveness of public health strategies for COVID-19 epidemic control in South Africa: a microsimulation modelling study. Lancet Glob Health. 2021;9(2):e120-e129. doi:1016/s2214-109x(20)30452-6
  61. Gibert K, Izquierdo J, Sànchez-Marrè M, Hamilton SH, Rodríguez-Roda I, Holmes G. Which method to use? An assessment of data mining methods in Environmental Data Science. Environ Model Softw. 2018;110:3-27. doi:1016/j.envsoft.2018.09.021
  62. Lachowycz K, Jones AP. Towards a better understanding of the relationship between greenspace and health: development of a theoretical framework. Landsc Urban Plan. 2013;118:62-69. doi:1016/j.landurbplan.2012.10.012
  63. Peng Y, Nagata MH. An empirical overview of nonlinearity and overfitting in machine learning using COVID-19 data. Chaos Solitons Fractals. 2020;139:110055. doi:1016/j.chaos.2020.110055
  64. Shickel B, Loftus TJ, Adhikari L, Ozrazgat-Baslanti T, Bihorac A, Rashidi P. DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning. Sci Rep. 2019;9(1):1879. doi:1038/s41598-019-38491-0
  65. Remais JV, Hess JJ, Ebi KL, et al. Estimating the health effects of greenhouse gas mitigation strategies: addressing parametric, model, and valuation challenges. Environ Health Perspect. 2014;122(5):447-455. doi:1289/ehp.1306744
  66. Epstein JM. Why model? J Artif Soc Soc Simul. 2008;11(4):12.
  67. Dahabreh IJ, Chan JA, Earley A, et al. A review of validation and calibration methods for health care modeling and simulation. In: Modeling and Simulation in the Context of Health Technology Assessment: Review of Existing Guidance, Future Research Needs, and Validity Assessment. Rockville, MD: Agency for Healthcare Research and Quality; 2017.
International Journal of Health Policy and Management

Articles in Press, Corrected Proof
Available Online from 30 January 2023

Files

History

  • Receive Date: 19 January 2022
  • Revise Date: 26 July 2022
  • Accept Date: 28 January 2023
  • First Publish Date: 30 January 2023

Share

How to cite

Statistics