Developing a National Set of Health Equity Indicators Using a Consensus Building Process

Document Type : Original Article


1 The Gertner Institute for Epidemiology and Health Policy Research, Sheba Medical Center, Ramat Gan, Israel

2 School of Public Health, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel

3 Strategic and Economic Planning Administration, Ministry of Health, Jerusalem, Israel

4 Public Health Services, Ministry of Health, Jerusalem, Israel

5 Medical Technology, Health Information and Research Directorate, Ministry of Health, Jerusalem, Israel

6 Department of Health Policy and Management, Faculty of Health Sciences & Guildford Glazer Faculty of Business and management, Ben Gurion University of the Negev, Beersheba, Israel


Considerable health inequities documented in Israel between communities, populations and regions, undermine the rights of all citizens to optimal health. The first step towards health equity is agreement on a set of national indicators, reflecting equity in healthcare provision and health outcomes, and allowing monitoring of the impact of interventions on the reduction of disparities. We describe the process of reaching a consensus on a defined set of national equity indicators.
The study was conducted between January 2019 and June 2020, in a multistage design: (A) Identifying appropriate and available inequity measures via interviews with stakeholders. (B) Agreement on the screening criteria (public health importance; gap characteristics; potential for change; public interest) and relative weighting. (C) Constructing the consultation framework as an online, 3-round Delphi technique, with a range of experts recruited from the health, welfare and education sectors.

Participants were of diverse age, gender, geographic location, religion and ethnicity, and came from academia, healthcare provision, government ministries and patient representative groups. Thirty measures of inequity, presented to participants, represented the following domains: Health promotion (11 indicators), acute and chronic morbidity (11), life expectancy and mortality (2), health infrastructures and affordability of care (4), education and employment (2). Of the 77 individuals contacted, 75 (97%) expressed willingness to participate, and 55 (73%) completed all three scoring rounds. The leading ten indicators were: Diabetes care, childhood obesity, adult obesity, distribution of healthcare personnel, fatal childhood injuries, cigarette smoking, infant mortality, ability to afford care, access to psychotherapy and distribution of hospital beds. Agreement among raters, measured as intra-class correlation coefficient (ICC), was 0.75.
A diverse range of consultants reached a consensus on the most important national equity indicators, including both clinical and system indicators. Results should be used to guide governmental decision-making and inter-sectoral strategies, furthering the pursuit of a more equitable healthcare system.


  1. Institute of Medicine (US) Committee on Quality of Health Care in America.Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001.  Accessed August 6, 2020.
  2. Whitehead M. The concepts and principles of equity and health. Health Promot Int. 1991;6(3):217-228. doi:10.1093/heapro/6.3.217
  3. Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet. 2008;372(9650):1661-1669. doi:10.1016/s0140-6736(08)61690-6
  4. Solar O, Irwin A. A Conceptual Framework for Action on the Social Determinants of Health.; 2010. Accessed February 4, 2021.
  5. Braveman P. Health disparities and health equity: concepts and measurement. Annu Rev Public Health. 2006;27:167-194. doi:10.1146/annurev.publhealth.27.021405.102103
  6. Healthy Prosperous Lives for All: The European Health Equity Status Report, 2019.   
  7. Nixon SA, Lee K, Bhutta ZA, et al. Canada's global health role: supporting equity and global citizenship as a middle power. Lancet. 2018;391(10131):1736-1748. doi:10.1016/s0140-6736(18)30322-2
  8. Atkinson J, Salmond C, Crampton P. NZDep2013 Index of Deprivation. Wellington: Department of Public Health, University of Otago; 2014.
  9. Abu-Saad K, Avni S, Kalter-Leibovici O. Health disparities monitoring in the U.S.: lessons for monitoring efforts in Israel and other countries. Isr J Health Policy Res. 2018;7(1):14. doi:10.1186/s13584-018-0208-1
  10. Measuring inequalities in the UK for the Sustainable Development Goals - Office for National Statistics. Accessed August 20, 2020.
  11. Income Inequality.; 2021. Accessed February 4, 2021.
  12. OECD. Health at a Glance. Paris: OECD; 2019. doi:10.1787/4dd50c09-en
  13. Clarfield AM, Manor O, Nun GB, et al. Health and health care in Israel: an introduction. Lancet. 2017;389(10088):2503-2513. doi:10.1016/s0140-6736(17)30636-0
  14. Population of Israel on the Eve of 2021.; 2021. Accessed February 4, 2021.
  15. State of Israel. Gaps between Center and Periphery: Selected Data from the Society in Israel Report No. 11. State of Israel; 2019.
  16. Jaffe A, Giveon S, Wulffhart L, et al. Adult Arabs have higher risk for diabetes mellitus than Jews in Israel. PLoS One. 2017;12(5):e0176661. doi:10.1371/journal.pone.0176661
  17. Health inequality and coping with it, 2019. The Ministry of Health, Jerusalem 2020. Accessed June 8, 2021. 
  18. Chernichovsky D, Bisharat B, Brill A, Sharony C. The Health of the Arab Israeli Population. Taub Center report; 2017.
  19. Muhsen K, Green MS, Soskolne V, Neumark Y. Inequalities in non-communicable diseases between the major population groups in Israel: achievements and challenges. Lancet. 2017;389(10088):2531-2541. doi:10.1016/s0140-6736(17)30574-3
  20. Shmueli A. Income-related inequalities in health and health services use in Israel. Isr J Health Policy Res. 2014;3:37. doi:10.1186/2045-4015-3-37
  21. Ministry of Health. Availability of community-based healthcare services in Israel: A spatial mapping.
  22. Balicer RD, Hoshen M, Cohen-Stavi C, et al. Sustained Reduction in Health Disparities Achieved through Targeted Quality Improvement: One-Year Follow-up on a Three-Year Intervention. Health Serv Res. 2015;50(6):1891-1909. doi:10.1111/1475-6773.12300
  23. Wilf-Miron R, Galai N, Gabali A, et al. Organisational efforts to improve quality while reducing healthcare disparities: the case of breast cancer screening among Arab women in Israel. Qual Saf Health Care. 2010;19(5):e36. doi:10.1136/qshc.2008.029645
  24. Horev T, Avni S. Strengthening the capacities of a national health authority in the effort to mitigate health inequity-the Israeli model. Isr J Health Policy Res. 2016;5:19. doi:10.1186/s13584-016-0077-4
  25. Theory NG-C, 2007 undefined. Bringing the mountain to the public: Dilemmas and contradictions in the procedures of public deliberation initiatives that aim to get “ordinary citizens” to deliberate. Accessed July 9, 2020.
  26. Skorecki K, Horton R. Israel: health and beyond. Lancet. 2017;389(10088):2551-2562. doi:10.1016/s0140-6736(17)30877-2
  27. Bleikh H. Poverty and Inequality in Israel: Trends and Decompositions.; 2016. Accessed May 25, 2020.
  28. Goldman S, Siman-Tov M, Bahouth H, et al. The contribution of the Israeli trauma system to the survival of road traffic casualties. Traffic Inj Prev. 2015;16(4):368-373. doi:10.1080/15389588.2014.940458
  29. Landeta J, Barrutia J, Lertxundi A. Hybrid Delphi: a methodology to facilitate contribution from experts in professional contexts. Technol Forecast Soc Change. 2011;78(9):1629-1641. doi:10.1016/j.techfore.2011.03.009
  30. Seow H, Snyder CF, Mularski RA, et al. A framework for assessing quality indicators for cancer care at the end of life. J Pain Symptom Manage. 2009;38(6):903-912. doi:10.1016/j.jpainsymman.2009.04.024
  31. Boylan S, Sainsbury E, Thow AM, et al. A healthy, sustainable and safe food system: examining the perceptions and role of the Australian policy actor using a Delphi survey. Public Health Nutr. 2019;22(16):2921-2930. doi:10.1017/s136898001900185x
  32. Lee J, Lee SH, Chang GT. Expert consensus on the development of a health-related questionnaire for the pediatric field of Korean medicine: a Delphi study. BMC Complement Med Ther. 2020;20(1):10. doi:10.1186/s12906-019-2796-x
  33. Wong ST, Browne AJ, Varcoe C, et al. Development of health equity indicators in primary health care organizations using a modified Delphi. PLoS One. 2014;9(12):e114563. doi:10.1371/journal.pone.0114563
  34. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. doi:10.1371/journal.pone.0020476
  35. Dimick J. What makes a “good” quality indicator?: comment on “Quality indicators for evaluating trauma care.” 2010. Accessed February 4, 2021.
  36. Blas E, Ataguba JE, Huda TM, Bao GK, Rasella D, Gerecke MR. The feasibility of measuring and monitoring social determinants of health and the relevance for policy and programme - a qualitative assessment of four countries. Glob Health Action. 2016;9:29002. doi:10.3402/gha.v9.29002
  37. Jaffe DH, Shmueli A, Ben-Yehuda A, et al. Community healthcare in Israel: quality indicators 2007-2009. Isr J Health Policy Res. 2012;1(1):3. doi:10.1186/2045-4015-1-3
  38. Maharaj V, Rahman F, Adamson L. Tackling child health inequalities due to deprivation: using health equity audit to improve and monitor access to a community paediatric service. Child Care Health Dev. 2014;40(2):223-230. doi:10.1111/cch.12011
  39. Goldberg DW, Wilson JP, Knoblock CA. From text to geographic coordinates: the current state of geocoding. J Urban Reg Inf Syst Assoc. 2007;19(1):33-46.
  40. Balicer RD, Shadmi E, Lieberman N, et al. Reducing health disparities: strategy planning and implementation in Israel's largest health care organization. Health Serv Res. 2011;46(4):1281-1299. doi:10.1111/j.1475-6773.2011.01247.x
  41. Wilf-Miron R, Tov OS, Lewinhoff I, et al. [Equity promotion in Maccabi Healthcare Services: from the Equality Report to an organizational action plan]. Harefuah. 2010;149(4):210-213.
  42. Weiss-Laxer NS, Crandall A, Okano L, Riley AW. Building a foundation for family health measurement in national surveys: a modified Delphi expert process. Matern Child Health J. 2020;24(3):259-266. doi:10.1007/s10995-019-02870-w
  43. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(1):10. doi:10.7275/pdz9-th90
  44. Freitas Â, Santana P, Oliveira MD, Almendra R, Bana ECJC, Bana ECCA. Indicators for evaluating European population health: a Delphi selection process. BMC Public Health. 2018;18(1):557. doi:10.1186/s12889-018-5463-0
  45. Snow J, Mann M. Qualtrics Survey Software: Handbook for Research Professionals. Qualtrics Labs, Inc; 2013.
  46. Keppel K, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital Health Stat 2. 2005(141):1-16.
  47. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012
  48. Weiss MS. Modification of the Kolmogorov-Smirnov statistic for use with correlated data. J Am Stat Assoc. 1978;73(364):872-875. doi:10.1080/01621459.1978.10480116
  49. Chan M. Linking child survival and child development for health, equity, and sustainable development. Lancet. 2013;381(9877):1514-1515. doi:10.1016/s0140-6736(13)60944-7
  50. Green MA, Evans CR, Subramanian SV. Can intersectionality theory enrich population health research? Soc Sci Med. 2017;178:214-216. doi:10.1016/j.socscimed.2017.02.029
  51. Bambra C, Gibson M, Sowden A, Wright K, Whitehead M, Petticrew M. Tackling the wider social determinants of health and health inequalities: evidence from systematic reviews. J Epidemiol Community Health. 2010;64(4):284-291. doi:10.1136/jech.2008.082743
  52. Fawcett S, Schultz J, Watson-Thompson J, Fox M, Bremby R. Building multisectoral partnerships for population health and health equity. Prev Chronic Dis. 2010;7(6):A118.
  53. Turner S, Ollerhead E, Cook A. Identifying research priorities for public health research to address health inequalities: use of Delphi-like survey methods. Health Res Policy Syst. 2017;15(1):87. doi:10.1186/s12961-017-0252-2
  54. Rowe G, Wright G. The Delphi technique: past, present, and future prospects—Introduction to the special issue. Technol Forecast Soc Change. 2011;78(9):1487-1490. doi:10.1016/j.techfore.2011.09.002
  55. Bobrovitz N, Parrilla JS, Santana M, Straus SE, Stelfox HT. A qualitative analysis of a consensus process to develop quality indicators of injury care. Implement Sci. 2013;8:45. doi:10.1186/1748-5908-8-45
  56. Frewer LJ, Fischer ARH, Wentholt MTA, et al. The use of Delphi methodology in agrifood policy development: some lessons learned. Technol Forecast Soc Change. 2011;78(9):1514-1525. doi:10.1016/j.techfore.2011.05.005
  57. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205-212. doi:10.1111/j.1365-2648.2006.03716.x
  58. Goluchowicz K, Blind K. Identification of future fields of standardisation: an explorative application of the Delphi methodology. Technol Forecast Soc Change. 2011;78(9):1526-1541. doi:10.1016/j.techfore.2011.04.014
  59. Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? evidence from systematic reviews. J Epidemiol Community Health. 2013;67(2):190-193. doi:10.1136/jech-2012-201257
  60. Marmot M. Health equity in England: the Marmot review 10 years on. BMJ. 2020;368:m693. doi:10.1136/bmj.m693
  61. Arora M, Chauhan K, John S, Mukhopadhyay A. Multi-sectoral action for addressing social determinants of noncommunicable diseases and mainstreaming health promotion in national health programmes in India. Indian J Community Med. 2011;36(Suppl 1):S43-49. doi:10.4103/0970-0218.94708
  62. Horton R. Offline: COVID-19 is not a pandemic. Lancet. 2020;396(10255):874. doi:10.1016/s0140-6736(20)32000-6
  63. Paremoer L, Nandi S, Serag H, Baum F. Covid-19 pandemic and the social determinants of health. BMJ. 2021;372:n129. doi:10.1136/bmj.n129
Volume 11, Issue 8
August 2022
Pages 1522-1532
  • Receive Date: 06 October 2020
  • Revise Date: 27 April 2021
  • Accept Date: 30 April 2021
  • First Publish Date: 23 June 2021