Portuguese Primary Healthcare and Prevention Quality Indicators for Diabetes Mellitus – A Data Envelopment Analysis

Document Type : Original Article


1 MEDCIDS - Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal

2 CINTESIS - Centre for Health Technology and Services Research, Porto, Portugal

3 USF Camélias, ACES Gaia (Grande Porto VII - ARS Norte), Vila Nova de Gaia, Portugal


Diabetes mellitus (DM) is a worldwide public health priority. The increasing prevalence and the budget constraints force to have effective healthcare, especially at the primary healthcare (PHC) level. We aim to assess primary care efficiency considering the best use of human resources to produce optimal diabetes care in terms of prevention quality indicators (PQIs) rates across national ACES (health centre groupings).
We conducted a two-stage data envelopment analysis (DEA) to assess the technical efficiency of 54 Portuguese primary care health centre groupings for the 2016-2017 biennium. In the first stage, efficiency scores were obtained through five output-oriented DEA models under vector return to scale (VRS) assumption, using three input variables representing key primary care human resources and one output representing each one of the five PQIs related to diabetes. In the second stage, Tobit regression models were estimated to assess the determinants of primary care efficiency in diabetes care.

A total of 13 ACES reached the efficiency frontier. Better managing human resources could reduce PQI rates by 52.3% in 2016 and 49.1% in 2017. Higher proportion of patients under 65 years old and better controlled with a hemoglobin A1c (HbA1c) ≤6.5% were associated with better efficiency in diabetes care, whereas higher prevalence of DM and unemployment worsened hospitalizations rates by diabetes short-term complications and lower-extremity amputation.
Inefficiency in DM care was found in most of the primary care settings which can substantially improve the avoidable hospitalization rates by DM using their current level human resources. These findings help to improve diabetes care by targeting human resources at primary care level, which should be integrated into performance assessments considering broader and integrated scopes.


  1. World Health Organization (WHO). Thirteenth General Programme of Work (GPW13): Methods for Impact Measurement. Geneva: WHO; 2020.
  2. Mackenbach JP, Karanikolos M, McKee M. The unequal health of Europeans: successes and failures of policies. Lancet. 2013;381(9872):1125-1134. doi:10.1016/s0140-6736(12)62082-0
  3. Kluge H. A new vision for WHO's European Region: united action for better health. Lancet Public Health. 2020;5(3):e133-e134. doi:10.1016/s2468-2667(20)30003-7
  4. Bindman AB, Grumbach K, Osmond D, Vranizan K, Stewart AL. Primary care and receipt of preventive services. J Gen Intern Med. 1996;11(5):269-276. doi:10.1007/bf02598266
  5. Starfield B. New paradigms for quality in primary care. Br J Gen Pract. 2001;51(465):303-309.  
  6. Bynum B. A history of chronic diseases. Lancet. 2015;385(9963):105-6. doi:10.1016/s0140-6736(15)60007-1
  7. Ramalho A, Castro P, Gonçalves-Pinho M, et al. Primary health care quality indicators: an umbrella review. PLoS One. 2019;14(8):e0220888. doi:10.1371/journal.pone.0220888
  8. Donaldson MS. Measuring the Quality of Health Care. Washington, DC: National Academy Press; 1999.
  9. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003;15(6):523-530. doi:10.1093/intqhc/mzg081
  10. Di Cesare M. Global trends of chronic non-communicable diseases risk factors. Eur J Public Health. 2019;29(Suppl 4):ckz185-196. doi:10.1093/eurpub/ckz185.196
  11. King H, Aubert RE, Herman WH. Global burden of diabetes, 1995-2025: prevalence, numerical estimates, and projections. Diabetes Care. 1998;21(9):1414-1431. doi:10.2337/diacare.21.9.1414
  12. Purdy S, Griffin T, Salisbury C, Sharp D. Ambulatory care sensitive conditions: terminology and disease coding need to be more specific to aid policy makers and clinicians. Public Health. 2009;123(2):169-173. doi:10.1016/j.puhe.2008.11.001
  13. Organisation for Economic Co-operation and Development (OECD). Health at a Glance 2019. Paris: OECD; 2019. doi:10.1787/4dd50c09-en
  14. American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care. 2010;33(Suppl 1):S62-69. doi:10.2337/dc10-S062
  15. Or Z. Exploring the Effects of Health Care on Mortality Across OECD Countries. Paris: OECD; 2001. doi:10.1787/716472585704
  16. International Diabetes Federation (IDF). IDF Diabetes Atlas. Brussels, Belgium: IDF; 2019.
  17. Institute for Health Metrics and Evaluation (IHME). GBD Compare. Seattle, WA: IHME, University of Washington; 2015.
  18. Kharroubi AT, Darwish HM. Diabetes mellitus: the epidemic of the century. World J Diabetes. 2015;6(6):850-867. doi:10.4239/wjd.v6.i6.850
  19. WHO. Diabetes Country Profile 2016. Portugal: WHO; 2016.
  20. Dictorate-General of Health (DGS). National Programme of the Prevention and Control of Diabetes. Lisbon, Portugal: DGS; 2008.
  21. Sarmento J, Rocha JVM, Santana R. Defining ambulatory care sensitive conditions for adults in Portugal. BMC Health Serv Res. 2020;20(1):754. doi:10.1186/s12913-020-05620-9
  22. Seringa J, Marques AP, Moita B, et al. Influence of diabetes on multiple admissions for ambulatory care sensitive conditions. Eur J Public Health. 2018;28(Suppl 4):cky214-157. doi:10.1093/eurpub/cky214.157
  23. Agency for Healthcare Research and Quality (AHRQ). QITM Version v6.0 Prevention Quality Indicators Technical Specifications Updates – Version 502 v2019 (ICD 10-CM/PCS). AHRQ; 2019.
  24. Sarmento J, Alves C, Oliveira P, Sebastião R, Santana R. [Characterization and evolution of avoidable admissions in Portugal: the impact of two methodologic approaches]. Acta Med Port. 2015;28(5):590-600. doi:10.20344/amp.6324
  25. Pinto A, Santos JV, Souza J, et al. Comparison and impact of four different methodologies for identification of ambulatory care sensitive conditions. Int J Environ Res Public Health. 2020;17(21):8121. doi:10.3390/ijerph17218121
  26. Ramalho A, Lobo M, Duarte L, Souza J, Santos P, Freitas A. Landscapes on prevention quality indicators: a spatial analysis of diabetes preventable hospitalizations in Portugal (2016-2017). Int J Environ Res Public Health. 2020;17(22):8387. doi:10.3390/ijerph17228387
  27. Vuik SI, Fontana G, Mayer E, Darzi A. Do hospitalisations for ambulatory care sensitive conditions reflect low access to primary care? an observational cohort study of primary care usage prior to hospitalisation. BMJ Open. 2017;7(8):e015704. doi:10.1136/bmjopen-2016-015704
  28. Helmer DA, Tseng CL, Rajan M, et al. Can ambulatory care prevent hospitalization for metabolic decompensation? Med Care. 2008;46(2):148-157. doi:10.1097/MLR.0b013e31815b9d66
  29. Warner EA, Ziboh AU. The effects of outpatient management on hospitalization for ambulatory care sensitive conditions associated with diabetes mellitus. South Med J. 2008;101(8):815-817. doi:10.1097/SMJ.0b013e31817cf785
  30. Donaldson MS. Measuring the Quality of Health Care: A Statement by The National Roundtable on Health Care Quality. Washington, DC: National Academies Press; 1999.
  31. Institute of Medicine (US) Committee on Quality of Health Care in America. Improving the 21st-century Health Care System. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
  32. Wolfe A. Institute of Medicine Report: crossing the quality chasm: a new health care system for the 21st century. Policy Polit Nurs Pract. 2001;2(3):233-235. doi:10.1177/152715440100200312
  33. Organisation for Economic Co-operation and Development (OECD). Scoping Paper on Health System Efficiency Measurement. OECD; 2016.
  34. Samut PK, Cafrı R. Analysis of the efficiency determinants of health systems in OECD countries by DEA and panel tobit. Soc Indic Res. 2016;129(1):113-132. doi:10.1007/s11205-015-1094-3
  35. Pelone F, Kringos DS, Valerio L, et al. The measurement of relative efficiency of general practice and the implications for policy makers. Health Policy. 2012;107(2-3):258-268. doi:10.1016/j.healthpol.2012.05.005
  36. Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units. Eur J Oper Res. 1978;2(6):429-444. doi:10.1016/0377-2217(78)90138-8
  37. Ramírez-Valdivia MT, Maturana S, Salvo-Garrido S. A multiple stage approach for performance improvement of primary healthcare practice. J Med Syst. 2011;35(5):1015-1028. doi:10.1007/s10916-010-9438-7
  38. Deidda M, Lupiáñez-Villanueva F, Codagnone C, Maghiros I. Using data envelopment analysis to analyse the efficiency of primary care units. J Med Syst. 2014;38(10):122. doi:10.1007/s10916-014-0122-1
  39. Cantor VJM, Poh KL. Integrated analysis of healthcare efficiency: a systematic review. J Med Syst. 2017;42(1):8. doi:10.1007/s10916-017-0848-7
  40. Baker D, Klein R. Explaining outputs of primary health care: population and practice factors. BMJ. 1991;303(6796):225-229. doi:10.1136/bmj.303.6796.225
  41. Milliken O, Devlin RA, Barham V, Hogg W, Dahrouge S, Russell G. Comparative efficiency assessment of primary care service delivery models using data envelopment analysis. Can Public Policy. 2011;37(1):85-109. doi:10.3138/cpp.37.1.85
  42. Banker RD, Conrad RF, Strauss RP. A comparative application of data envelopment analysis and translog methods: an illustrative study of hospital production. Manage Sci. 1986;32(1):30-44. doi:10.1287/mnsc.32.1.30
  43. Salinas-Jiménez J, Smith P. Data envelopment analysis applied to quality in primary health care. Ann Oper Res. 1996;67(1):141-161. doi:10.1007/bf02187027
  44. Bates JM, Baines DL, Whynes DK. Assessing efficiency in general practice: an application of data envelopment analysis. Health Serv Manage Res. 1998;11(2):103-108. doi:10.1177/095148489801100204
  45. Stefko R, Gavurova B, Kocisova K. Healthcare efficiency assessment using DEA analysis in the Slovak Republic. Health Econ Rev. 2018;8(1):6. doi:10.1186/s13561-018-0191-9
  46. Kringos DS, Boerma WG, Hutchinson A, van der Zee J, Groenewegen PP. The breadth of primary care: a systematic literature review of its core dimensions. BMC Health Serv Res. 2010;10:65. doi:10.1186/1472-6963-10-65
  47. Amado CAF, Dyson RG. Exploring the use of DEA for formative evaluation in primary diabetes care: an application to compare English practices. J Oper Res Soc. 2009;60(11):1469-1482. doi:10.1057/jors.2008.160
  48. Borg S, Gerdtham UG, Eeg-Olofsson K, Palaszewski B, Gudbjörnsdottir S. Quality of life in chronic conditions using patient-reported measures and biomarkers: a DEA analysis in type 1 diabetes. Health Econ Rev. 2019;9(1):31. doi:10.1186/s13561-019-0248-4
  49. Testi A, Fareed N, Ozcan YA, Tanfani E. Assessment of physician performance for diabetes: a bias-corrected data envelopment analysis model. Qual Prim Care. 2013;21(6):345-357.
  50. Thorsen M, McGarvey R, Thorsen A. Diabetes management at community health centers: examining associations with patient and regional characteristics, efficiency, and staffing patterns. Soc Sci Med. 2020;255:113017. doi:10.1016/j.socscimed.2020.113017
  51. Zakowska I, Godycki-Cwirko M. Data envelopment analysis applications in primary health care: a systematic review. Fam Pract. 2020;37(2):147-153. doi:10.1093/fampra/cmz057
  52. Hollingsworth B. Measuring efficiency in health care: analytic techniques and health policy. Economica. 2010;77(305):205-256. doi:10.1111/j.1468-0335.2009.00763.x
  53. Sahin I, Ozcan YA. Public sector hospital efficiency for provincial markets in Turkey. J Med Syst. 2000;24(6):307-320. doi:10.1023/a:1005576009257
  54. BI-CSP - Portuguese Primary Health Care Indicators Control Panel. https://bicsp.min-saude.pt/pt/investigacao/Paginas/Matrizindicadorescsp_publico.aspx?isdlg=1.
  55. Dictorate-General of Health: Bilhete de Identidade dos Indicadores dos Cuidados de Saúde Primários Contratualização. 2017. https://www.sns.gov.pt/wp-content/uploads/2017/04/bilhete_identidade_indicadores_contratualizacao_2017.pdf.
  56. Statistics Portugal. Instituto Nacional de Estatística - As pessoas: 2017. Lisboa: INE; 2019. https://www.ine.pt/xurl/pub/320467122.
  57. Khezrimotlagh D, Chen Y. Decision Making and Performance Evaluation Using Data Envelopment Analysis. Switzerland: Springer; 2018. International Series in Operations Research and Management Science. Vol 269.
  58. Aluttis C, Bishaw T, Frank MW. The workforce for health in a globalized context--global shortages and international migration. Glob Health Action. 2014;7:23611. doi:10.3402/gha.v7.23611
  59. Crisp N, Chen L. Global supply of health professionals. N Engl J Med. 2014;370(10):950-957. doi:10.1056/NEJMra1111610
  60. Avkiran NK. Productivity Analysis in the Service Sector with Data Envelopment Analysis. In book: Excellence in Research Australia (ERA) - Collection. Camira. 2002. UQ Business School, The University of Queensland QLD 4072, Australia; 2006. http://www.users.on.net/~necmi/financesite/DEA%20Book%203rd%20Edition%202006_AVKIRAN.pdf.  
  61. Cameron A, Trivedi P. Microeconometrics Using Stata. College Station, TX: Stata Press; 2010.
  62. Atkinson SE, Wilson PW. Comparing mean efficiency and productivity scores from small samples: a bootstrap methodology. J Product Anal. 1995;6(2):137-152. doi:10.1007/bf01073408
  63. Organisation for Economic Co-operation and Development (OECD). OECD Reviews of Health Care Quality: Portugal 2015. OECD; 2015.
  64. Sripa P, Hayhoe B, Garg P, Majeed A, Greenfield G. Impact of GP gatekeeping on quality of care, and health outcomes, use, and expenditure: a systematic review. Br J Gen Pract. 2019;69(682):e294-e303. doi:10.3399/bjgp19X702209
  65. Pascoe S, Kirkley J, Gráboval D, Morrison-Paul CJ. Measuring and Assessing Capacity in Fisheries. Rome, Italy: Food and Agriculture Organization (FAO); 2003.
  66. Varabyova Y, Müller JM. The efficiency of health care production in OECD countries: a systematic review and meta-analysis of cross-country comparisons. Health Policy. 2016;120(3):252-263. doi:10.1016/j.healthpol.2015.12.005
  67. Ramalho A, Castro P, Lobo M, Souza J, Santos P, Freitas A. Integrated quality assessment for diabetes care in Portuguese primary health care using prevention quality indicators. Prim Care Diabetes. 2021;15(3):507-512. doi:10.1016/j.pcd.2021.01.001
  68. Sibbald B, McDonald R, Roland M. Shifting care from hospitals to the community: a review of the evidence on quality and efficiency. J Health Serv Res Policy. 2007;12(2):110-117. doi:10.1258/135581907780279611
  69. Winpenny EM, Miani C, Pitchforth E, King S, Roland M. Improving the effectiveness and efficiency of outpatient services: a scoping review of interventions at the primary-secondary care interface. J Health Serv Res Policy. 2017;22(1):53-64. doi:10.1177/1355819616648982
  70. Ahmed S, Hasan MZ, MacLennan M, et al. Measuring the efficiency of health systems in Asia: a data envelopment analysis. BMJ Open. 2019;9(3):e022155. doi:10.1136/bmjopen-2018-022155
  71. Kringos DS, Boerma WG, Hutchinson A, Saltman RB. Building Primary Care in A Changing Europe. Copenhagen, Denmark: European Observatory on Health Systems and Policies; 2015.
Volume 11, Issue 9
September 2022
Pages 1725-1734
  • Receive Date: 06 January 2021
  • Revise Date: 07 June 2021
  • Accept Date: 26 June 2021
  • First Publish Date: 26 July 2021