The Implementation of Improvement Interventions for “Low Performing” and “High Performing” Organisations in Health, Education and Local Government: A Phased Literature Review
Background There is limited understanding about whether and how improvement interventions are effective in supporting failing healthcare organisations and improving the quality of care in high-performing organisations. The aim of this review was to examine the underlying concepts guiding the design of interventions aimed at low and high performing healthcare organisations, processes of implementation, unintended consequences, and their impact on costs and quality of care. The review includes articles in the healthcare sector and other sectors such as education and local government.
Methods We carried out a phased rapid systematic review of the literature. Phase one was used to develop a theoretical framework of organisational failure and turnaround, and the types of interventions implemented to improve quality. The framework was used to inform phase 2, which was targeted and focused on organisational failure and turnaround in healthcare, education and local government settings. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement to guide the reporting of the methods and findings and the Mixed Methods Appraisal Tool (MMAT) as a quality assessment tool. The review protocol was registered with PROSPERO (CRD: 42019131024).
Results Failure is frequently defined as the inability of organisations to meet pre-established performance standards and turnaround as a linear process. Improvement interventions are designed accordingly and are focused on the organisation, with limited system-level thinking. Successful interventions included restructuring senior leadership teams, inspections, and organisational restructuring by external organisations. Limited attention was paid to the potential negative consequences of the interventions and their costs.
Conclusion Dominant definitions of success/failure and turnaround have led to the reduced scope of improvement interventions, the linear perception of turnaround, and lack of consideration of organisations within the wider system in which they operate. Future areas of research include an analysis of the costs of delivering these interventions in relation to their impact on quality of care.
Walshe K, Shortell SM. When things go wrong: how health care organizations deal with major failures. Health Aff (Millwood). 2004;23(3):103-111. doi:10.1377/hlthaff.23.3.103
Hockey PM, Bates DW. Physicians' identification of factors associated with quality in high- and low-performing hospitals. Jt Comm J Qual Patient Saf. 2010;36(5):217-223. doi:10.1016/s1553-7250(10)36035-1
Ravaghi H, Mannion R, Sajadi HS. Organizational failure in an NHS hospital trust: a qualitative study. Health Care Manag (Frederick). 2015;34(4):367-375. doi:10.1097/hcm.0000000000000087
Filochowski J. Too Good to Fail. Harlow, United Kingdom: Pearson Education Ltd; 2013.
Hafner JM, Williams SC, Morton DJ, Koss RG, Loeb JM. From bad to better: a qualitative assessment of low-performing hospitals that improved their smoking cessation counseling performance. J Clin Outcomes Manag. 2008;15(7):329-337.
Vaughn VM, Saint S, Krein SL, et al. Characteristics of healthcare organisations struggling to improve quality: results from a systematic review of qualitative studies. BMJ Qual Saf. 2019;28(1):74-84. doi:10.1136/bmjqs-2017-007573
Curry LA, Spatz E, Cherlin E, et al. What distinguishes top-performing hospitals in acute myocardial infarction mortality rates? a qualitative study. Ann Intern Med. 2011;154(6):384-390. doi:10.7326/0003-4819-154-6-201103150-00003
Brewster AL, Cherlin EJ, Ndumele CD, et al. What works in readmissions reduction: how hospitals improve performance. Med Care. 2016;54(6):600-607. doi:10.1097/mlr.0000000000000530
Tricco AC, Langlois EV, Straus SE. Rapid Reviews to Strengthen Health Policy and Systems: A Practical Guide. World Health Organization, Alliance for Health Policy and Systems Research; 2017.
Ferlie E, Crilly T, Jashapara A, Peckham A. Knowledge mobilisation in healthcare: a critical review of health sector and generic management literature. Soc Sci Med. 2012;74(8):1297-1304. doi:10.1016/j.socscimed.2011.11.042
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. doi:10.1136/bmj.b2535
Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf. 2015;24(5):325-336. doi:10.1136/bmjqs-2014-003620
Jones E, Lees N, Martin G, Dixon-Woods M. Describing methods and interventions: a protocol for the systematic analysis of the perioperative quality improvement literature. Syst Rev. 2014;3:98. doi:10.1186/2046-4053-3-98
Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117. doi:10.1186/1471-2288-13-117
Pace R, Pluye P, Bartlett G, et al. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012;49(1):47-53. doi:10.1016/j.ijnurstu.2011.07.002
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159.
Jas P, Skelcher C. Performance decline and turnaround in public organizations: a theoretical and empirical analysis. Br J Manag. 2005;16(3):195-210. doi:10.1111/j.1467-8551.2005.00458.x
Rice JK, Malen B, Baumann P, et al. The persistent problems and confounding challenges of educator incentives: the case of TIF in Prince George's County, Maryland. Educ Policy. 2012;26(6):892-933. doi:10.1177/0895904812465708
Rosenberg L, Christianson MD, Hague Angus M. Improvement efforts in rural schools: experiences of nine schools receiving School Improvement Grants. Peabody J Educ. 2015;90(2):194-210. doi:10.1080/0161956x.2015.1022109
Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety-net and non-safety-net hospitals. JAMA. 2008;299(18):2180-2187. doi:10.1001/jama.299.18.2180
Marsh JA, Bush-Mecenas S, Hough H. Learning from early adopters in the new accountability era: insights from California's CORE waiver districts. Educ Adm Q. 2017;53(3):327-364. doi:10.1177/0013161x16688064
Mannion R, Davies HT, Marshall MN. Cultural characteristics of "high" and "low" performing hospitals. J Health Organ Manag. 2005;19(6):431-439. doi:10.1108/14777260510629689
Hochman M, Briggs-Malonson M, Wilkes E, et al. Fostering a commitment to quality: best practices in safety-net hospitals. J Health Care Poor Underserved. 2016;27(1):293-307. doi:10.1353/hpu.2016.0008
Heck RH, Chang J. Examining the timing of educational changes among elementary schools after the implementation of NCLB. Educ Adm Q. 2017;53(4):649-694. doi:10.1177/0013161x17711480
Beeri I, Navot D. Turnaround management strategies in local authorities: managerial, political and national obstacles to recovery. J Manag Organ. 2014;20(1):121-138. doi:10.1017/jmo.2014.17
Beeri I. Turnaround management strategies in public systems: the impact on group-level organizational citizenship behavior. Int Rev Adm Sci. 2012;78(1):158-179. doi:10.1177/0020852311430284
Yapp C, Skelcher C. Improvement boards: building capability for public service improvement through peer support. Public Money Manag. 2007;27(4):285-292. doi:10.1111/j.1467-9302.2007.00596.x
Meyers CV, Hitt DH. Planning for school turnaround in the United States: an analysis of the quality of principal-developed quick wins. Sch Eff Sch Improv. 2018;29(3):362-382. doi:10.1080/09243453.2018.1428202
Nicolaidou M, Ainscow M. Understanding failing schools: perspectives from the inside. Sch Eff Sch Improv. 2005;16(3):229-248. doi:10.1080/09243450500113647
Orr MT, Berg B, Shore R, Meier E. Putting the pieces together: leadership for change in low-performing urban schools. Educ Urban Soc. 2008;40(6):670-693. doi:10.1177/0013124508324018
VanGronigen BA, Meyers CV. How state education agencies are administering school turnaround efforts: 15 years after No Child Left Behind. Educ Policy. 2019;33(3):423-452. doi:10.1177/0895904817691846
Beeri I. Governmental strategies towards poorly-performing municipalities: from narrow perceptions to ineffective policies. Lex localis - Journal of Local Self-Government. 2013;11(1):33-52. doi:10.4335/220
Jas P. The role of interim managers in performance improvement: evidence from English local authorities. Public Money Manag. 2013;33(1):15-22. doi:10.1080/09540962.2013.744890
Gagliardi AR, Nathens AB. Exploring the characteristics of high-performing hospitals that influence trauma triage and transfer. J Trauma Acute Care Surg. 2015;78(2):300-305. doi:10.1097/ta.0000000000000506
Finnigan KS, Daly AJ, Stewart TJ. Organizational learning in schools under sanction. Educ Res Int. 2012;2012:270404. doi:10.1155/2012/270404
Chapman C, Harris A. Improving schools in difficult and challenging contexts: strategies for improvement. Educ Res. 2004;46(3):219-228. doi:10.1080/0013188042000277296
Tsai TC, Jha AK, Gawande AA, Huckman RS, Bloom N, Sadun R. Hospital board and management practices are strongly related to hospital performance on clinical quality metrics. Health Aff(Millwood). 2015;34(8):1304-1311. doi:10.1377/hlthaff.2014.1282
Mintrop H, Trujillo T. The practical relevance of accountability systems for school improvement: a descriptive analysis of California schools. Educ Eval Policy Anal. 2007;29(4):319-352. doi:10.3102/0162373707309219
Turner D, Skelcher C, Whiteman P, Hughes M, Jas P. Intervention or persuasion? strategies for turnaround of poorly-performing councils. Public Money Manag. 2004;24(4):217-226. doi:10.1111/j.1467-9302.2004.00423.x
Chang AM, Cohen DJ, Lin A, et al. Hospital strategies for reducing emergency department crowding: a mixed-methods study. Ann Emerg Med. 2018;71(4):497-505.e494. doi:10.1016/j.annemergmed.2017.07.022
Rose L. Better Leadership for Tomorrow. London: NHS England; 2015.
Aboumatar HJ, Chang BH, Al Danaf J, et al. Promising practices for achieving patient-centered hospital care: a national study of high-performing US hospitals. Med Care. 2015;53(9):758-767. doi:10.1097/mlr.0000000000000396
Beeri I. The measurement of turnaround management strategies in local authorities. Public Money Manag. 2009;29(2):131-136. doi:10.1080/09540960902768046
Willis L. Is the process of special measures an effective tool for bringing about authentic school improvement? Manag Educ. 2010;24(4):142-148. doi:10.1177/0892020610379314
Willmott R. Structure, agency and school effectiveness: researching a 'failing' school. Educ Stud. 1999;25(1):5-18. doi:10.1080/03055699997936
Parsons C. Challenged school–challenged society: stacking the odds against the poor. Educ Rev. 2013;65(3):267-283. doi:10.1080/00131911.2013.772127
Perryman J. School leadership and management after special measures: discipline without the gaze? School Leadership & Management. 2005;25(3):281-297. doi:10.1080/13634230500116355
Perryman J. Panoptic performativity and school inspection regimes: disciplinary mechanisms and life under special measures. J Educ Policy. 2006;21(2):147-161. doi:10.1080/02680930500500138
Gorton J, Williams M, Wrigley T. Inspection judgements on urban schools: a case for the defence. Urban Rev. 2014;46(5):891-903. doi:10.1007/s11256-014-0309-2
Ehren MCM, Altrichter H, McNamara G, O’Hara J. Impact of school inspections on improvement of schools—describing assumptions on causal mechanisms in six European countries. Educ Assess Eval Account. 2013;25(1):3-43. doi:10.1007/s11092-012-9156-4
Allen T, Walshe K, Proudlove N, Sutton M. Measurement and improvement of emergency department performance through inspection and rating: an observational study of emergency departments in acute hospitals in England. Emerg Med J. 2019;36(6):326-332. doi:10.1136/emermed-2018-207941
Boyd A, Addicott R, Robertson R, Ross S, Walshe K. Are inspectors' assessments reliable? ratings of NHS acute hospital trust services in England. J Health Serv Res Policy. 2017;22(1):28-36. doi:10.1177/1355819616669736
Castro-Avila A, Bloor K, Thompson C. The effect of external inspections on safety in acute hospitals in the National Health Service in England: a controlled interrupted time-series analysis. J Health Serv Res Policy. 2019;24(3):182-190. doi:10.1177/1355819619837288
Millar R, Greenhalgh J, Rafferty AM, McLeod H, Mannion R. Towards a framework for partnering as an intervention for improvement: lessons for NHS providers from a realist synthesis of theoretical, empirical, and stakeholder evidence. Available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019149009. Published 2019.
Sahlin-Andersson K, Engwall L. The Expansion of Management Knowledge: Carriers, Flows, and Sources. Stanford: Stanford University Press; 2002.
Sahlin K, Wedlin L. Circulating ideas: imitation, translation and editing. In: The Sage Handbook of Organizational Institutionalism. Los Angeles: Sage; 2008:218-242.
Nicolini D, Mengis J, Meacheam D, Waring J, Swan J. Recovering the performative role of innovations in the global travel of healthcare practices. In: Swan J, Newell S, Nicolini D, eds. Mobilizing Knowledge in Health Care: Challenges for Management and Organization. Oxford: Oxford University Press; 2016:19-35.
McGivern G, Dopson S, Ferlie E, et al. “Epistemic Fit” and the mobilization of management knowledge in healthcare. In: Swan J, Newell S, Nicolini D, eds. Mobilizing Knowledge in Health Care: Challenges for Management and Organization. Oxford: Oxford University Press; 2016:23.
Millar R, Mannion R, Freeman T, Davies HT. Hospital board oversight of quality and patient safety: a narrative review and synthesis of recent empirical research. Milbank Q. 2013;91(4):738-770. doi:10.1111/1468-0009.12032
Jones L, Pomeroy L, Robert G, Burnett S, Anderson JE, Fulop NJ. How do hospital boards govern for quality improvement? a mixed methods study of 15 organisations in England. BMJ Qual Saf. 2017;26(12):978-986. doi:10.1136/bmjqs-2016-006433
Jones L, Pomeroy L, Robert G, et al. Explaining organisational responses to a board-level quality improvement intervention: findings from an evaluation in six providers in the English National Health Service. BMJ Qual Saf. 2019;28(3):198-204. doi:10.1136/bmjqs-2018-008291
Vindrola-Padros, C., Ledger, J., Barbosa, E. C., & Fulop, N. J. (2022). The Implementation of Improvement Interventions for “Low Performing” and “High Performing” Organisations in Health, Education and Local Government: A Phased Literature Review. International Journal of Health Policy and Management, 11(7), 874-882. doi: 10.34172/ijhpm.2020.197
MLA
Cecilia Vindrola-Padros; Jean Ledger; Estela Capelas Barbosa; Naomi J. Fulop. "The Implementation of Improvement Interventions for “Low Performing” and “High Performing” Organisations in Health, Education and Local Government: A Phased Literature Review". International Journal of Health Policy and Management, 11, 7, 2022, 874-882. doi: 10.34172/ijhpm.2020.197
HARVARD
Vindrola-Padros, C., Ledger, J., Barbosa, E. C., Fulop, N. J. (2022). 'The Implementation of Improvement Interventions for “Low Performing” and “High Performing” Organisations in Health, Education and Local Government: A Phased Literature Review', International Journal of Health Policy and Management, 11(7), pp. 874-882. doi: 10.34172/ijhpm.2020.197
VANCOUVER
Vindrola-Padros, C., Ledger, J., Barbosa, E. C., Fulop, N. J. The Implementation of Improvement Interventions for “Low Performing” and “High Performing” Organisations in Health, Education and Local Government: A Phased Literature Review. International Journal of Health Policy and Management, 2022; 11(7): 874-882. doi: 10.34172/ijhpm.2020.197