A Continuous Quality Improvement Intervention to Improve Antenatal HIV Care Testing in Rural South Africa: Evaluation of Implementation in a Real-World Setting

Background

Continuous quality improvement (CQI) provides a range of time-tested and adaptable techniques to diagnose and manage quality problems in clinics using real-time data. It provides clinical teams with the skills they need to implement evidence-based practice. CQI’s potential to improve quality of health services within available resources makes it an attractive intervention in resource-limited settings¹ particularly in relation to Sustainable Development Goals.^2,3 There is a vast literature on “quality improvement” initiatives including audit and feedback,^4,5 health worker training and supervision,^6-8 Lean,^9-11 Six Sigma,^9,10,12,13 Lean Six Sigma,^10,14 CQI ([Total Quality Management], the Model for Improvement),^10,15-17 and others,^18,19 all aimed at improving service quality. We defined CQI²⁰ as that which applies a set of standardised tools used flexibly based on contextual needs, specifically: root-cause analyses performed with process maps²¹ and fishbone diagrams,²² then design and testing of local solutions using iterative Plan-Do-Study-Act (PDSA) cycles²³ with the aid of real-time data trends plotted on run charts.²⁴ Although there are some randomised controlled trials (RCTs) underway that test CQI as a single intervention in resource-limited primary care settings,^25,26 to our knowledge there are only 2 completed trials of CQI in such settings.^27-29 Despite the promise of CQI and rigorous evaluation, these studies failed to demonstrate a positive impact, instead identifying several constraints limiting uptake. Constraints included challenges with staff turnover, lack of available leadership, inadequate ongoing support,²⁷ challenges with data quality, staffing shortages, weak infrastructure, and civil unrest.^28,29

In settings with high pregnancy and postpartum HIV prevalence and incidence such as South Africa^30-32 — where prevalence of pre-treatment drug resistance is also increasing³³ — confirming response to antiretroviral therapy (ART) among women living with HIV, timely revision of the ART regimen among those failing treatment,^34,35 and early diagnosis of incident HIV for timely ART initiation among women previously not living with HIV,³⁶ are essential. Procedural measures of quality on this care pathway are HIV viral load (VL) monitoring of pregnant women living with HIV, and repeat HIV testing of pregnant women not living with HIV. These are critical components of elimination of mother-to-child transmission of HIV (eMTCT) programmes^37,38 alongside optimising ART adherence^39-41 and primary prevention of maternal HIV infection.^41-43 South Africa decentralised HIV care to primary care nurses in order to broaden ART coverage,⁴⁴ and removed CD4+ T-cell eligibility criteria for pregnant and breastfeeding women in January 2015 (Option B+); the guidelines also recommend more frequent HIV VL monitoring and repeat HIV testing for this population.⁴⁵ In September 2016 ART was expanded to all people living with HIV regardless of CD4 count (Universal Test and Treat, UTT).⁴⁶ Amidst a rapidly evolving ART programme and concerns about implementation gaps of these essential HIV care tests,^{31,32,34,47,48} improving antenatal HIV care quality within available resources is crucial.

We implemented CQI in 7 primary care clinics in rural South Africa between July 2015 and January 2017. We evaluated CQI using a stepped-wedge cluster RCTs, with an embedded process evaluation, to investigate the impact of CQI on quality of antenatal HIV services in rural South Africa.⁴⁹ Our primary endpoints were direct process indicators of antenatal care (ANC) quality designed to eliminate MTCT: (i) HIV VL monitoring among pregnant women living with HIV, and (ii) repeat HIV testing among pregnant women not living with HIV.⁴⁹ These process indicators, if improved, are expected to have a downstream impact on MTCT. In our primary impact evaluation we found that CQI significantly increased VL monitoring but not repeat HIV testing. Importantly, there were gaps in clinical documentation and HIV care testing rates fell short of expected targets.⁵⁰

Process evaluations can highlight determinants of intervention uptake and ‘normalisation’ within the broader health system, to inform policy planning prior to scale-up of a seemingly desirable intervention.⁵¹

In this paper we present results of our process evaluation of CQI as implemented in our stepped-wedge cluster RCT. We aimed to investigate the process through which the main trial endpoints of VL monitoring and repeat HIV testing were achieved. In particular, we aimed to identify determinants of practice, and whether ‘normalisation’ of CQI into routine services could occur in this setting, by examining the following: (i) health worker participation in CQI by describing ‘dose’ and ‘reach’; (ii) the ‘black box’ of implemented changes in practice; (iii) time trends in endpoint achievements and time to intervention uptake; and (iv) CQI mentor and health worker experiences of implementing the intervention.

Methods

We first summarise the Management and Optimisation of Nutrition, Antenatal, Reproductive, Child health and HIV care (MONARCH) CQI intervention before describing the process evaluation study design and associated methodology.

The MONARCH CQI Intervention

The MONARCH (NCT02626351) trial has been previously described in detail.⁴⁹ Briefly, the intervention supported nurses and other health workers providing ANC in primary healthcare clinics to identify and implement approaches to improve adherence to 2015 South African national eMTCT guidelines (Table 1), particularly HIV VL monitoring and repeat HIV testing (HIV care tests). Table 1 compares the key changes in VL monitoring and HIV testing guidelines from 2013 to 2015: this helps us interpret the pre-intervention testing rates for each primary endpoint and effort required to integrate new knowledge into routine practice.

A team of 3 certified CQI mentors (2 isiZulu-speaking nurses and a data clerk) — supported by a data manager, improvement advisor, and scientific advisor (Centre for Rural Health, CRH, University of KwaZulu-Natal) — travelled to the study site from Durban to deliver the intervention.⁴⁹ All CQI mentors had previous experience in supporting CQI in South African healthcare facilities elsewhere. They identified a clinic team of 4-7 health workers at each study clinic (clinic CQI team), guided by the clinic operational manager. They worked collaboratively to guide the clinic CQI team to identify areas for improvement and test solutions. They used standardised CQI tools including fishbone diagrams,²² clinical workflow maps (process maps),²¹ PDSA cycles,²³ and run charts.²⁴ Change ideas were implemented flexibly according to clinic needs, and were shared between clinics at the end of each intervention step during action learning sessions.¹⁷ Routine clinic registers were sourced to identify women eligible for HIV care tests during CQI activities, as antenatal medical records (maternity case records, MCRs) are retained by pregnant women until delivery. The same registers were accessed to track CQI progress using run charts.

Theory of Change

As previously described,⁴⁹ we hypothesized that this complex intervention would improve clinical processes through support provided by CQI mentors to health workers, which, supplemented by real-time data on performance, would enhance motivation. Collaborative root-cause analyses of clinical process gaps would enable health workers to identify and implement simple workable solutions. Health workers’ ability to continue routine clinical activities in parallel, as well as willingness and availability to participate in CQI were assumed.

Study Design

To investigate the process of implementing CQI in this context we used convergent mixed methods,⁵² guided by Normalisation Process Theory (NPT)⁵³ and the Tailored Implementation of Chronic Diseases (TICD) checklist.⁵⁴

NPT was developed to explain whether and how new technologies are taken up by healthcare teams.⁵³ This framework allows investigation of how contextual factors such as resources, organisational culture and hierarchy, enable a new approach to be implemented into routine practice and thus ‘normalised.’ It suggests that successful integration requires 4 sorts of work for teams: (i) to make sense of the new approach (render it coherent); (ii) to maintain their own engagement with and delivery of the new approach and involve others (cognitive participation); (iii) to actually do the work of the new approach and ensure others are doing it (collective action); and (iv) monitor how well the approach is being implemented and thus its effectiveness (reflexive monitoring).⁵³

The TICD checklist was designed to identify determinants of practice to facilitate developing context-specific tailored interventions to improve healthcare quality. The checklist identifies factors that could influence uptake of interventions, organised in 7 broad domains: guideline factors, individual healthcare professional factors, patient factors, professional interactions, incentives and resources, capacity for organisational change, and social, political and legal factors.⁵⁴

Study Setting

The study was located in the Hlabisa sub-district of KwaZulu-Natal, 220 km north of Durban, South Africa. HIV prevalence among women of reproductive age is ~37%.⁴⁷ The local sub-district hospital, Hlabisa Hospital, oversees management of all 17 primary healthcare clinics in the area — this includes processing laboratory tests including VL, and provision of drug supplies. VL monitoring is routinely available at all clinics in the area. Clinics are nurse-led and receive medical officer support from Hlabisa Hospital about weekly. Clinical care is provided by professional nurses who are supervised by an operational manager (senior professional nurse). Professional nurses are assisted by lay counsellors (who perform HIV counselling and testing) and enrolled nurses. Data capturers are primarily responsible for capturing routine clinic data onto the national ART programme database, TIER.Net.

The process evaluation was conducted at all 7 primary healthcare clinics participating in the MONARCH trial.

Data Sources

We extracted data on ‘dose,’ ‘reach,’ how each clinic implemented CQI, and CQI mentor experiences of implementation from the CRH CQI mentor field notes. These notes detailed actual visit dates and type, attendance registers, root-cause analyses, improvement interventions (including start and review dates of PDSA cycles), and successes and challenges of implementation (including impressions of health worker understanding of eMTCT guidelines).

We sourced descriptive primary endpoint data from medical records of women ≥18 years old at delivery.⁴⁹

Health worker experiences of implementing CQI were gathered from 15-30 minute semi-structured interviews conducted one month after the Intervention step at a particular clinic. We invited available health workers to interview, targeting those in leadership roles such as the operational manager where possible. Interviews were audio-recorded with consent and transcribed verbatim. The interview topic guide was informed by NPT (Supplementary file 1).

Data Analysis

Our analysis proceeded in 5 steps. First, to understand ‘dose’ and ‘reach’ of CQI we summarised clinic size and setting, planned versus actual number of CQI meetings (‘dose’), and health worker participation in CQI meetings (‘reach’).

Second, to understand the ‘black box’ of implemented changes in practice, we summarised key root causes of poor HIV care testing and implemented change ideas.

Third, we summarised start and review dates of the first PDSA cycle as a proxy estimate of time to intervention uptake and assimilation. We then described each primary endpoint achievement by time step at each clinic, (time trends) to better understand the delayed CQI intervention effect we identified in our quantitative impact evaluation.^50,55 The proportion of women living with HIV receiving a VL test ever in pregnancy, and proportion of women not living with HIV receiving a repeat HIV test ever in pregnancy, were estimated per clinic per time step. Women were allocated to a time step according to date of delivery. We assigned women to a particular clinic based on the first antenatal clinic visited.⁵⁰

Fourth, we undertook a framework analysis of data from all CRH reports and field notes, to understand determinants of CQI implementation and ‘normalisation’ from the perspective of the CQI mentors. These documents were reviewed several times by one of the authors (HMY) to develop a coding frame: (i) knowledge and understanding of eMTCT guidelines; (ii) buy-in to CQI and willingness to acknowledge gaps in clinical practice; (iii) staff handover and communication; (iv) steps to identify and track women eligible for each test; (v) steps to follow up test results; (vi) routine data quality; (vii) availability of necessary supplies and space; (viii) availability of necessary healthcare personnel; and (ix) patient factors. This coding frame was applied across all collated documents to identify potential drivers of low HIV care testing prior to intervention start and factors influencing implementation. The codes were then organised using the TICD framework.⁵⁴ How each factor may have influenced intervention delivery was also elucidated.

Fifth, transcripts from semi-structured interviews — to understand determinants of CQI implementation and ‘normalisation’ from the clinic health worker perspective — were reviewed several times (HMY), and a coding frame guided by NPT⁵³ with additional emergent themes was developed. The overlapping codes included: (i) coherence (did health workers understand differences in CQI compared with old ways of working, did they understand, what they were expected to do, did they believe it might work); (ii) cognitive participation (were health workers willing to engage with the new activities, was there ownership); (iii) actual feasibility of implementing CQI and confidence in its potential (collective action); (iv) reflexive monitoring (impact on personal role, impact on patients, sustainability of CQI); (v) other contextual information including gaps in clinical processes; (vi) communicating with colleagues; (vii) tracking patients; (viii) insights on rationale for national guidelines on VL and repeat HIV testing; (ix) challenges implementing UTT. Barriers and facilitators of CQI implementation were then identified according to the TICD framework.⁵⁴

Finally, quantitative and qualitative data were summarised by clinic, using a mixed methods matrix.⁵⁶

Results

We first present CQI participation by clinic health workers followed by the ‘black box’ of implemented changes. Second, we summarise time to PDSA cycle start and review, and variation in endpoint achievement by time step. Third, we present CQI mentor and health worker experiences of CQI implementation using the TICD framework. Finally, we provide an overview of the findings using a mixed methods matrix.

Health Worker Participation in CQI: “Dose” and “Reach”

The main resource investment for CQI implementation was CQI mentor and clinic health workers’ time (Table 1 in our linked primary impact evaluation).⁵⁰ All clinics participated in CQI activities after randomisation and staff were trained on all CQI tools. Almost all scheduled CQI meetings took place as planned, and all clinics utilised all CQI tools. Buy-in to CQI and the CRH team was strong with several clinics requesting additional visits (Table 2) — this resulted in a higher total ‘dose.’ Health worker participation in CQI visits (‘reach’) was less than anticipated, particularly among senior staff cadres (Table 3) — this was largely due to competing clinical commitments or staffing shortages (Table S3). Given challenges with attendance and staff turnover, training on CQI tools and PDSA cycles had to be repeated. Apart from expected contextual adaptation, the intervention was not modified.

The “‘Black Box’” of Implemented Practice Changes

The CQI mentors trained all clinics on all CQI tools and utilised them consistently during implementation. At the beginning of each intervention step, root-cause analyses were conducted with process maps of patient flow and fishbone diagrams as part of a situational analysis at each clinic. There were discrepancies between monthly monitoring and evaluation (M&E) summary statistics and clinic registers, and incompleteness, suggesting problems with data quality (Supplementary file 1).

Each clinic had different organisational challenges (Table S1). There were differences in patient workload, building size, lay counsellor absence (one clinic), and computer breakdown (4 clinics). Common root causes — across most clinics — for low HIV care testing rates included poor clinical documentation, poor filing of VL results, and lack of a patient tracking system. Although the implemented change ideas (eg, patient tracking notebook for VL monitoring and repeat HIV testing) were similar (Figure 1), it was essential to adapt each solution to the clinic context. This resulted in heterogeneity of implementation.

Close

Figure 1. Drivers of Low VL Monitoring and Repeat HIV Testing Rates and Implemented Improvement Activities. (a) change ideas and package for improving VL monitoring; (b) change ideas and package implemented for improving repeat HIV testing; (c) change ideas and package implemented for other interventions including patient flow to streamline waiting times (therefore increased likelihood of patients being available for VL or HIV testing) and data quality. *All pregnant women living with HIV had a separate ART file based at the clinic, as did other individuals in the general ART programme. However, the MCR is retained by women until delivery and functions as a key communication modality between health workers particularly given that women may deliver at a facility other than their ANC clinic. Diagram format adapted from Osibo et al.28 Abbreviations: ANC, antenatal care; ART, antiretroviral therapy; eMTCT, elimination of mother-to-child transmission; MCR, maternity case record (antenatal medical record); VL, viral load.

Time to First PDSA Cycle Uptake and Time Trends in Endpoint Achievement

Timing of PDSA Cycle Reviews

Table S2 summarises timing of the first PDSA cycle in relation to intervention rollover and review of the first PDSA cycle, by clinic. Larger clinics experienced delays in starting and/or reviewing (55-63 days) their first PDSA cycle whereas the smallest clinic with low clinical workload was able to rapidly start and review their improvement activities (5-7 days).

Descriptive Primary Endpoint Achievements by Time Step

Medical records from 2160 women who delivered were analysed. HIV prevalence was 47%.⁵⁰ Over the whole study, 56% women living with HIV had a VL performed ever in pregnancy (of whom 52% had a documented result) and 94% had an ART prescription. Repeat HIV testing among pregnant women not living with HIV was 67%.⁵⁰

There was an overall improvement in the HIV VL endpoint in post-intervention steps compared with pre-intervention steps (Figure 2a), consistent with our quantitative impact evaluation.⁵⁰ There was considerable fluctuation in repeat HIV testing in post-intervention steps compared with pre-intervention. A higher pre-intervention repeat HIV testing rate was also evident (Figure 2b) and may explain the lack of CQI effect on improving repeat HIV testing in our quantitative impact evaluation.⁵⁰

Close

Figure 2. Proportion of Participants Achieving Primary Endpoints Ever During Pregnancy, Per Clinic Per Step.Thick black lines mark the point of rollover to the intervention. (a) proportion of women living with HIV who had a VL test ever in pregnancy; (b) proportion of women not living with HIV who had a repeat HIV test ever in pregnancy. Abbreviation: VL, viral load. N/A: no observations during that step. As clinics 3a and 3b had a small number of participants per step, their proportions should be interpreted with caution Clinics are listed in order of randomisation.

Determinants of CQI Implementation And “Normalisation”: CQI Mentor and Clinic Health Worker Experiences (TICD Framework)

All clinics participated in process evaluation data collection. Semi-structured interviews with health workers lasted 15-60 minutes based on health worker availability at the time. One to 3 health workers were interviewed per clinic based on availability, totalling 16 interviews across all facilities: 2 operational managers, 4 professional nurses, 2 enrolled nurses, 3 data capturers, 2 nutrition advisors and 3 lay counsellors were interviewed.

Below, we summarise CRH CQI mentor experiences (detailed in Table S3) and health worker interviews (detailed in Table S4).

Guidelines Factors

Guidelines factors may be considered in 2 categories: South African national eMTCT guidelines, and the CQI intervention as a new practice ‘guideline.’

South African National eMTCT Guidelines and Other Factors: Because the focus of CQI was national eMTCT guidelines implementation, the guidelines were an enabler of the intervention. During the last few months of the MONARCH trial (September 2016), South Africa rolled out UTT,⁴⁶ with likely knock-on effects on CQI throughout clinics due to increased demand on HIV services (Tables S3, S4).

In March 2016, routine national Department of Health (DoH) M&E registers were overhauled for efficiency and the new registers no longer included an antenatal VL monitoring indicator: health workers had to depend entirely on the informal notebook created for patient tracking (Figure 1). Thus barriers to CQI activities included temporary disruptions during retraining on the new M&E registers and additional documentation for maintaining the HIV VL tracking notebook.

CQI methodology as a ‘guideline’: All interviewed health workers reported feeling supported by the CRH team and enjoyed working with them (Table S4). At the final action learning session several health workers expressed a wish to continue working with the CRH team after project end, as they felt they needed ongoing support. Visible improvements in practice (such as improved teamwork with identifying eligible women for HIV care tests) were also enablers of the intervention (Tables S3, S4). However, feasibility was challenged due to increased effort to implement improvement activities (such as better clinical documentation and implementing the patient tracking system), and incompatibility of routine DoH M&E registers with our primary endpoints although the latter were direct applications of national eMTCT guidelines. As a result, delays in implementing and/or reviewing the first PDSA cycle were noted at some clinics (Table S2).

Individual Health Professional Factors

Key enablers of CQI implementation as reported by respondents, were buy-in to CQI methodology and motivation to improve quality of clinical services. All interviewed health workers found CQI methodology to be novel and interesting, and appreciated working with the CQI mentors. Some highlighted that CQI identified gaps in their current clinical practice and was an ‘eye-opener’ (Table S4). Although gaps in eMTCT domain knowledge were observed (Tables S1, S3), health workers reported better motivation to follow up patients once they understood the rationale of the guidelines (Table S4). However, challenges were many: limited self-efficacy from needing operational manager approval to implement any improvement activity, the need for repeated training and supervision to improve routine data quality, and poor clinical documentation in medical records (Tables S3, S4).

Patient Factors

Respondents also identified patient needs and behaviours that contributed to CQI feasibility. Good patient rapport with health workers in the smaller more rural clinics (Table S3), and being proactive at following up results with their health worker (Tables S3, S4) may have facilitated better HIV care testing. However, some patients prematurely departed the clinic due to long queues, missing their HIV care tests (Tables S3, S4). Other patients’ mobile phones were inoperative or the number was incorrect, making it difficult to schedule follow-up visits. Health workers noted that many women first register for ANC later in pregnancy, making it difficult to conduct HIV care tests according to guidelines (Table S4). Finally, some patients were inflexible with changes to clinic workflow and lodged complaints (Table S3).

Professional Interactions

Staff interactions were important for disseminating CQI skills and continuity of improvement activities. Enablers were teamwork and collegiality (Tables S3, S4). However, professional hierarchy hindered implementation even with good teamwork: it was difficult for lower cadre staff in some clinics to share new knowledge with their superiors, even though the former were more available to participate in CQI activities (Table S3). The operational manager was essential for decision-making to start a new activity (Tables S3, S4). Dissemination of CQI skills from clinic CQI team members to non-members was inconsistent. Data quality improvement and patient follow-up activities needed rigorous communication between different cadres of staff but were not implemented consistently during heavy clinical workload or staffing shortages. Communication and handover were additional challenges when an individual responsible for a particular activity went on leave (Table S3).

Incentives and Resources

There were many resource constraints that likely impacted intervention implementation and general health service delivery. Most notable was staffing shortages in all clinics, particularly professional nurses — the shortages were worsened by resignations and 2 deaths (Table S3). As professional nurses are the main health workers providing ante- and post-natal care, their limited participation in CQI (Table 3) would have reduced translation of improvement activities to clinical care. Operational manager unavailability due to other commitments delayed approval of improvement activities. Opportunities for HIV retesting were missed during lay counsellor study leave – for example in one clinic ~120 eligible women not living with HIV did not receive repeat HIV tests over a 3-month period (Table S3). Additional constraints included limited building space, computer breakdown, and lack of printing equipment and essential clinical monitoring forms (Tables S1, S3). HIV test kit stockouts were observed in 4 clinics and ART stockouts in one (Table S3). One nurse highlighted the importance of hiring more lay counsellors as it was difficult to take on the additional responsibility of HIV testing and counselling with their existing workload (Table S4). These constraints also contributed to delays in implementing and/or reviewing PDSA cycles.

Capacity for Organisational Change

The CQI mentors provided support for organisational change. However, limited operational manager availability may have reduced capacity for organisational change, as the operational manager was required to approve all activities. Professional hierarchy may have reduced individual health worker self-efficacy and ownership of change (Tables S3, S4).

Health workers overcame their initial resistance to change, given the need to improve the quality of patient care, and, as we have seen, in interviews were enthusiastic about CQI. While some believed CQI was sustainable, others highlighted that staff transfers to other facilities (resulting in loss of CQI memory at the facility) or lack of leadership to motivate a culture of quality, would reduce sustainability of CQI. Moreover, they believed an ongoing external stimulus from the CQI mentors or elsewhere (such as district supervisors) would be necessary to continue motivating them to continue CQI activities as CQI was not yet embedded in their practice; therefore the lack of such a stimulus was expected to negate progress in the future (Table S4).

Social and Political Factors

In South Africa, lay counsellors have provided HIV counselling since 1995⁵⁷ and HIV testing since 2010,⁵⁸ predominantly funded by donors. To formally recognise lay counsellor employment status within the DoH, the KwaZulu-Natal DoH redeployed lay counsellors to different roles in 2015.^59,60 As pointed out by one nurse, it was challenging to take on that extra work during lay counsellor absence (Table S4).

Determinants of CQI Implementation and “Normalisation”: Mixed Methods Matrix

Fidelity of implementation was high from the side of the CQI mentors: all clinics were trained on all CQI tools and utilised them, the mentors adhered to the study timeline and minimum CQI dosage was exceeded. They also worked hard to encourage health worker participation in a supportive manner.

Conversely, fidelity was lower from the side of clinic health workers. Health workers were enthusiastic about CQI and reported better motivation through understanding eMTCT guidelines. Key challenges influencing implementation included staffing shortages and turnover, data quality challenges, poor clinical documentation, needing operational manager approval for implementing CQI activities, and patient factors such as early departure from clinics due to long queues. There were additional resource constraints such as shortage of clinical monitoring forms and HIV test kits (Table 4).

Table 4 summarises factors influencing delivery and ‘normalisation’ of CQI by clinic. It shows that health worker participation in CQI (‘reach’) was limited, particularly among key staff. Many clinics were unable to immediately improve VL monitoring during the intervention time step according to the stepped-wedge design, and several clinics had no improvement in repeat HIV testing. There were delays in implementing the first PDSA cycle and/or review of the first PDSA cycle after implementation, particularly in larger clinics.

Discussion

We demonstrate that CQI can be an effective means of critical self-appraisal and process improvement in resource-limited nurse-led primary care clinics. The implemented workflow improvements were simple. Our findings complement our quantitative impact evaluation which found that CQI had a significant impact on improving HIV VL monitoring among pregnant women living with HIV, but did not improve repeat HIV testing among pregnant women not living with HIV.⁵⁰ Although we expected that all pregnant women would receive the appropriate HIV care test at least once in pregnancy in this HIV hyperendemic setting, actual achievements fell well short. Despite health worker enthusiasm for CQI and the CQI mentors, and a higher-than-expected ‘dose’ of CQI, ‘reach’ was limited as clinic health workers had difficulty finding time to consistently attend meetings. Whilst implementation fidelity was high on the part of the CQI mentors, fidelity was lower from the side of the clinic health workers attributable to several health system limitations.

There were many challenges which reduced ‘reach’ and fidelity of CQI implementation. Within the broader health system and local context, some issues were ‘fixed’ (such as small clinic buildings, paper-based M&E systems, general staffing shortages, gaps in routine data quality, and redeployment of lay counsellors). Superimposed on these were several short and medium-term ‘shocks’: the death of 2 nurses and resignations without replacement, an overhaul of the routine M&E system, and the rollout of UTT.

Staffing shortages particularly influenced health worker availability to participate and engage in CQI meetings and improvement activities (reduced ‘reach’). Knowledge gaps of the content and rationale of 2015 eMTCT guidelines would have further delayed improvements. Unavailability of the operational manager delayed the decision to implement and review improvement activities. Challenges limiting health workers’ ability to identify patients eligible for HIV care tests included poor clinical documentation and routine data quality, and the paper-based M&E system (requiring rigorous documentation and communication between staff cadres). Poor data quality would have also reduced accuracy of estimated monthly HIV care testing targets and estimates of progress towards those targets. The ART stockouts highlight a broader health system issue prevalent in several sub-Saharan African countries.^61-64 Given the staffing shortages, it is likely that many clinics in our study had difficulty undertaking proactive stocktaking to maintain continuous supplies. There may have been higher-level supply chain challenges as well, albeit outside the scope of our study. However, it is possible that supply chain management at all levels, is amenable to improvement initiatives as demonstrated elsewhere in sub-Saharan Africa.^65,66 Many of these factors reflect the importance of adequate time and support for health workers to improve knowledge, skills, and maintain CQI activities.

The lack of improvement in repeat HIV testing warrants additional consideration. Clinics were conducting repeat HIV testing of pregnant women at 32 weeks’ gestation following an HIV screen at first antenatal visit based on 2013 eMTCT guidelines (Table 1), resulting in relatively higher pre-intervention repeat HIV testing. Furthermore, HIV counselling and testing may have become embedded as an exclusively lay counsellor role prior to their redeployment, therefore other staff may have been reluctant to take on the task, as described during interviews. Patient characteristics such as lack of perceived risk,⁶⁷ presenting for ANC late in pregnancy, or leaving the clinic early due to long queues may have also contributed. The period of HIV test kit stockouts, albeit brief, may have played a role.

These different determinants of implementation would have had a different net effect on intervention fidelity at each clinic. Although some process improvements may have been immediate, comprehensive assimilation of CQI clinic-wide is likely to have taken time beyond the duration of our study if health workers remained committed and had time available. The health worker and CQI mentor reports indicate that CQI may not be normalised in this resource-limited context, without ongoing external support from CQI mentors, leadership, and ownership of change and investment in operational capacity. To further expand on the issue of leadership, clinic managers may benefit from onsite mentorship to enhance their advanced planning, problem solving, priority setting, and leadership and management skills as described in some studies.^68-70 Such capacity building is likely to strengthen clinic operations — and health worker availability to participate in CQI activities regularly — whilst improving staff skills, morale, and patient health outcomes. This is achieved through shifting towards a more proactive approach to clinical service provision than a reactive approach.⁶⁸

Comparison With Other Studies

Our findings are supported by other published studies of CQI in sub-Saharan African primary care settings. A trial in Nigeria identified enthusiastic participation in CQI as a key driver of success.²⁸ While patient waiting times were reduced with CQI, establishing new improvement processes including data quality, was slow. Similarly, a trial of CQI implementation in Malawi was challenged by staff turnover, lack of available leadership, and inadequate ongoing support.²⁷ Both studies highlighted the need for extra time beyond the study schedule for full uptake of CQI. A quasi-experimental CQI study in Tanzania and Uganda experienced routine data quality challenges, shortage of medical supplies, and lack of health worker prioritisation of one endpoint.⁷¹

Implementing CQI as part of the expanding eMTCT programme in South Africa and elsewhere has had successes and challenges.^72-75 Three studies demonstrated improvements in ANC quality of care indicators,^72,74,75 and one showed improved uptake of skilled health worker deliveries.⁷⁴ However, another study encountered several challenges including resistance to CQI methodology and external CQI mentors (resulting in considerable time delays), and randomisation sometimes selecting out willing participants.⁷³ The CQI mentors and methodology were very welcome in our study, perhaps because we had fewer participating facilities and held meetings at each clinic to introduce CQI before the trial began.

Other CQI evaluations in non-primary care settings in sub-Saharan Africa highlight similar considerations. Despite good acceptance of the intervention, there were data quality challenges,⁷⁶ poor roads during bad weather conditions affecting patient access to services,⁷⁶ staffing shortages,⁷⁷ high staff turnover,⁷⁷ and patient-level barriers such as stigma.⁷⁷

Strengths and Limitations

This study adds to the emerging evidence base for determinants of success in implementing CQI in resource-limited primary care settings. Conducting data collection in parallel to the main trial helped reduce measurement bias and optimised recall of health workers’ experiences. The complementary quantitative and qualitative data sources provided nuanced perspectives of CQI implementation. We were able to describe antenatal HIV health services in the area in detail with important implications for policy-makers. We were also able to obtain insights into time delays of CQI implementation, which may inform future research study designs.

Our process evaluation had some limitations. We conducted final analyses and synthesis of data from the process evaluation after we knew the primary results of the stepped-wedge cluster RCT. Our interpretation of the data may have been different if we had conducted analyses for the impact and process evaluations separately. Although we identified features of the intervention indicative of whether it would be sustainable, we were unable to measure long term sustainability after the main trial ended. Measuring intervention sustainability is a crucial part of a study such as ours, given the interest in CQI by policy-makers elsewhere in South Africa,^72,78 and is an important consideration for design of future studies. Finally, although implementation fidelity was high on the part of the CQI mentors, we did not measure the quality of their mentorship. It is possible that the quality of mentorship may not have been as high as expected and that the CQI mentors themselves needed additional training and supervision. However, given the close support by the improvement advisor during implementation,⁵⁰ this is unlikely to have been a major issue.

Policy Implications

Our process evaluation of CQI in resource-limited primary care settings demonstrates that it is possible to implement simple solutions to improve quality of services. The team of local CQI mentors, flexible with accommodating additional requests for support, were able to build and maintain rapport with health workers.

Although the main resources investment for CQI is effort, prior to scaling up such a potentially desirable intervention, it is important to consider the intervention in the context of the broader health system. This is because there may be positive or detrimental effects on other services or components of the health system during scale up, including quality of other clinical services, funding, or other resources.

Our findings on some of the health service limitations are also relevant to the goal of eMTCT. VL monitoring must improve substantially to (i) document actual ART response rather than rely on self-reported adherence; (ii) timely revise a failing ART regimen; (iii) optimise infant prophylaxis; and (iv) prevent onward transmission of HIV, particularly for women initiating ART later in pregnancy,^34,79-81 and particularly given routine availability of the test. A recent systematic review identified that VL monitoring in resource-limited settings can be cost-effective if implemented judiciously and results acted upon.⁸² However, VL monitoring must be implemented alongside other interventions to facilitate and maintain VL suppression, including adherence to ART.⁴⁰ Early diagnosis of incident HIV in pregnancy is also essential to allow early initiation of ART and VL suppression by delivery,^31,36 therefore repeat HIV testing is essential to identify women who seroconvert during pregnancy. Finally, primary prevention of HIV should form part of the eMTCT programme, as demonstrated in countries that succeeded in eMTCT such as Cuba and Thailand.^41,42

Conclusions

CQI is a flexible method for self-appraisal and process improvement in healthcare services. Despite health worker enthusiasm, staffing and other resource shortages limited ‘reach’ and fidelity of CQI in our study. CQI requires considerable support from the health system to realise its full potential in resource-limited settings.^3,83 Strategies to empower junior staff to take ownership of change, better support from clinic managers, better communication between staff cadres, and efforts to improve routine data quality are important for maximal impact of improvement efforts. There is also an argument for reinstating lay counsellors or increasing nursing resource capacity in this setting. Moreover, designing routine M&E data collection instruments compatible with available clinical guidelines is critical to facilitate patient follow-up where electronic systems are not yet available.

An important next step is to identify and rigorously test context-specific and sustainable solutions with a wide range of stakeholders, to address these health systems gaps — while considering the impact of the intervention on other components of the health system⁸⁴ —to provide high quality healthcare services in resource-limited primary care settings.

Acknowledgements

The authors wish to thank the UKZN Centre for Rural Health for their efforts in implementing the CQI intervention, South African National Department of Health partners including primary health workers, and MONARCH study participants. We also extend our thanks to all colleagues at AHRI for their support with programme operations

Ethical issues

Ethical approval for the study was obtained from the University of KwaZulu-Natal Biomedical Research Ethics Committee (BREC, reference BE209/14). Approvals were obtained from district and sub-district level DoH staff prior to commencing the CQI intervention. Written informed consent was obtained from health workers prior to conducting semi-structured interviews. Standard DoH approvals for commencing the study were also obtained as part of a Memorandum of Understanding between AHRI and the Department of Health.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

SW designed the process evaluation study in collaboration with TB, HMY, WDM, MM, and DP. HMY coordinated the main trial, performed the analysis and wrote the manuscript. HMY, WDM, MM, JWDN, CH, AJ, KP, FAP, DP, TB, and SW reviewed and edited the manuscript. TB is the principal investigator of the MONARCH trial. HMY and CH coordinated data collection for the process evaluation. All authors read and approved the final manuscript.

Funding

The MONARCH programme was co-funded by the Delegation of the European Commission to South Africa, EuropeAid/134286/L/ACT/ZA and the Wellcome Trust (through core funding to AHRI). The contents of this document are the sole responsibility of the authors and their affiliated institutions, and can under no circumstances be regarded as reflecting the position of the European Union.

AHRI receives core funding from the UK Wellcome Trust grant 082384/Z/07/Z and Howard Hughes Medical Institute. The AHRI Population Intervention Platform is partially funded by the South African Population Research Infrastructure Network (SAPRIN), South African Department of Science and Technology.

HMY is supported by an Australian Government Research Training Program (RTP) Scholarship, University of New South Wales, Sydney, Australia. The Kirby Institute is funded by the Australian Government Department of Health and Ageing, and is affiliated with the Faculty of Medicine, UNSW Sydney. JWDN is supported by the Alexander von Humboldt Foundation. TB is supported by the Alexander von Humboldt Professor award, funded by the Federal Ministry of Education and Research; the Wellcome Trust; and the NICHD of NIH (R01-HD084233), NIA of NIH (P01-AI112339), as well as FIC of NIH (D43-TW009775).

Authors’ affiliations

¹The Kirby Institute, University of New South Wales, Sydney, NSW, Australia. ²Africa Health Research Institute, KwaZulu-Natal, South Africa. ³School of Clinical Medicine, Discipline of Obstetrics and Gynaecology, University of KwaZulu-Natal, Durban, South Africa. ⁴School of Nursing and Public Health, University of KwaZulu-Natal, Durban, South Africa. ⁵Heidelberg Institute of Global Health (HIGH), Faculty of Medicine and University Hospital, University of Heidelberg, Heidelberg, Germany. ⁶King’s College Hospital NHS Foundation Trust, London, UK. ⁷Division of Infection and Immunity, University College London, London, UK. ⁸Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, MA, USA. ⁹Institute for Global Health, University College London, London, UK. ¹⁰Institute for Health & Wellbeing, University of Glasgow, Glasgow, UK.

Supplementary files

References

1. Leatherman S, Ferris TG, Berwick D, Omaswa F, Crisp N. The role of quality improvement in strengthening health systems in developing countries. Int J Qual Health Care 2010; 22(4):237-243. doi: 10.1093/intqhc/mzq028 [Crossref] [ Google Scholar]
2. Mensah Abrampah N, Syed SB, Hirschhorn LR. Quality improvement and emerging global health priorities. Int J Qual Health Care 2018; 30(suppl 1):5-9. doi: 10.1093/intqhc/mzy007 [Crossref] [ Google Scholar]
3. Kruk ME, Gage AD, Arsenault C. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health 2018; 6(11):e1196-e1252. doi: 10.1016/s2214-109x(18)30386-3 [Crossref] [ Google Scholar]
4. Ivers N, Jamtvedt G, Flottorp S. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012(6):CD000259. doi: 10.1002/14651858.CD000259.pub3 [Crossref]
5. Gude WT, Brown B, van der Veer SN. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci 2019; 14(1):39. doi: 10.1186/s13012-019-0887-1 [Crossref] [ Google Scholar]
6. Ayieko P, Ntoburi S, Wagai J. A multifaceted intervention to implement guidelines and improve admission paediatric care in Kenyan district hospitals: a cluster randomised trial. PLoS Med 2011; 8(4):e1001018. doi: 10.1371/journal.pmed.1001018 [Crossref] [ Google Scholar]
7. Blacklock C, Gonçalves Bradley DC, Mickan S. Impact of contextual factors on the effect of interventions to improve health worker performance in sub-Saharan Africa: review of randomised clinical trials. PLoS One 2016; 11(1):e0145206. doi: 10.1371/journal.pone.0145206 [Crossref] [ Google Scholar]
8. World Health Organization, Organization for Economic Co-operation and Development, The World Bank. Delivering Quality Health Services: A Global Imperative for Universal Health Coverage. Geneva: World Health Organization, Organization for Economic Co-operation and Development, The World Bank; 2018.
9. Young T, Brailsford S, Connell C, Davies R, Harper P, Klein JH. Using industrial processes to improve patient care. BMJ 2004; 328(7432):162-164. doi: 10.1136/bmj.328.7432.162 [Crossref] [ Google Scholar]
10. Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco: Jossey-Bass; 2009.
11. D’Andreamatteo A, Ianni L, Lega F, Sargiacomo M. Lean in healthcare: a comprehensive review. Health Policy 2015; 119(9):1197-1209. doi: 10.1016/j.healthpol.2015.02.002 [Crossref] [ Google Scholar]
12. Liberatore MJ. Six Sigma in healthcare delivery. Int J Health Care Qual Assur 2013; 26(7):601-626. doi: 10.1108/ijhcqa-09-2011-0054 [Crossref] [ Google Scholar]
13. Antony J, Palsuk P, Gupta S, Mishra D, Barach P. Six Sigma in healthcare: a systematic review of the literature. Int J Qual Reliab Manag 2018; 35(5):1075-1092. doi: 10.1108/ijqrm-02-2017-0027 [Crossref] [ Google Scholar]
14. de Koning H, Verver JP, van den Heuvel J, Bisgaard S, Does RJ. Lean six sigma in healthcare. J Healthc Qual 2006; 28(2):4-11. doi: 10.1111/j.1945-1474.2006.tb00596.x [Crossref] [ Google Scholar]
15. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989; 320(1):53-56. doi: 10.1056/nejm198901053200110 [Crossref] [ Google Scholar]
16. Kaluzny AD, McLaughlin CP, Kibbe DC. Continuous quality improvement in the clinical setting: enhancing adoption. Qual Manag Health Care 1992; 1(1):37-44. [ Google Scholar]
17. Institute for Healthcare Improvement (IHI). The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston: IHI; 2003.
18. Larson E, Mbaruku GM, Cohen J, Kruk ME. Did a quality improvement intervention improve quality of maternal health care? Implementation evaluation from a cluster-randomized controlled study. Int J Qual Health Care 2020; 32(1):54-63. doi: 10.1093/intqhc/mzz126 [Crossref] [ Google Scholar]
19. Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low-resource settings?. Lancet 2005; 366(9490):1026-1035. doi: 10.1016/s0140-6736(05)67028-6 [Crossref] [ Google Scholar]
20. Rubenstein L, Khodyakov D, Hempel S. How can we recognize continuous quality improvement?. Int J Qual Health Care 2014; 26(1):6-15. doi: 10.1093/intqhc/mzt085 [Crossref] [ Google Scholar]
21. Layton A, Moss F, Morgan G. Mapping out the patient’s journey: experiences of developing pathways of care. Qual Health Care 1998; 7 Suppl:S30-36. [ Google Scholar]
22. Bonetti PO, Waeckerlin A, Schuepfer G, Frutiger A. Improving time-sensitive processes in the intensive care unit: the example of ‘door-to-needle time’ in acute myocardial infarction. Int J Qual Health Care 2000; 12(4):311-317. doi: 10.1093/intqhc/12.4.311 [Crossref] [ Google Scholar]
23. Berwick DM. Developing and testing changes in delivery of care. Ann Intern Med 1998; 128(8):651-656. doi: 10.7326/0003-4819-128-8-199804150-00009 [Crossref] [ Google Scholar]
24. Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf 2011; 20(1):46-51. doi: 10.1136/bmjqs.2009.037895 [Crossref] [ Google Scholar]
25. Yotebieng M, Behets F, Kawende B, Ravelomanana NLR, Tabala M, Okitolonda EW. Continuous quality improvement interventions to improve long-term outcomes of antiretroviral therapy in women who initiated therapy during pregnancy or breastfeeding in the Democratic Republic of Congo: design of an open-label, parallel, group randomized trial. BMC Health Serv Res 2017; 17(1):306. doi: 10.1186/s12913-017-2253-9 [Crossref] [ Google Scholar]
26. Naidoo K, Gengiah S, Yende-Zuma N. Addressing challenges in scaling up TB and HIV treatment integration in rural primary healthcare clinics in South Africa (SUTHI): a cluster randomized controlled trial protocol. Implement Sci 2017; 12(1):129. doi: 10.1186/s13012-017-0661-1 [Crossref] [ Google Scholar]
27. Colbourn T, Nambiar B, Bondo A. Effects of quality improvement in health facilities and community mobilization through women’s groups on maternal, neonatal and perinatal mortality in three districts of Malawi: MaiKhanda, a cluster randomized controlled effectiveness trial. Int Health 2013; 5(3):180-195. doi: 10.1093/inthealth/iht011 [Crossref] [ Google Scholar]
28. Osibo B, Oronsaye F, Alo OD. Using small tests of change to improve PMTCT services in Northern Nigeria: experiences from implementation of a continuous quality improvement and breakthrough series program. J Acquir Immune Defic Syndr 2017; 75 Suppl 2:S165-S172. doi: 10.1097/qai.0000000000001369 [Crossref] [ Google Scholar]
29. Oyeledun B, Phillips A, Oronsaye F. The effect of a continuous quality improvement intervention on retention-in-care at 6 months postpartum in a PMTCT program in Northern Nigeria: results of a cluster randomized controlled study. J Acquir Immune Defic Syndr 2017; 75 Suppl 2:S156-S164. doi: 10.1097/qai.0000000000001363 [Crossref] [ Google Scholar]
30. Drake AL, Wagner A, Richardson B, John-Stewart G. Incident HIV during pregnancy and postpartum and risk of mother-to-child HIV transmission: a systematic review and meta-analysis. PLoS Med 2014; 11(2):e1001608. doi: 10.1371/journal.pmed.1001608 [Crossref] [ Google Scholar]
31. Dinh TH, Delaney KP, Goga A. Impact of maternal HIV seroconversion during pregnancy on early mother to child transmission of HIV (MTCT) measured at 4-8 weeks postpartum in South Africa 2011-2012: A National Population-Based Evaluation. PLoS One 2015; 10(5):e0125525. doi: 10.1371/journal.pone.0125525 [Crossref] [ Google Scholar]
32. Goga A, Chirinda W, Ngandu NK. Closing the gaps to eliminate mother-to-child transmission of HIV (MTCT) in South Africa: Understanding MTCT case rates, factors that hinder the monitoring and attainment of targets, and potential game changers. S Afr Med J 2018; 108(3a):S17-S24. doi: 10.7196/samj.2017.v108i3b.12817 [Crossref] [ Google Scholar]
33. Gupta RK, Gregson J, Parkin N. HIV-1 drug resistance before initiation or re-initiation of first-line antiretroviral therapy in low-income and middle-income countries: a systematic review and meta-regression analysis. Lancet Infect Dis 2018; 18(3):346-355. doi: 10.1016/s1473-3099(17)30702-8 [Crossref] [ Google Scholar]
34. Myer L, Essajee S, Broyles LN. Pregnant and breastfeeding women: a priority population for HIV viral load monitoring. PLoS Med 2017; 14(8):e1002375. doi: 10.1371/journal.pmed.1002375 [Crossref] [ Google Scholar]
35. World Health Organization (WHO). Consolidated Guidelines on the Use of Antiretroviral drugs for Treating and Preventing HIV Infection: Recommendations for a Public Health Approach. 2nd ed. WHO; 2016.
36. Read PJ, Mandalia S, Khan P. When should HAART be initiated in pregnancy to achieve an undetectable HIV viral load by delivery?. AIDS 2012; 26(9):1095-1103. doi: 10.1097/QAD.0b013e3283536a6c [Crossref] [ Google Scholar]
37. Cawley C, McRobie E, Oti S. Identifying gaps in HIV policy and practice along the HIV care continuum: evidence from a national policy review and health facility surveys in urban and rural Kenya. Health Policy Plan 2017; 32(9):1316-1326. doi: 10.1093/heapol/czx091 [Crossref] [ Google Scholar]
38. Ezeanolue EE, Powell BJ, Patel D. Identifying and prioritizing implementation barriers, gaps, and strategies through the Nigeria implementation science alliance: getting to zero in the prevention of mother-to-child transmission of HIV. J Acquir Immune Defic Syndr 2016; 72(Suppl 2):S161-166. doi: 10.1097/qai.0000000000001066 [Crossref] [ Google Scholar]
39. Hodgson I, Plummer ML, Konopka SN. A systematic review of individual and contextual factors affecting ART initiation, adherence, and retention for HIV-infected pregnant and postpartum women. PLoS One 2014; 9(11):e111421. doi: 10.1371/journal.pone.0111421 [Crossref] [ Google Scholar]
40. Nachega JB, Uthman OA, Anderson J. Adherence to antiretroviral therapy during and after pregnancy in low-income, middle-income, and high-income countries: a systematic review and meta-analysis. AIDS 2012; 26(16):2039-2052. doi: 10.1097/QAD.0b013e328359590f [Crossref] [ Google Scholar]
41. Taylor M, Newman L, Ishikawa N. Elimination of mother-to-child transmission of HIV and Syphilis (EMTCT): Process, progress, and program integration. PLoS Med 2017; 14(6):e1002329. doi: 10.1371/journal.pmed.1002329 [Crossref] [ Google Scholar]
42. Lenzer J. How Cuba eliminated mother-to-child transmission of HIV and syphilis. BMJ 2016; 352:i1619. doi: 10.1136/bmj.i1619 [Crossref] [ Google Scholar]
43. Thisyakorn U. Elimination of mother-to-child transmission of HIV: lessons learned from success in Thailand. Paediatr Int Child Health 2017; 37(2):99-108. doi: 10.1080/20469047.2017.1281873 [Crossref] [ Google Scholar]
44. Nyasulu JC, Muchiri E, Mazwi SL, Ratshefola M. NIMART rollout to primary healthcare facilities increases access to antiretrovirals in Johannesburg: an interrupted time series analysis. S Afr Med J 2012; 103(4):232-236. doi: 10.7196/samj.6380 [Crossref] [ Google Scholar]
45. National Department of Health South Africa. National Consolidated Guidelines: For the Prevention of Mother-to-Child Transmission of HIV (PMTCT) and the Management of HIV in Children, Adolescents and Adults. Pretoria: National Department of Health; 2015.
46. National Department of Health South Africa. Implementation of the Universal Test and Treat Strategy for HIV Positive Patients and Differentiated Care for Stable Patients. Pretoria: National Department of Health South Africa; 2016.
47. Massyn N, Peer N, Padarath A, Barron P, Day C. District Health Barometer 2014-15. Durban: Health Systems Trust; 2015.
48. Myer L, Phillips TK, McIntyre JA. HIV viraemia and mother-to-child transmission risk after antiretroviral therapy initiation in pregnancy in Cape Town, South Africa. HIV Med 2017; 18(2):80-88. doi: 10.1111/hiv.12397 [Crossref] [ Google Scholar]
49. Chetty T, Yapa HMN, Herbst C. The MONARCH intervention to enhance the quality of antenatal and postnatal primary health services in rural South Africa: protocol for a stepped-wedge cluster-randomised controlled trial. BMC Health Serv Res 2018; 18(1):625. doi: 10.1186/s12913-018-3404-3 [Crossref] [ Google Scholar]
50. Yapa HM, De Neve J-W, Chetty T. The impact of continuous quality improvement on coverage of antenatal HIV care tests in rural South Africa: results of a stepped-wedge cluster-randomised controlled implementation trial. PLoS Med 2020; 17(10):e1003150. doi: 10.1371/journal.pmed.1003150 [Crossref] [ Google Scholar]
51. Moore GF, Audrey S, Barker M. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350:h1258. doi: 10.1136/bmj.h1258 [Crossref] [ Google Scholar]
52. Cresswell JW, Cresswell JD. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 5th ed. Los Angeles: SAGE Publications; 2018.
53. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology 2009; 43(3):535-554. doi: 10.1177/0038038509103208 [Crossref] [ Google Scholar]
54. Flottorp SA, Oxman AD, Krause J. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci 2013; 8:35. doi: 10.1186/1748-5908-8-35 [Crossref] [ Google Scholar]
55. Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis of stepped wedge trials. Contemp Clin Trials 2015; 45(Pt A):55-60. doi: 10.1016/j.cct.2015.07.006 [Crossref] [ Google Scholar]
56. O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ 2010; 341:c4587. doi: 10.1136/bmj.c4587 [Crossref] [ Google Scholar]
57. Richter LM, Rooyen Hv, Solomon V, Griesel D, Durrheim K. Putting HIV/AIDS counselling in South Africa in its place. Society in Transition 2001; 32(1):148-154. doi: 10.1080/21528586.2001.10419039 [Crossref] [ Google Scholar]
58. Mwisongo A, Mehlomakhulu V, Mohlabane N, Peltzer K, Mthembu J, Van Rooyen H. Evaluation of the HIV lay counselling and testing profession in South Africa. BMC Health Serv Res 2015; 15:278. doi: 10.1186/s12913-015-0940-y [Crossref] [ Google Scholar]
59. Médecins Sans Frontières (MSF). Bending the curves of the HIV/TB epidemic in KwaZulu-Natal. Cape Town: MSF; 2016.
60. Hu J, Geldsetzer P, Steele SJ. The impact of lay counselors on HIV testing rates: quasi-experimental evidence from lay counselor redeployment in KwaZulu-Natal, South Africa. AIDS 2018; 32(14):2067-2073. doi: 10.1097/qad.0000000000001924 [Crossref] [ Google Scholar]
61. Hwang B, Shroufi A, Gils T. Stock-outs of antiretroviral and tuberculosis medicines in South Africa: a national cross-sectional survey. PLoS One 2019; 14(3):e0212405. doi: 10.1371/journal.pone.0212405 [Crossref] [ Google Scholar]
62. Médecins Sans Frontières (MSF). Stock Outs in South Africa: Second Annual Report. Johannesburg: MSF; 2015.
63. Gils T, Bossard C, Verdonck K. Stockouts of HIV commodities in public health facilities in Kinshasa: Barriers to end HIV. PLoS One 2018; 13(1):e0191294. doi: 10.1371/journal.pone.0191294 [Crossref] [ Google Scholar]
64. Tong C, Suzan-Monti M, Sagaon-Teyssier L. Treatment interruption in HIV-positive patients followed up in Cameroon’s antiretroviral treatment programme: individual and health care supply-related factors (ANRS-12288 EVOLCam survey). Trop Med Int Health 2018; 23(3):315-326. doi: 10.1111/tmi.13030 [Crossref] [ Google Scholar]
65. Bowser D, Krech L, Mabirizi D, Chang AY, Kapaon D, Bossert T. Associations between practices and behaviors at the health facility level and supply chain management for antiretrovirals: evidence from Cameroon, Namibia, and Swaziland. Glob Health Sci Pract 2019; 7(2):300-316. doi: 10.9745/ghsp-d-19-00063 [Crossref] [ Google Scholar]
66. Bravo MP, Peratikos MB, Muicha AS. Monitoring Pharmacy and Test Kit Stocks in Rural Mozambique: US President’s Emergency Plan for AIDS Relief Surveillance to Help Prevent Ministry of Health Shortages. AIDS Res Hum Retroviruses 2020; 36(5):415-426. doi: 10.1089/aid.2019.0057 [Crossref] [ Google Scholar]
67. Alemu YM, Ambaw F, Wilder-Smith A. Utilization of HIV testing services among pregnant mothers in low income primary care settings in northern Ethiopia: a cross sectional study. BMC Pregnancy Childbirth 2017; 17(1):199. doi: 10.1186/s12884-017-1389-2 [Crossref] [ Google Scholar]
68. Edwards LJ, Moisés A, Nzaramba M. Implementation of a health management mentoring program: year-1 evaluation of its impact on health system strengthening in Zambézia province, Mozambique. Int J Health Policy Manag 2015; 4(6):353-361. doi: 10.15171/ijhpm.2015.58 [Crossref] [ Google Scholar]
69. Bradley E, Hartwig KA, Rowe LA. Hospital quality improvement in Ethiopia: a partnership-mentoring model. Int J Qual Health Care 2008; 20(6):392-399. doi: 10.1093/intqhc/mzn042 [Crossref] [ Google Scholar]
70. Manzi A, Hirschhorn LR, Sherr K, Chirwa C, Baynes C, Awoonor-Williams JK. Mentorship and coaching to support strengthening healthcare systems: lessons learned across the five Population Health Implementation and Training partnership projects in sub-Saharan Africa. BMC Health Serv Res 2017; 17(Suppl 3):831. doi: 10.1186/s12913-017-2656-7 [Crossref] [ Google Scholar]
71. Waiswa P, Manzi F, Mbaruku G. Effects of the EQUIP quasi-experimental study testing a collaborative quality improvement approach for maternal and newborn health care in Tanzania and Uganda. Implement Sci 2017; 12(1):89. doi: 10.1186/s13012-017-0604-x [Crossref] [ Google Scholar]
72. Bhardwaj S, Barron P, Pillay Y. Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up using quality improvement. S Afr Med J 2014; 104(3 Suppl 1):239-243. doi: 10.7196/samj.7605 [Crossref] [ Google Scholar]
73. Mate KS, Ngidi WH, Reddy J, Mphatswe W, Rollins N, Barker P. A case report of evaluating a large-scale health systems improvement project in an uncontrolled setting: a quality improvement initiative in KwaZulu-Natal, South Africa. BMJ Qual Saf 2013; 22(11):891-898. doi: 10.1136/bmjqs-2012-001244 [Crossref] [ Google Scholar]
74. Mwaniki MK, Vaid S, Chome IM, Amolo D, Tawfik Y. Improving service uptake and quality of care of integrated maternal health services: the Kenya Kwale district improvement collaborative. BMC Health Serv Res 2014; 14:416. doi: 10.1186/1472-6963-14-416 [Crossref] [ Google Scholar]
75. Youngleson MS, Nkurunziza P, Jennings K, Arendse J, Mate KS, Barker P. Improving a mother to child HIV transmission programme through health system redesign: quality improvement, protocol adjustment and resource addition. PLoS One 2010; 5(11):e13891. doi: 10.1371/journal.pone.0013891 [Crossref] [ Google Scholar]
76. Singh K, Speizer I, Handa S. Impact evaluation of a quality improvement intervention on maternal and child health outcomes in Northern Ghana: early assessment of a national scale-up project. Int J Qual Health Care 2013; 25(5):477-487. doi: 10.1093/intqhc/mzt054 [Crossref] [ Google Scholar]
77. Ciampa PJ, Tique JA, Jumá N. Addressing poor retention of infants exposed to HIV: a quality improvement study in rural Mozambique. J Acquir Immune Defic Syndr 2012; 60(2):e46-52. doi: 10.1097/QAI.0b013e31824c0267 [Crossref] [ Google Scholar]
78. Gray A, Vawda Y. South African Health Review. Durban: Health Systems Trust; 2017.
79. Myer L, Dunning L, Lesosky M. Frequency of viremic episodes in HIV-infected women initiating antiretroviral therapy during pregnancy: a cohort study. Clin Infect Dis 2017; 64(4):422-427. doi: 10.1093/cid/ciw792 [Crossref] [ Google Scholar]
80. Mandelbrot L, Tubiana R, Le Chenadec J. No perinatal HIV-1 transmission from women with effective antiretroviral therapy starting before conception. Clin Infect Dis 2015; 61(11):1715-1725. doi: 10.1093/cid/civ578 [Crossref] [ Google Scholar]
81. Vrazo AC, Sullivan D, Ryan Phelps B. Eliminating mother-to-child transmission of HIV by 2030: 5 strategies to ensure continued progress. Glob Health Sci Pract 2018; 6(2):249-256. doi: 10.9745/ghsp-d-17-00097 [Crossref] [ Google Scholar]
82. Barnabas RV, Revill P, Tan N, Phillips A. Cost-effectiveness of routine viral load monitoring in low- and middle-income countries: a systematic review. J Int AIDS Soc 2017; 20(Suppl 7). doi: 10.1002/jia2.25006 [Crossref]
83. Nambiar B, Hargreaves DS, Morroni C. Improving health-care quality in resource-poor settings. Bull World Health Organ 2017; 95(1):76-78. doi: 10.2471/blt.16.170803 [Crossref] [ Google Scholar]
84. World Health Organization. Systems Thinking for Health Systems Strengthening. Geneva: Alliance for Health Policy and Systems Research, 2009.