Local Stakeholders’ Perceptions About the Introduction of Performance-Based Financing in Benin: A Case Study in Two Health Districts

Background: Performance-Based Financing (PBF) has been advanced as a solution to contribute to improving the performance of health systems in developing countries. This is the case in Benin. This study aims to analyse how two PBF approaches, piloted in Benin, behave during implementation and what effects they produce, through investigating how local stakeholders perceive the introduction of PBF, how they adapt the different approaches during implementation, and the behavioural interactions induced by PBF. Methods: The research rests on a socio-anthropological approach and qualitative methods. The design is a case study in two health districts selected on purpose. The selection of health facilities was also done on purpose, until we reached saturation of information. Information was collected through observation and semi-directive interviews supported by an interview guide. Data was analysed through contents and discourse analysis. Results: The Ministry of Health (MoH) strongly supports PBF, but it is not well integrated with other ongoing reforms and processes. Field actors welcome PBF but still do not have a sense of ownership about it. The two PBF approaches differ notably as for the organs in charge of verification. Performance premiums are granted according to a limited number of quantitative indicators plus an extensive qualitative checklist. PBF matrices and verification missions come in addition to routine monitoring. Local stakeholders accommodate theoretical approaches. Globally, staff is satisfied with PBF and welcomes additional supervision and training. Health providers reckon that PBF forces them to depart from routine, to be more professional and to respect national norms. A major issue is the perceived unfairness in premium distribution. Even if health staff often refer to financial premiums, actually the latter are probably too weak — and ‘blurred’ — to have a lasting inciting effect. It rather seems that PBF motivates health workers through other elements of its ‘package’, especially formative supervisions. Conclusion: If the global picture is quite positive, several issues could jeopardise the success of PBF. It appears crucial to reduce the perceived unfairness in the system, notably through enhancing all facilities’ capacities to ensure they are in line with national norms, as well as to ensure financial and institutional sustainability of the system.


Introduction
Many public health systems in developing countries, especially in Africa, are confronted to structural and functioning problems leading to poor performance. Face to the failure of past reform strategies, Performance-Based Financing (PBF) -which can basically be defined as 'a mechanism by which health providers are, at least partially, funded on the basis on their performance' (1) -has been advanced as a new health reform approach to contribute to improving the performance of those systems (1,2) 1 . This is notably the case in Benin where several donors have piloted PBF programmes in the health sector, especially the Belgian Development Agency (BTC) and the World Bank (WB). In the context of a joint health system strengthening platform, the 34 health districts of the country should soon be covered by a partner supporting PBF. Dialogue has been initiated to harmonise the approaches pursued by the different promoters so as to scale PBF up in a unified way. The two main PBF approaches experimented in Benin and studied in this paper diverge at several respects. They are both based on the separation of functions between four types of stakeholders -health service providers; those who supervise them and verify declared results; those who buy results; plus a regulatory body, namely the Ministry of Health (MoH)and the principle of contracting their relationships. They both focus on the quality of healthcare. However, differences are noted at the levels of the policy formulation and design; institutions in place; indicators; and costs at stakes (6). Following encouraging experiences in various countries, PBF in the health sector has now been adopted as pilots or national policy by many African countries and communities of practice have developed so as to share experiences and professionalise the business (4,5,(7)(8)(9). However, the jury is still out as for its impact (10)(11)(12), and numerous theoretical and empirical studies mention possible contradictory effects linked to the introduction of PBF in a complex system (13)(14)(15)(16). There is a growing amount of papers written on PBF, but a Cochrane systematic review of PBF in low and middle income countries points to the fact that more studies are necessary (11). Except for one case study in Uganda, most evaluations of PBF to date have been 'black box' studies, investigating the magnitude of the effects from PBF, in contrast to explaining how and why effects come about ('open-box') (15). Yet, understanding why and how PBF produces effects is crucial because PBF is actually an approach or 'package' with various 1. There is a plethora of terms commonly used in this field. Witter et al. (3) consider that " [t]he core features of PBF, as currently practiced in reform packages […], are summarized as follows: -They take a supply-side approach, meaning performance-based incentives are earned by service providers; -Payments are targeted at individual health facilities and administrations, often with trickle-down to health workers; -There is most often some split of functions between regulation, purchasing, fund-holding, verification and service delivery, although the practices vary by context; -Payments are linked to outputs, modified by quality indicators.
[…] The core concept is to promote a results-orientation by linking incentives to desired outputs and encouraging entrepreneurial behavior by staff and managers. This is done by establishing a set of more explicit contractual arrangements between different players. […] According to the definition adopted by the African PBF Community of Practice, in August 2010, PBF 'applies market forces but seeks to correct market failures to attain health gains'". See also (4) and (5) for more explanations about the PBF approach. elements which differ from country to country. In Benin, the WB project was designed as a randomised control trial aimed at testing the effect of various designs, and will be soon subject to an impact evaluation. However, existing PBF pilots in the country did not include in the first place a 'soft' analysis on its implementation and adaptive process, nor its perceptions and effects over those in charge of implementing them, so as to 'open the black box' and understand how -and why -it functions -including what elements of the 'PBF package' are the most effective in that context. The independent research presented in this paper intends to add new evidence by investigating local stakeholders' perceptions about PBF. It aims to analyse how the two different PBF approaches behave during implementation and what effects they produce over stakeholders in charge of implementing them. It investigates how local stakeholders whom implement PBF on the field (mostly health service providers) perceive the introduction of PBF, how they adapt the approaches during implementation (operational adjustments), and the behavioural interactions induced by the introduction of PBF in two health districts in South Benin -one supported by BTC and one by the WB. Especially, the study analyses how PBF is currently operating PBF from an actor's point of view, thus looking for possible "policy-implementation gaps", together with behavioural effects induced by PBF (search for a number of behavioural effects predicted by the literature, as well as possible other unexpected results).

Methods
It has to be noted in the first place that what we call 'PBF approach' in this article may not exactly correspond to what the WB, BTC or the literature defines. Rather, we refer to what stakeholders call and accept as such. Indeed, as for any policy, there might be some gap between policy and practice. A first gap can be found between the PBF 'pure approaches' such as found in the literature (4,5) and the approaches developed in Benin. A second gap, which is more important from the perspective of sociology and grounded studies, relates to the divergences between what is supposed to be done according to the country frameworks, and what is observed on the field (which could be called 'temporary results'). Being conscious of that, one cannot appreciate a policy or theory by comparing it to norms or other policies. A more appropriate position is to assess the functioning and effectiveness of a policy through analysing its implementation practices, its evolutions and the results it produces in a given context, based on how it is applied and how stakeholders perceive it. This is what we intended to do in this study. We do not investigate the first gap presented above, but we focus on the second one. We make no judgement about the WB and BTC approaches, but take them for granted and try to appreciate how stakeholders make it happen and implement that policy, what it becomes on the ground and how it behaves in stakeholders' perception. While to date, most evaluations of PBF in developing countries aimed either to describe and compare designs (17) or to assess its impact over health results indicators (18,19), a group of academics and PBF practitioners recently developed a more comprehensive framework for monitoring and evaluating PBF, aimed at analysing its interactions with health systems. It is structured around five domains of: (i) context and its influences on PBF; (ii) development process; (iii) design; (iv) implementation; and (v) effects on health systems (3). The research presented in this paper inspires from this framework and mainly focusses on the fourth domain, namely how stakeholders implement PBF and what direct effects it has on their behaviours. Yet, the other domains are also investigated so as to better understand the processes at stakes. To be able to understand what is happening 'inside the black box' of PBF, this research also inspires from the actor-centred approach developed by Renmans et al. (20). Besides, the research intends to isolate, whenever possible, the influence of the various elements of the PBF 'package' . Practitioners have identified a number of best practices and criteria to establish how far a project is 'PBF' (4,5). As some of these were still not effective when we performed our study and/or were too difficult to grab through interviews with local stakeholders, we mostly turned our attention to the following elements: performance premiums, overall resource increase, using indicators to clarify priorities, strengthened 'vertical' control, strengthened community control, increased management freedom, and capacity building initiatives provided in the context of PBF. The research protocol (available upon request -in French) was conceived by a multidisciplinary group of academics and researchers, who also consulted practitioners. It was not approved officially by an institutional review board, but agreed with MoH officials. All interviewed persons were informed of the research and gave their consent. The research protocol rests mainly on a socio-anthropological approach and qualitative methods. It follows an inductive approach, based on the description and analysis of what we observed in the field, without intending to compare this to the norms written on paper in the programmes under scrutiny. The design rests on a case study in two health districts selected on purpose as one has been subject to the first pilot introduction of PBF by BTC, namely the district of Comè, and the other (Lokossa-Athièmè) is supported by the WB in the same administrative department of the country -so as to ensure similar contexts. Within those two districts, the selection of health facilities was also done on purpose, so as to reflect varying viewpoints about the two PBF pilot initiatives. In each district, the sample included the District Health Management Team (DHMT), the technical assistant managing each donor's programme, the district hospital and 5 to 6 health centres, until we reached saturation of information. Especially in the WB district, three health facilities included in the sample were under treatment, while three belonged to control groups (with/ without increased management autonomy). This enabled to widen the range of viewpoints, triangulate information and grab health staff expectations about PBF -even those who did not yet fully participate in the programme, but were exposed to some elements of the approach. The methodology for collecting information rested on observation and semidirective interviews supported by an interview guide. It was triangulated by document analysis enabling to collect quantitative data on processes and results. Beforehand, a rapid analysis of the relevance of PBF approaches in their context, as well as the formulation process (domains 1 to 3 of Witter et al. 's framework) (3) was led on the basis of document analysis and a number of interviews at central level. In total, 27 persons were interviewed in the district of Comè (2 belonging to the managerial hierarchy, 20 health service providers, 3 administrative staff and 2 technical assistants), 19 were interviewed in the district of Lokossa-Athièmè (3 belonging to the managerial hierarchy, 12 health service providers, 3 administrative staff and 1 technical assistant) and 7 persons were interviewed at central level (donors and MoH representatives). Data was collected in early October (Comè), November (Comè and central level) and December 2013 (Lokossa-Athièmè). Witter et al. (3) highlight key questions for monitoring and evaluation and propose a systematic approach to monitoring the effects of PBF, but this was not sufficient to precisely guide our research. Thus we completed our analytical framework by identifying a number of behavioural effects that are predicted by the literature on PBF and more generally incentive theory (13,14,20,21). The literature indeed points to a number of positive effects of PBF over health system performance, notably with respect to increased efforts to meet the targets, but also in terms of increased management freedom and participation to evidence-based decision-making; but also, to possible perverse effects stemming from distorted efforts towards remunerated tasks. These general predicted effects were introduced in the interview guide. The latter was used as a checklist for data collection and analysis. It was structured against five main themes: stakeholders' perceptions about PBF relevance, process and design; detection of potential policy-implementation gaps; positive/expected effects of PBF over health staff 's behaviour; perverse/unexpected effects of PBF over health staff 's behaviour; and effects over the health system, plus some information on costs aimed at cross-checking documentary information. Data was collected against this checklist and in a second step, qualitative data was analysed as usually in social science, through a contents and discourse analysis. The results section below follows the structure of the conceptual framework developed by Witter et al. (3), focusing (except for the context) on stakeholders' perceptions over those domains.

Results
The context PBF has been initiated in Benin through pilot projects led by several donors. In particular, BTC initiated the payment of performance premiums in 2008 in the framework of a project supporting the district of Comè, and the WB started to conceive a PBF programme in 2008 as well. In addition, other donors also introduced pilot experiments notably at community level, notably UNICEF and NGOs. PBF started to be effective in the field in 2012. The MoH strongly supports PBF, this is why it was decided to scale it up with support from other donors (this should be effective in the beginning of 2015).  (6), plus information gathered from our field investigation Table 1 below describes some characteristics of the two PBF designs analysed in this study, namely the BTC and the WB approaches. Even if comparing the country approaches with norms is not the purpose of this study, let us mention that the two approaches diverge from PBF good practices (4,5) at several respects, notably due to the relatively low amounts involved (compared to the standard of having a PBF programme budget of no less than 3 US dollars per capita per year, of which at least 70% is used for subsidies, community contracts and infrastructure input units) (4). Moreover, there is no real competition between providers, the private sector is not involved, and autonomy of health facility managers is limited and constrained by lack of capacity.
Perceptions about the formulation of the policy Before launching its current PBF programme in 2012, BTC had already tested the introduction of performance premiums in the district of Comè in a previous project. The system was not yet labelled 'PBF' even if it resembled it. It was developed through a participative, bottom-up approach, thus BTC built on that experience to design its approach, in an action-research dynamic. As for the WB, it mainly built on the successful and well-documented Rwandan experience (18,19). In the two visited districts, field actors regret not having been consulted during PBF policy formulation. Field and external actors perceive neither the foundations, nor the perspectives of the PBF policy. They welcome PBF but still do not have a sense of ownership about it, notably because it does not totally meet their expectations and constraints. Except for the previous BTC project, there has not been any public debate involving local/operational stakeholders about the diagnosis leading to the relevance and choice of the PBF approach 2 . Moreover, the coexistence of two approaches in neighbour districts disturbs health staff, especially in the district supported by BTC where healthcare providers feel injured because premiums are lower than in WB treatment districts. Some unfairness is also perceived by staff in the WB district's control groups versus treatment groups.

Perceptions about the design of Performance-Based Financing approaches
The BTC approach rests on pre-existing bodies, notably for verification which is based on peer-review, plus community counter-verification. It does not provide very high financial premiums, except for the DHMT. On the contrary, the WB approach better suits international PBF standards and encompasses the creation of new bodies for contract development, verification and coaching, which is delegated to an international cabinet, plus a local NGO for community counter-verification. The financial premiums are sensibly higher -except for the DHMT which had not yet received premiums at the time of our survey. In the district supported by BTC, health service providers suspect their hierarchy to monopolise the benefits of PBF, and thus would prefer having an independent body being in charge of managing it. However, other voices at central level fear that such an independent body entails high cost and may not be financially and institutionally sustainable after donors leaveat least as long as no institutional change is made to increase management freedom at operational level. Under the two approaches under scrutiny in this study, performance premiums are granted according to the level of a limited number of quantitative indicators (see Table 1 and has already enabled to build management and practice capacities. However, under the two approaches, interviewed actors have complained about the duplication of tools, as PBF matrices and supervisions come in addition to routine monitoring, for instance. Nevertheless, health staff feels they have insufficient knowledge about how PBF is designed and especially how premiums are distributed among them. More fundamentally, it emerges from our interviews that health staff perceives PBF as 'one more project' that is dissociated from other ongoing processes and reforms. In particular, PBF was not conceived in coherence with decentralisation (there is no role for decentralised authorities in the two designs under study), partial fee exemptions, and it is still not very clear how PBF scaling up will be articulated with the universal health coverage agenda and institutions. PBF brings additional premiums that pile upon many others and, as far as the WB project is concerned, is to date insufficiently integrated into sustainable institutions. PBF is not yet owned and viewed by health workers as a way of improving their performance; instead, priorities promoted through PBF are viewed as donors' ones.

Perceptions about the implementation process of Performance-Based Financing
The two approaches have been implemented progressively; they have evolved during implementation so as to adapt to needs; and at the time of our survey, the WB approach was still only partially implemented, with some elements (community counter-verification and DHMT assessment) not being in place yet. For instance, under the WB approach, the DHMT was initially not granted any supplementary budget to do the PBF verification missions (which come in addition to usual monitoring and supervision), and therefore had to find other resources to do it, namely the funds collected from user fees. In some facilities supported by BTC, staff disagrees with the design of some tools (records…) required by PBF, and thus makes adjustments to them. A certain lack of transparency is also regretted at this level. However, globally, interviewed staff is satisfied with PBF and welcomes additional supervision and training being provided. A major issue under both approaches is the perceived unfairness of PBF towards operational staff. First, healthcare providers under the BTC approach feel they receive much lower premiums than those enrolled in the WB projecteven if this perception is somewhat biased because not all facilities are under treatment and receive high premiums in WB districts. More importantly, the commonly perceived unfairness of the system stems from the fact that (contrary to standard PBF best practices) the same indicators and the same quality checklist are applied to all health facilities, whatever their initial endowment and situation. Therefore, disadvantaged facilities which do not respect national norms because they lack material, equipment or staff, are automatically penalised, however strong efforts they make. Besides, so far the two PBF approaches neither include activities aimed at putting all health facilities in line with national norms, nor comprise special incentives to encourage staff to work in disadvantaged areas. The BTC project initially had a budget for scaling-up health centres' equipment, but this was converted into the budget necessary to finance PBFso that some health centres keep much below national norms without having sufficient means to upgrade their equipment. Finally, unfairness is also perceived at the level of premium sharing -this is especially the case in the BTC district, where staff suspects its hierarchy to monopolise all benefits stemming from PBF, as confirmed by the level of premiums reported by the people we interviewed.

Perceptions about the effects of Performance-Based Financing
After only three quarters of implementation, in the two surveyed districts, PBF had already produced a number of positive effects on actors' behaviour. If both donors acknowledge good progress in indicators, interviewed healthcare providers reckon that PBF forces them to depart from routine, to be more professional and to respect national norms, which was often not the case before. They have been sensitised to better welcome patients, to reduce absenteeism, to better communicate with them and with each other, to improve quality of registers and other data, to use it for management purpose and finally, to improve quality of care. As a result, the workload and time worked have increased, which is nevertheless insufficiently compensated by performance premiums. Moreover, while PBF encourages collaboration and teamwork within health structures under both models, so far it does not seem to foster collaboration and teamwork between levels of the health system under the WB model, probably because it relies mostly on external actors. Not many negative effects have been detected so far, but we have gathered indications that the incentive to conform to norms and fulfil registers might be 'seasonal' , that is, more efforts are being made in the few days before verification missions, so that one can question the sustainability of results. We also observed a few inclinations to shirking, frustrations and latent conflict between staff categories; and feel that premiums could quickly be taken for granted. Even after three verification rounds, the staff we interviewed started to feel blase about PBF and thus reduce effort compared to the second round. An interesting result from our study, which contributes to understanding what takes place inside the PBF 'black box' , stems from the fact that, even if health staff often refer to financial premiums in their discourse, actually the latter are probably too weak-and 'blurred' into so many others-to have a real, lasting inciting effect. This is especially the case in the BTC approach since, as shown in Table 1, while the DHMT monopolised substantial individual premiums at the time of our visit, staff in district hospitals could only expect to get between 32 and 80 US dollars per quarter, and health centre staff only between 10 and 30 US dollars. It rather seems that PBF motivates health workers through other elements of its 'package': especially, regular formative supervisions have already enabled to strengthen management and clinical capacities, thus play an important role in improving performance. Moreover, PBF fosters emulation amongst health facilities as well as improvement in data collection and use for management purpose. It is early to demonstrate effects of PBF on the health system but at the time we performed our study, one could already notice positive effects on the health information system. The share of performance premiums not being distributed to health staff was not yet used, but should be used to make investments enabling to enhance facilities' capacities.

Discussion
Our case studies have methodological limitations that prevent from generalising results, as we only visited one out of five districts supported by BTC, and one out of eight districts supported by the WB. Furthermore, they were limited to studying the perceptions of local stakeholders, namely healthcare providers and a few administrative staff, technical assistants and managers. This might seem restrictive since PBF could be studied from many angles. Our study therefore deserves to be completed by other enquiries (notably among communities, facilities' management committees, etc.) as well as by an analysis of epidemiological data under a sufficiently long period, so as to analyse PBF impacts. Yet, our study is a rigorous qualitative one, and the method used to collect data until saturation of information guarantees its robustness. The qualitative approach we used, namely interviewing health staff about their perceptions about PBF, enables to understand what sense local actors make from this approach which-even if supported by the central level-they perceive as externallydriven (22). Our finding that local stakeholders in charge of implementation adjusted theoretical approaches because they do not totally fit their needs and constraints is totally in line with what is often pointed out by socio-anthropological development studies (23). Stakeholders' perceptions pointed to the risk that staff become blase with PBF and reduce their effort over time. This should be monitored closely during the pursuit of implementation, and a challenge will be to sustain momentum and ensure that premiums are not taken for granted by health staff, as it has been observed in the past for other premiums (24,25). A more troubling and global finding emerging from our study is that thus far, PBF is not sufficiently anchored within a coherent and comprehensive reform approach. Stakeholders perceive it as some additional programme, and the two cases we studied rather constitute hybrid approaches containing partial elements of PBF juxtaposing on old rules and institutions. A comparison between how the two PBF approaches under consideration in this paper are currently implemented and international best practices (4,5) points to some design flaws that prevent PBF to be fully operational. Especially, one can point the relatively low effective management freedom of health facilities (even those under treatment in the WB approach since they still rely on centralised resource allocation rules for most of their budget as well as for personnel management), the disconnection from other reform processes such as decentralisation, the duplication with usual activities (notably monitoring) and the relatively low budget of PBF (amounting to about 1 US dollar per capita per year in the BTC approach and 2.17 US dollars per capita per year in the WB approach, which is judged insufficient compared to the international standard of about 3 US dollars (4,5). A very good point for PBF in Benin however, is that the approaches implemented are centred on patients and improving quality of care. They have been initiated in an un-harmonised way, but actually have much in commonespecially the approach of limiting to a small number of quantitative indicators and a checklist of quality indicators linked to the respect of national norms-so that they could easily converge when being scaled up. The one and most important diverging element is about who supervises health facilities and verifies the validity of reported data: existing bodies (BTC approach) vs. an independent body (WB approach). The latter's independence is likely to be more credible, but on the other hand it entails important costs (especially since currently the Contract Development and Verification (CDV) agency is an international one) and may not be financially and institutionally sustainable when donors pull out, so long as no comprehensive institutional change is made along the health system. More generally, the total 'opportunity' costs of the two PBF approaches implemented in Benin are not known. The financial costs exposed above do not take account of nonfinancial, 'hidden' transaction costs such as those incurred at all levels of the health system-and especially at the level of service delivery-by additional reporting requirements. This should be carefully studied so as to minimise 'unproductive' cost and favour financial sustainability in the long run. Finally, other authors warn against the fact that PBF does not by itself reduce inequities in service provision among the population, unless equity targets are specified (26). In Benin, it is too early to study the effects of PBF on equity towards the population, but we have pointed above that existing designs are perceived as unfair towards health service providers. Moreover, experience shows that health service quality cannot be improved by acting on health providers' behaviour only, but is also determined by other health system conditions -notably inputs and equipment (27). This argues in favour of investing in the first place so as to allow all health structures to respect national equipment norms before implementing PBF.

Conclusion
As a conclusion, in line with its objectives, our case study in two districts enabled to better understand how PBF is perceived by actors in charge of implementing it, how it is operationalised and what behaviour changes it brings. Stakeholders' perceptions show a quite positive global picture of PBF. However, they also point to a number of issues which could jeopardy the success of PBF and thus should be monitored and possibly responded by policy-makers. Especially, stakeholders' perceptions pointed the issue of unfairness for health staff inherent in the two PBF approaches under study, which is illustrated by the fact that operational staff receive very low premiums compared to DHMT staff in the BTC approach, and that all facilities are subject to the same requirements while they do not have the same resources and assets. It appears crucial to try and reduce the perceived unfair distribution of PBF benefits among different programmes and staff, notably through enhancing all facilities' capacities so as them to be in line with national norms; distributing premiums in favour of those who create results on the field; and possibly adapting indicators and checklists and/or providing extra incentives to encourage staff to work in disadvantaged areas. Finally, in view of scaling PBF up, our study also showed that it would be important to better communicate with field actors so as to guarantee its social acceptability and ownership, and to pay attention to the policy coherence as well as financial and institutional sustainability of the system over the long run. In-depth socio-anthropological studies could be very useful at those respects, to monitor and adapt the approach during implementation so as to align it with evolving health policies.

Ethical issues
The research protocol (available upon request -in French) was conceived by a multidisciplinary group of academics and researchers, who also consulted practitioners. It was not approved officially by an institutional review board, but agreed with MoH officials. All interviewed persons were informed of the research and gave their consent.