Assessing Performance of Botswana's Public Hospital System: The Use of the World Health Organization Health System Performance Assessment Framework

Background: Very few studies have assessed performance of Botswana public hospitals. We draw from a large research study assessing performance of the Botswana Ministry of Health (MoH) to evaluate the performance of public hospital system using the World Health Organization Health Systems Performance Assessment Framework (WHO HSPAF). We aimed to evaluate performance of Botswana public hospital system; relate findings of the assessment to the potential for improvements in hospital performance; and determine the usefulness of the WHO HSPAF in assessing performance of hospital systems in a developing country. Methods: This article is based on data collected from document analysis, 54 key informants comprising senior managers and staff of the MoH (N=40) and senior officers from stakeholder organizations (N=14), and surveys of 42 hospital managers and 389 health workers. Data from documents and transcripts were analyzed using content and thematic analysis while data analysis for surveys was descriptive determining proportions and percentages. Results: The organizational structure of the Botswana’s public hospital system, authority and decision-making are highly centralized. Overall physical access to health services is high. However, challenges in the distribution of facilities and inpatient beds create inequities and inefficiencies. Capacity of the hospitals to deliver services is limited by inadequate resources. There are significant challenges with the quality of care. Conclusion: While Botswana invested considerably in building hospitals around the country resulting in high physical access to services, the organization and governance of the hospital system, and inadequate resources limit service delivery. The ongoing efforts to decentralize management of hospitals to district level entities should be expedited. The WHO HSPAF enabled us to conduct a comprehensive assessment of the public hospital system. Though relatively new, this approach proved useful in this study.


Introduction
Public hospitals are a central component of the Botswana health system, accounting for over 60% of hospital beds and national health sector expenditure (1,2). These hospitals play a major role in delivering preventive, curative, diagnostic, and rehabilitative services. They also act as referral centres from primary healthcare facilities. The hospitals, therefore, profoundly influence performance of the entire health system. In view of their centrality to health system performance, worldwide there is increasing pressure on hospitals to improve their performance (3). In Botswana, communities are very critical of hospitals evidenced by the number of negative media reports (4,5) and increasing litigation cases. Internationally, health policy-makers and managers are trying alternative interventions aimed at improving performance of public hospitals (6). However, there is lack of systematic analysis of how hospitals function and perform, and the challenges they face, that could be used to support policy decisions and strategies for improvement (3). Generally, there is lack of consensus on the best methodologies for assessing hospital performance. Consequently, researchers use a variety of approaches. In developing countries, particularly in Africa, evaluation of hospital performance has focused on efficiency using economic models, most commonly Data Envelopment Analysis (DEA) (7)(8)(9). In these studies, efficiency is commonly defined as the ratio of inputs to outputs. The focus, therefore, is on resources used by hospitals to achieve a certain level of outputs. While several advantages of using DEA to assess performance of hospitals have been cited in these studies, including its usefulness as a benchmarking tool in identifying efficient and inefficient hospitals, its utility in terms of informing policy decisions is limited. This is due to the inability of this methodology to identify the underlying causes of inefficiencies. In Botswana, there are few studies assessing efficiency of public hospitals (10,11). Using DEA, Tlotlego et al. (11) concluded that over 70% of the hospitals were run inefficiently between 2006 and 2008. An earlier study by Ramanathan et al. (10) identified inefficient health districts and hospitals. Although these studies were helpful, guidance on the causes of inefficiencies was limited. The need for a multidimensional and comprehensive assessment that could provide a broad range of policy levers to improve hospital performance remains. This article addresses this need. Drawing from a large research study assessing performance of the Botswana Ministry of Health (MoH) (12), this article analyses the functioning and performance of public hospitals in Botswana using the World Health Organization Health Systems Performance Assessment Framework (WHO HSPAF) with a focus on health service delivery function (12)(13)(14). Overall, the aims of the paper are to: 1) evaluate the performance of Botswana's public hospital system; 2) relate findings of the assessment to the potential for improvements in hospital performance in Botswana; and 3) determine the usefulness of the WHO HSPAF in assessing performance of hospital systems in a developing country.

Conceptual approach to hospital performance assessment: WHO HSPAF
The WHO HSPAF framework originates from the WHO report 2000 which identified four functions of health systems: stewardship; financing; resource generation; and health service delivery (Figure 1). These functions were later broken down to what is commonly referred to as the six building blocks of the health system (14). Although the WHO HSPAF has been used to evaluate national health systems, the use of this framework to evaluate subcomponents of the system including organizations such as hospitals is advocated (15). The health service delivery function, the aspect of the framework used in this article, entails the actual delivery of health services (16). The focus is on how the health service delivery system is organized and managed, its ability to make services available, its capacity to deliver services in terms of the resource inputs, and the quality of services provided (16)(17)(18). Based on this, we identified four assessment domains: organization and governance, service availability, service capacity and service quality. Under each domain indicators were developed or adapted from the work of organizations working in the area of health systems performance assessment particularly the WHO and United States Agency for International Development (USAID) ( Table 1). These indicators also drew from management science assessment of public hospitals as organizations. However, it is worth noting that, from organizational literature, developing indicators for assessing performance of organizations is a challenging task. This is due to lack of convergence on appropriate criteria for organizational assessment (19,20). Consequently, there is consensus in acknowledging the value driven nature of such assessments. Steers (20) noted that selection of appropriate criteria is influenced to some extent by the evaluator's own frame of reference while Cameron (19) argued that criteria are based on the values and preferences individuals hold for the organization. In addition, while assessment is generally associated with use of numbers where variables are quantitative and considered objective and easy to manipulate, most of the variables in measuring performance of organizations are qualitative (21). Hence, are viewed as subjective and difficult to quantify. The use of qualitative indicators in assessing healthcare organizations and systems has received significant focus over the years because of their ability to provide feedback that could guide decision-making (22,23). The emphasis in this study was on indicators that are relevant and actionable to provide policymakers the opportunity to influence the performance of Botswana's public hospital system. Validity and reliability of the indicators were enhanced through triangulation of methodologies and data collection methods.

Assessment Domains
Organization and governance This domain addresses the organizational arrangements, leadership and management in public hospitals. Organizational arrangements have been found to have a profound impact on service delivery even when the resource inputs are available (6,16). The focus is on how the public hospital system is organized in relation to the (MoH), leadership structures within the hospitals, and locus of authority and decision-making.

Service availability
This area assesses physical access to services, giving an indication of how resources are distributed across the population (17,18). Specific indicators include the proportion of the population within a specified distance of a health facility, the number and distribution of inpatient beds per 10,000 population reflecting the ease of physical access, and access to inpatient care respectively.

Service capacity
This addresses the ability of the facilities to deliver health services (17,18). Capacity is determined by availability of resources, including hospital buildings, medical equipment, medicines and medical supplies, information technology and skilled and motivated personnel.

Service Quality
For health interventions to promote health and save lives, they must be of acceptable quality. The indicators in this area include availability of structures and strategies to improve quality of care and the perceptions of providers and users of health services about the quality of services (17,18).

Methods
Botswana has a total of 26 public hospitals widely distributed across the country ( Figure 2). The hospital system is structured into three levels based on the population served  (24). They are found in smaller villages, have bed capacity of 20-70 beds, and each serves a population of up to 10,000. The seven district hospitals represent the next level of care. They are found in major villages and towns with bed capacity of 71-250 beds. The three referral hospitals (two general and one psychiatric) provide specialist services in various areas (24). All public hospitals are directly under the MoH which provides policy guidance, overall supervision and all resources necessary for delivery of services. Until recently primary healthcare was the responsibility of the Ministry of Local Government. The research study from which this article is drawn adopted a case study approach (25) and used a mixed methods research design in assessing performance of the Botswana MoH (12). The dominant methodology was qualitative using document analysis, key informants and focus group interviews. The quantitative arm of the study comprised surveys for managers of hospitals and health workers. Both qualitative and quantitative strands of the research occurred simultaneously answering related aspects of the study questions (26,27). Data were collected in 2009 and 2010. This article is based on data from document analysis, key informant interviews and surveys of managers and health workers. A list of indicators in Table 1 guided data collection.

Document Analysis
Data were abstracted from published and unpublished documents which included National Development Plans; MoH policies, plans, and consultancy reports; annual reports of public hospitals; national statistics reports and related reports from government and other agencies; research reports or papers; and media reports. We searched electronic databases such as PubMed, Science Direct, Ovid, Scopus, Web of Knowledge, ProQuest, and Google Scholar for published articles.

Key informants interviews
A total of 54 key informants were purposively selected and recruited through personal contact. A snowballing technique was used to identify some of the informants. The participants comprised policy-makers, senior managers and staff of the MoH (N= 40) including a total of nine retired employees who held key positions in the Ministry and senior officers from various stakeholder organizations (N= 14), including nongovernmental, private and professional organizations. All interviews were audio taped and transcribed.

Surveys
A total of 42 managers from 25 out of 26 public hospitals participated in the survey. One hospital was excluded because it was on a transition from mission to government owned. These comprised hospital superintendents, chief medical officers, chief nursing officers/registered nurse, and hospital managers where such positions were available. A total of 389 out of 523 randomly selected health workers from 25 hospitals participated in the survey. The focus was on health workers responsible for direct patient care. These were nurses, doctors, pharmacists/technicians, and medical laboratory scientists/technicians. While efforts were made to use the staff lists from the government human resource database and hospital lists, this was challenging because the records were outdated. In addition, many employees were not available for various reasons including further education, transfers and resignations. The sample comprised nurses (66%), pharmacists/pharmacy technicians (13%), medical doctors and laboratory scientists at 10% each and others. Both surveys addressed views of participants on services provided in public hospitals, which is the main focus in this article, and broader issues of staff satisfaction. Data were collected through selfadministered questionnaires which were pre-tested.

Data analysis
Data from documents and transcripts were analyzed using content and thematic analysis respectively guided by Miles and Huberman's approach of data reduction, data display and conclusion drawing/verification (28). A deductive approach was adopted with the study indicators acting as the organizing framework. For this article, data analysis for both surveys was mainly descriptive determining proportions and percentages using SPSS 19 (SPSS Inc., Chicago, IL, USA) and Microsoft Excel 2007. Data from the different research strands were analyzed separately and integrated at the interpretation and discussion phases. .. " Some of the participants with this view indicated that when doctors are in charge of hospitals, they focus mainly on clinical services, neglecting non-clinical aspects of the facilities such as administration and engineering services. Educational preparation of non-clinical hospital managers was also a source of concern. Qualifications for these hospital   . Lack of clarity between the role of the hospital superintendents and hospital managers was also noted. The two cadres were observed to have management responsibilities with no job descriptions creating conflicts in some of the hospitals.

Authority and decision-making
The hospitals receive their budgets from the department of clinical services. Once the budget has been allocated, managers have limited authority on the use of funds: "After funds have been sub-warranted to facilities, they do not have much decision to make really except to follow the set rules of using the money from the votes for the purposes for which they were allocated for. They do not have the rights to use the money the way they feel like…" From the managers' survey, only 31% felt they had control on budget allocations for their hospitals and the same proportion felt there was flexibility in the use of funds. Half of the managers felt that senior management of hospitals cannot make important decisions about use of funds without undue consultation with the MoH. Lack of control on human resource functions in particular recruitment and staff promotions was also a concern for managers ( Figure 3). However, 50% of the managers felt they had control over staff deployment. Centralized control of decisions affecting the welfare of patients was noted: "...when you wanted to procure food for patients…you have to make an order here, then the order is delivered. You sign some papers that you received the order, and this thing goes from one office to the other until it is approved, but then the payment has to be done at the Ministry... "

Service availability
Proportion of population within a specified distance from a health facility Through the MoH and the Ministry of Local Government responsible for clinics and health posts, the country has an extensive network of health facilities. Physical access to health services increased from a national average of 88% of the population within 15km radius of a health facility in 1997 to 84% of the population within 5km radius in 2007 (24,30). Despite this achievement, there are disparities in the distribution of facilities creating inequities in access particularly for rural populations. In 2006/2007, 96% of the urban population was within 5km radius of a health facility compared to 72% of rural population (30). Some geographical areas were greatly disadvantaged such as Kweneng West, South East and Boteti where only 5%, 14%, and 22% of their population were within 5km radius of a health facility respectively. Some participants attributed the poor distribution of health facilities to poor priority setting and rationalization of infrastructure: "...a hospital for Moshupa was one of the top hospitals on the list. One for Tonota was also right at the top. And we had gone to Shakawe where we found people and they were waiting to go 400km to Maun for hospital treatment. We went to Gumare where we found a ward shared by males and females and it is so far from anywhere…" While the number of hospitals has remained constant since 2000, the MoH focused on upgrading structures considered to be in disrepair. Four district hospitals and a psychiatric referral hospital were recently upgraded.

Number and distribution of inpatient hospital beds per 10,000 population
At a national level, the number of inpatient beds has shown a steady increase with expansion of physical infrastructure from 3,572 beds in 1998 to 4,239 beds in 2008 (1) representing an increase of 18.7% beds in ten years. The ratio of beds per 10,000 population remained relatively stable at 22-24 beds/10,000 population over the years (Figure 4) until the recent upgrading of hospitals that increased the number of beds to 5013 in 2010 bringing the bed/population ratio to 27.9 beds/10,000 population. This is considered quite reasonable for developing countries (31). However, concerns have been raised on the distribution of beds which is considered inequitable and inefficient in some places (31,32). Averages as low as 6.6 beds/10,000 population in some areas and as high as 50.9 beds/10,000 population in others were reported (31). A challenge in the distribution of inpatient beds is illustrated by the bed occupancy rates in public hospitals. Over the years, some district hospitals recorded suboptimal bed occupancy rates. For example, between 2000 and 2005 the bed occupancy rate in these hospitals ranged between 40% and 61% (33,34). Some of these hospitals were upgraded resulting in significant overcapacities. On the contrary, the two general referral hospitals, Princess Marina and Nyangabgwe, were under considerable pressure for beds with occupancy rates of 87-120% (33,34). In 2009, the bed occupancy rates were 222% and 143% for Princess Marina and Nyangabgwe respectively (35). The major challenge cited concerning the referral hospitals is the lack of primary or district hospitals within their localities which subject these facilities to providing services that could be provided in lower level facilities (36).

Service capacity Status of hospital buildings
The physical infrastructure for health in Botswana has been described as more than adequate. The newly upgraded district hospitals were described as some of the best in the region. However, overcapacity in these hospitals attributed to ineffective planning processes was cited: "

Molepolole was a 125 bed hospital. Its average census in the year before it closed was 62, and you replace it with a 332 bed hospital! If you had a 60 bed hospital with 120 patients in it, I could see why you might need a bigger hospital. But when you have got a 125 bed hospital with only 60 patients per day in it, why would you want to build a 300 bed hospital?"
Concerns were raised country wide on the lack of maintenance for facilities. From the surveys, only 35% of the health workers felt that their hospitals were well maintained, a view shared by only 21% of the managers. Most of the managers indicated lack of preventive maintenance plans (79%) and management lack of control over maintenance issues (60%).

Availability of functional medical equipment
The findings on medical equipment were mixed. While most hospitals particularly in rural areas struggled with lack of equipment, others reported modern and sophisticated equipment. Where such equipment was reported, some concerns were raised on its appropriateness and the ability to optimize its use considering the skills and experiences of personnel: "We tend to procure state-of-the-art equipment, complicated equipment. We bring it into our facilities where we are using (allow me to use) third world trained doctors, third world experienced doctors who may have never seen that machine...I buy the machine that is supposed to do 150 different things for me, but when it comes here I do only 5. The rest I don't know...So we have these machines, very complicated, but not fully utilized". Maintenance of equipment was an issue for many hospitals. More than half of the health workers (57%) in the survey felt that equipment in their hospitals was poorly maintained. This view was shared by 64% of the managers. About 60% of the managers also indicated that their hospitals do not have functional procedures for equipment maintenance and the same proportion felt that hospital management had no control over maintenance of equipment.

Availability of Medicines
Shortage and erratic supply of medicines was reported by hospitals across the country. This was mainly attributed to inefficient supply chain management systems: "The Ministry provides the money for the procurement of medicines. With the money that is currently provided by government, if we had an efficient procurement and distribution system, the drugs should be adequate for our national needs. But the supply chain management is inefficient..." The poor supply of medicines limited access to services with patients having to travel long distances from one facility to another looking for drugs: "...of course we pride ourselves that each and every patient has access to services within 5km radius of a health facility. But that doesn't guarantee that they will find drugs every time they get there...As a result, they spend time moving from clinic to clinic spending money...In terms of access really we still have a long way to go... " The financial costs involved in getting medicines in terms of travel and sometimes purchasing from private pharmacies is a major concern for the Botswana public (37).

Availability of information technology to support delivery of health services
The MoH made considerable strides in promoting the use of Information and Communications Technology (ICT) in healthcare. A number of patient management information systems were introduced in hospitals. These included the Integrated Patient Management System implemented in few hospitals with plans to roll it out to all facilities, teleradiology in national referral hospitals, the Patient Information Management System dedicated to management of patients on antiretroviral treatment and cancer and Tuberculosis (TB) registration systems (36). While the Ministry's efforts are recognized, challenges of multiple standalone systems with limited interoperability were noted. Despite the introduction of these systems in public hospitals, most health workers (53%) felt their hospitals did not have adequate Information Technology (IT) to support provision of quality patient care. This was shared by 34% of health managers while 29% felt that technology was adequate.

Availability of skilled and motivated human resources
Almost all the hospitals reported a general shortage of staff that limits delivery of services and achievement of planned hospital activities. This finding is supported by the survey results where most of the health workers (74%) and managers (69%) indicated that their hospitals have inadequate numbers of staff. The main challenges reported were in the area of specialist personnel resulting in unnecessary referrals. Challenges were cited within the human resource management processes. Lack of integrated planning characterized by disjoint in human resource, infrastructure and service delivery planning created inefficiency in public hospitals: "When it comes to planning, there we are poor…We have the infrastructure. It comes up. All of a sudden we don't have people…We run around looking for people and yet we have designed it. We said we want this facility to provide cardiac services. We buy cardiac machines, no cardiology trained staff…" Inefficient deployment and utilization of staff were seen to affect delivery of services: "But there is also a challenge where specialists are not placed in their area of expertise. This happens often because of promotion. For example, theatre nurses may be promoted to psychiatric hospital... "

Service quality
Quality management systems Very few hospitals reported their quality management related activities. Quality improvement strategies and plans were observed to be outdated in 2009 (38). Hospitals were also seen to lack focus on quality issues characterized by lack of evidence of clinical governance structures and strategies, failure to address quality issues in management meetings, and lack of procedures and evidence of conducting clinical audits or monitoring effectiveness of care (38). Several participants interviewed concurred with these observations: "When it comes to the quality of clinical care, I have a problem...because I am not able to assess whether we are giving good quality care or not. I don't have the system to assess. The quality system is not functioning very well...and personally I feel the quality is not good enough... " Contrary to this finding on the lack of quality systems, most of the managers (76%) indicated that quality management system is actively used in their hospitals to improve the quality of care. This view was shared by less than half of the health workers (49%).

Performance standards
From the key informant interviews, participants cited a lack of national performance standards in public hospitals contributing to variability in performance. This was noted as a concern particularly considering that the majority of the medical doctors in Botswana come from different countries with a different background: "At the moment there are very few documented standards within the health system and those that do exist, most are not aware of. Basically most people work according to the standards by which they were trained which may be good but they come from different places and they have different standards... " However, from the surveys, more than half of the health workers (55%) and managers (59%) indicated that their hospitals have good performance standards. Lack of hospital accreditation mechanisms, weak legislation and policies regulating quality of care were also cited as some of the challenges.

Provider perception of the quality of care
Healthcare providers seem to have a more positive view of the quality of services provided in public facilities. In a study conducted by the MoH, half of the participants rated the quality of care as good (39). The same study cited a lack of basic equipment, an unclean environment, lack of privacy and poor referral systems as issues affecting delivery of services. Similarly, despite challenges noted on availability of resources in the current study, most of the managers (74%) felt that the quality of services has improved. This view was not shared by participants from the interviews, who generally felt that there were substantive quality issues in hospitals. These spanned from inadequate resources through to failure of the MoH to support the hospitals in managing their quality related problems.

Patient perceptions of quality on care
The perspective of users of health services on the quality of care contrasted with that of healthcare providers. Over the years, studies conducted on patient satisfaction highlighted a myriad of concerns including inadequate staff, and poor staff attitudes (40). The views of the general public expressed through call-in radio programs were consistent with the findings of these studies (5). In general, there are concerns around shortage of resources, particularly health personnel, which contributes to long waiting time and poor access to services as a radio caller noted: "Our hospitals do not have resources and there is shortage of medical doctors. It is possible to be booked to see a doctor after three months, when you come you are told he/she is not available because he is alone…You find one doctor helping so many patients, or you are told he/she is still out to help other patients while your own illness progresses and finally you give up and go home without assistance".

Discussion and policy implications
We conducted a comprehensive assessment of the public hospital system in Botswana using the WHO HSPAF, in particular the health service delivery function. While such assessments are most commonly done using numerical performance indicators, this study relied mainly on qualitative indicators focusing on structures, processes, functions and resource inputs as opposed to outcomes. In this regard, we identified the strengths and weaknesses in the system that policy-makers and stakeholders can use as levers to improve the system for better outcomes.

Organization and governance
Our analysis has demonstrated that the organizational structure of the Botswana public hospital system is hierarchical and highly centralized. The locus of authority and decisionmaking is the MoH headquarters. This leaves hospital managers with limited control of key resources necessary for delivery of health services such as the budget, equipment and human resources. This situation affects managers' capacity to respond to local needs and priorities and may also act as a disincentive for cost containment and quality improvement creating inefficiencies in the delivery of services, as noted elsewhere (6). Many countries that have experienced similar challenges of over centralization creating inefficiencies in their hospital sectors undertook hospital reforms as part of the broader health sector reforms of the 1980s through to the 1990s (6,41). The most common reform was management autonomy (6) which entailed the transfer of power and decision-making from central ministries to health managers or other entities such as district health boards. Giving health managers decision rights over key resources is viewed as an incentive to improve performance (6). While evidence on the impact of such an intervention on hospital performance is limited (6,42), some studies reported some degree of improvement. In Uganda, autonomous private not-for-profit hospitals had better quality of care and were more successful in managing personnel and medicines than public hospitals with limited autonomy (43). Improvements in clinical and financial management were reported in Kenyatta National Referral Hospital following the granting of autonomy (44). For Botswana, no major structural change has occurred in public hospitals since the inception of the public health system until recently when the need for some form of reconfiguration to enhance overall efficiency of the system became a key policy issue. There are now ongoing efforts to establish management structures at district level to take over responsibility for health service delivery. While this will address the existing challenges of over centralization, some forms of oversight and accountability mechanisms will be needed to ensure that the hospitals continue to respond to national health priorities. Examples of these mechanisms include strong central planning, use of targets and contractual agreements found in countries like Zambia and New Zealand (45,46). Significant leadership challenges were reported in the internal structure of public hospitals. In 2009/2010, the MoH introduced general managers without clear job descriptions creating ambiguity and blurring of roles. Consequently, the clinician-general manager conflict (47) which is beginning to emerge is characterized by tension around power, authority and control. This can have a negative impact on service delivery. Clear job descriptions for both cadres are necessary for the smooth running of the hospitals. Appropriate training of these managers is also important (47,48).

Service availability
At a national level, Botswana has invested significantly in health infrastructure in terms of hospitals and primary healthcare clinics ensuring access to services. However, inequities were observed. This underscores the importance of moving beyond national averages in analysis of health system performance to sub-regional and even local levels to unmask some of these inequities that may affect health outcomes. There are considerable challenges facing tertiary care in the Botswana's two national referral hospitals. The lack of lower level hospitals where these referrals hospitals are located has a profound impact on their functioning. These challenges may include limited access to tertiary care for populations outside the two cities as the referral hospitals are forced to function as local hospitals. In Lusaka, Zambia, a study revealed that due to lack of a lower level hospital, over 80% of patients at the University Teaching Hospital, which is the country's national referral hospital, were from Lusaka district (49) causing inefficiencies in the referral system. There is also possibility of limited access to hospital services for communities within the cities as high occupancy rates increase the chances of denied admissions. Other challenges associated with high occupancy rates include poor patient outcomes including increased risk of hospital-associated infections (50), limited capacity of the referral hospitals to respond to emergencies (51) with negative impact on patient outcomes, and poor staff motivation related to high workload.

Service capacity
Although it was not possible to accurately determine the number of health workers in Botswana's public hospitals due to lack of a robust human resource information system, hospitals throughout the country reported inadequate numbers particularly in specialist areas. While health worker shortages are a global problem especially in developing countries (13), the critical issue for Botswana is effective utilization of human resources. Poor deployment creates inefficiencies through an inappropriate skill match which may contribute to poor service quality and staff motivation. Poor staff retention is also a concern that contributes to high attrition rates. Inadequate maintenance of hospital buildings, non-functional and inequitable distribution of equipment, and poor availability of medicines are some of the key challenges facing public hospitals in Botswana. These challenges will need to be addressed if acceptable levels of healthcare services are to be provided (13).

Service quality
Public hospitals are faced with a myriad of quality related problems. The absence of quality management and improvement structures, and lack of practice standards to guide delivery of services had a negative impact on the quality of care. Several participants cited the proposed establishment of the Health Inspectorate department in the MoH as an opportunity to improve the quality of patient care in public hospitals. This department is envisaged to develop and oversee implementation of national healthcare standards. In addition, the MoH has engaged the Council for Health Services Accreditation of Southern Africa (COHSASA) which is a regional organization that confers accreditation on health facilities that comply with specified quality standards (52). Accreditation is reported to have been piloted in a few hospitals. The success of these initiatives, however, will depend on the nature and strength of incentives to motivate behaviour change within the hospitals. Accountability is identified as one of the elements that influence hospital performance (6). Strong accountability mechanisms will be needed to hold managers responsible and accountable for good results.

Usefulness of the World Health Organization Health System Performance Assessment Framework in assessing performance of hospitals
We used an aspect of the WHO HSPAF, the health service delivery function, to guide assessment of Botswana's public hospital system. In this regard this article is contributing to development of methodologies for assessing health systems performance, particularly on the use of WHO HSPAF at subsystem and organizational levels. Unlike earlier studies conducted in Botswana assessing hospital performance using economic methods focusing on efficiency, the use of the WHO HSPAF in this analysis provided an opportunity to take a comprehensive view of the hospital system identifying strengths and weakness in various facets of hospital system performance. This approach is in line with the notion that health systems are complex entities that require a multidimensional approach to performance assessment (53,54). Development of performance indicators to measure performance was a key activity in this analysis. While some works in this area have been done by organizations such as USAID and WHO, our focus was on developing indicators that are relevant to Botswana and that would provide policyand decision-makers with information that addressed their needs, and hence, can be used to improve the system. In this analysis, we also adopted an approach that recognizes that organizations or systems have multiple stakeholders who might have different expectations and criteria by which they judge their performance (55). The opportunity provided by the use of WHO HSPAF in terms of multiple assessment domains and indicators allowed the different stakeholders which included policy-makers from the MoH, representatives of non-governmental organizations, hospital managers, health workers and the general public to contribute their own perspectives to the analysis. In taking this multi-constituency approach, divergent views in assessment of performance were acknowledged as differing perspectives of stakeholders (55). The essence of measuring performance is to be able to make improvements in areas where performance is lacking. By focusing on processes, structures and resource inputs (53,56), the use of the HSPAF enabled the assessment of the hospital system to specifically indicate problems contributing to poor performance. The indicators used in this analysis are mainly process indicators as opposed to outcomes indicators. While outcome measures are useful in identifying major areas of concern, they can fail to identify levers of system performance.
On the other hand, process indicators have been credited for providing feedback that can be used to improve system performance by identifying dysfunctional processes (57).

Limitations
The use of the WHO HSPAF in assessing performance of health systems is relatively new. Although there might be some level of consensus around the domains of assessment such as capacity of the system, availability and quality of services, performance indicators are still under-developed and there is generally lack of consensus on the type of indicators to use. The indicators used in this analysis are therefore specific to Botswana making comparability across countries a challenge. In addition, these indicators have not been widely used or researched. More research on the use of this current methodological approach is needed. While assessment is usually associated with numbers, most of the indicators used in this analysis are qualitative and suffer quality issues such as validity, reproducibility, specificity and sensitivity (58). To address some of these quality issues in this study, we reviewed key documents, interviewed various stakeholders, and conducted a survey of health managers and workers to get a broad and complementary view of performance. In addition, some aspects of performance are difficult to measure (59) and yet important. While acknowledging the challenges with indicators, Girard et al. (58) caution against overemphasis on methodological rigour at the expense of pragmatism.

Conclusion
We successfully used the WHO HSPAF to conduct a comprehensive assessment of the public hospital system in Botswana. Although these hospitals are under pressure to improve performance, they are faced with considerable challenges that will need to be addressed if changes in performance are to occur. The organizational structure and governance of public hospitals have a negative impact on the hospitals' performance. Lack of management autonomy limits the ability of hospital managers to influence performance of their facilities. The ongoing efforts to establish district health management structures to coordinate and manage health service delivery at district levels will need to be expedited and sufficient capacity built to ensure that these structures can effectively handle their responsibilities. The important role of health system resources in service delivery needs to be appreciated and concerted efforts made to improve resource allocation and utilization.