Quality of Sick Child-Care Delivered by Community Health Workers in Tanzania

Background: Community health worker (CHW) interventions to manage childhood illness is a strategy promoted by the global health community which involves training and supporting CHW to assess, classify and treat sick children at home, using an algorithm adapted from the Integrated Management of Childhood Illness (IMCI). To inform CHW policy, the Government of Tanzania launched a program in 2011 to determine if community case management (CCM) of malaria, pneumonia and diarrhea could be implemented by CHW in that country. Methods: This paper reports the results of an observational study on the CCM service delivery quality of a trial cohort of CHW in Tanzania, called WAJA. In 2014, teams of data collectors, employees of the Ministry of Health and Social Welfare trained in IMCI, assessed the IMCI skills rendered by a sample of WAJA on sick children who presented to WAJA with illness signs and symptoms in their communities. The assessment included direct observations of WAJA IMCI episodes and expert re-assessment of the same children seen by WAJA to assess the congruence between the assessment, classification and treatment outcomes of WAJA cases and those from cases conducted by expert re-assessors. Results: In the majority of cases, WAJA correctly assess sick children for CCM-treatable illnesses (malaria, pneumonia, and diarrhea) and general danger signs (90% and 89%, respectively), but too few correctly assess for physical danger signs (39%); on classification in the majority of cases (73%) WAJA correctly classified illness, though more for CCM-treatable illnesses (83%). In majority of cases (78%) WAJA treated children correctly (84% of malaria, 74% pneumonia, and 71% diarrhea cases). Errors were often associated with lapses in health systems support, mainly supervision and logistics. Conclusion: CCM is a feasible strategy for CHW in Tanzania, who, in the majority of cases, implemented the approach as well as IMCI expert re-assessors. Nevertheless, for CCM to be effective, in Tanzania, a strategy to implement it must be coordinated with efforts to strengthen local health systems


Implications for policy makers •
Community case management (CCM) of malaria, pneumonia and diarrhea by community health workers (CHWs) is a feasible strategy for accelerating childhood mortality reduction in settings of rural Tanzania.

•
In a large majority of cases, CHW in Tanzania can implement protocols for classifying and treating prominent causes of childhood mortality as well as their Integrated Management of Childhood Illness (IMCI) trained medical professionals.

•
For CCM to be successful in Tanzania, and settings like it, coordinated efforts are required that strengthen systems for the extension of essential logistics and supervision from healthcare facilities to communities.
CHW in community settings.The third, and last, objective of this study was to contextualize these results in the wider milieu of rural health systems where the pilot program was held.For our third objective, we direct readers' attention, also, to an earlier publication that draws upon qualitative data from the evaluation of this program and illustrates the perceptions of CHW, clinic-based healthcare workers, district healthcare managers and community members on the feasibility, effectiveness and acceptability of the CCM program. 17n 2007, the Government of Tanzania developed the Primary Healthcare Services Development Program (Swahili acronym MMAM) which called for a recognized national cadre of CHW. 18In 2010, the United Nations Children's Fund (UNICEF) conducted a situation analysis to clarify optimal ways to operationalize a national CHW program. 19heir report recommended that CHW be based in the community, linked to the health system as a salaried and nationally recognized cadre, and formally trained to provide an integrated package of services addressing preventative and curative aspects of primary healthcare, emphasizing maternal, newborn and child health as well as IMCI and management of malaria, pneumonia and diarrhea.
In 2010, the Tanzanian Training Center for International Health (TTCIH), the Ifakara Health Institute (IHI), Columbia University, and the Ministry of Health and Social Welfare launched The Connect Project, which operationalized a CHW program based on the above recommendations, in three rural districts. 20Connect is designed as randomized controlled implementation trial that examines the effect of introducing CHW (called Wawezashaji wa Afya ya Jamii or WAJA in Swahili) on maternal and child health and survival in three rural districts over 5 years.Connect is registered with the International Standard Randomized Controlled Trial Register number (ISRCTN96819844).This paper explains the implementation and results of an observational study of the quality of sick child-care provided by WAJA, using CCM, within the Connect intervention areas.How WAJA performed vis-à-vis national standards, their specific clinical errors, and the systems factors that shape the quality of care, as well as the implications of these for CHW policy in Tanzania, are reviewed and discussed.

Study Setting
This study occurred between June and August 2014 in three rural districts, Kilombero, Rufijii, and Ulanga, which were the intervention districts for the Connect Project.Of these villages, 101 were included within the Connect trial, and 50 were randomized to receive the WAJA intervention, with the remaining 51 villages serving as controls with a total of 370 000 people under observation in the trial, half exposed to the WAJA intervention.This study took place only in WAJA intervention villages.
Study Subjects A total of 142 WAJA were recruited and trained in three cohorts between October 2010 and July 2013.Of these 60 were included in this study, approximately 20 from Rufiji, which comprises of one-third of the study area, and 40 from Kilombero and Ulanga districts.All WAJA were selected based on community requirements and the minimum eligibility criteria outlined for civil service employment in Tanzania.This includes (1) 'form-four' education level (equivalent to 10th grade educational-level in the United States); (2)  residency in the community they will serve; and (3) selection by their communities.Recruitment processes were led by district leadership, namely District Educational Officers to verify applicants' eligibility and members of Council Health Management Teams (CHMT), in partnership with leaders of intervention communities, who posted and screened applications, interviewed those qualified and selected finalists to stand for village election.Those WAJA elected by their communities underwent an academic year (9 months) of training at the TTCIH, which developed a CHW curriculum that incorporated the biological, clinical and community aspects of primary healthcare.Core competencies included in the training were health promotion and preventive services including HIV/STI prevention and counseling, education on family planning and distribution of oral contraceptives and condoms, safe motherhood promotion, and aspects of focused antenatal care.As well basic community treatment protocols were included that covered essential newborn care, IMCI, and CCM for malaria, pneumonia and diarrhea.In line with the protocol for civil service employment, for the duration of the project all WAJA were salaried employees of their respective CHMTs.
The CCM component of the training occupied seven sessions and involved both classroom training and clinical practice.CCM implementation was phased into the field operation during the first month of their deployment in order to ensure the presence of supervisors to monitor and coach WAJA to provide CCM in their village, monitoring the standard of care that had been established during training.The content of the WHO/UNICEF, Caring for the sick child in the community package for children aged 2 months to 5 years, was adapted by the MOHSW (known in Swahili as Bangu Kitita) for use by CHW, such as WAJA, to perform IMCI.WAJA were specifically trained to assess and classify children presenting uncomplicated cases of fever (as a proxy for malaria), a cough with fast breathing (as a proxy for pneumonia), diarrhea, eye infections, anemia, measles, malnutrition, and ear infections, as well as check their immunization status, assess for general and physical danger signs, and refer them as appropriate.

Study Design and Sample Size
This was a cross-sectional study of the quality of WAJA sick child-care using CCM.Our aim was to enroll a representative sample of WAJA vis-a-vis the geographic spread of the study area, population distribution, and the demographic characteristics of the WAJA themselves.The overall sampling frame (n = 142) included all WAJA that had completed training between 2010 and 2013.The selection of WAJA was stratified by district, as indicated above, because WAJA deployment was concentrated more heavily in Kilombero and Ulanga district (the larger two study sites) where two-thirds of WAJA resided, whereas only one-third resided in Rufiji district.Accordingly, 40 WAJA from the former districts and 20 from the later district were selected respectively, for a total sample size of 60.In addition to the region where they were deployed, the 60 WAJA were selected to be representative of the entire group in terms of their gender, age, and training cohort.Based on WAJA supervision reports from the month prior to the assessment (May 2014) it was expected that WAJA would see an average of five sick children over the course of a week, with each WAJA observed for a week in total.Thus the actual sample size, with an anticipated total of 300 sick child observations, was estimated to detect a +/-6.2%precision in our point estimates of the proportion of sick children assessed, classified and treated correctly by WAJA for all illness classifications, estimated at 50% (precision being higher for proportions that are farther from the mean, ie, 15% of 75%).
In this, we assume 95% confidence and a design effect of 1.2 to account for intra-class correlation of observations among children seen by the same WAJA.Sampled WAJA were asked to provide routine IMCI visits everyday over a period of a week during which the research teams made unannounced visits to observe them in their respective communities.Of all the children whom WAJA encountered during this study, 43 (12%) were not enrolled because they did not meet the eligibility standards described above.Eight other cases (2%) were removed from the study after the data was collected if research managers could not attest to its reliability after data reconciliation steps, which involved data collectors and supervisors.Within the week spent with each sampled WAJA, the research team was able to conduct five valid, usable sick child observations per WAJA.

Outcomes
Quality of care was defined using indicators of correct case management based on existing consensus indicators for the quality of sick childcare developed for IMCI by the World Health Organization (WHO). 21These were then adapted to reflect the Tanzania clinical guidelines to assess the quality of IMCI in facilities, and again, to ensure relevance for the community settings in which WAJA work.Box 1 describes the definitions of correct assessment, CCM-treatable illness and danger signs.The primary outcomes of interest were children checked for fever, cough, fast breathing and diarrhea, children assessed for general and physical danger signs, children whose classification from WAJA matched that of the expert re-assessor, children with CCM-treatable classifications whose classification from WAJA matched that of the expert re-assessor, children with CCM-treatable classifications whose treatment from WAJA matched that of the expert reassessor.
Secondary outcomes included rational use of antibiotics and ACT, tabulating the frequency of WAJA dispensing medication to children without suspected malaria or pneumonia.Measures of health system support reflect the frequency of supervision visits made to the WAJA during the quarter prior to their enrollment in the assessment, the availability of essential drugs and supplies to WAJA, and use of service delivery data for planning at the village level.

Data Analysis
Data were entered into Stata 13 (College Station, TX), with data processing and reconciliation taking place in Dar es Salaam, Tanzania and New York, USA.Key indicators were calculated as proportions and with 95% CI, adjusted for clustering of sick child consultations performed by the same WAJA.The inter-rater agreement for key variables, that is the proportion of classification and treatment recommended by IMCI expert re-assessors that were also enacted by WAJA, is also reported as a kappa statistic with 95% confidence intervals.A scale proposed by Landis and Koch 22 was used to interpret the magnitude of agreement for a range of kappa values as follows: Fair: K = 0.21-0.40,Moderate: K = 0.41-0.60,Substantial: K = 0.61-0.80,Almost perfect: K = 0.81-1.00.We performed a 'simplified clinical pathways analysis' in order to identify the proportion of WAJA errors at each step of the CCM algorithm (assessment, classification, and treatment) for consultations where a child was given a classification from the expert IMCI re-assessor.

WAJA Characteristics
As explained above, this assessment took place as an embedded study within Connect, a randomized controlled implementation trial of the impact of WAJA deployment on child health and mortality.Impact results of Connect are being prepared for dissemination in a separate publication.Table 1 presents the characteristics of the WAJA included in the sample.WAJA were predominantly male (63%), and the median age was 28 with a range from 23 to 41.As per the sampling methodology, the WAJA representatively reflect the relative size of their respective training cohorts, with 40% coming from cohort 1 and 2, trained in 2010-2011 and 2011-2012, respectively, and 20% coming from the third cohort, trained in 2012-2013.Client registers from the three months prior to the assessment were available for review by data collectors, who excluded five WAJA registers because the information in them seemed unrealistic and could not be explained by WAJA.The median number of sick children aged two months to 5 years seen by these WAJA (n = 55) during this period was 66, with median ranging between 44 and 100.
Patient Characteristics A total of 300 sick child consultations were observed in the study (Table 2).Though additional sick children were provided care during the study, these were ultimately removed from the analysis because they were aged less than two months or over 5 years, did not present a new complaint to WAJA or required immediate referral.Roughly a third of the children observed 1-year old or younger (34%), a quarter between one and two years old (26%), and less than a quarter were between three and five years of age (22%).More than half of the children observed were male (54%).The vast majority of children observed were accompanied by a female caregiver (97%).The most common complaints as reported by the caregivers to the WAJA were cough (63%), fever (51%) and diarrhea (14%).About three-fourths (76%) of caregivers reported more than one complaint, and the median number of complaints was two.(14), cold/runny nose (14), flu (13), skin rash (4), ear problem (2), mouth sores (1), general weakness/headache (7).

Case Mix
Table 3 shows the case mix of sick children seeking care for a new episode of illness from the WAJA on the day of the assessment visit, as defined by the IMCI-trained medical professional who re-assessed the children.The majority of the sick children presented with at least one CCM-treatable illnesses (29% with malaria, 26% with cough and difficulty breathing and 17% with diarrhea).Roughly one-fifth of the sick children observed presented with at least one danger sign (19%), or an uncomplicated fever (19%).The most common danger signs were palmar pallor, diarrhea with blood in stool, fever for more than seven days, and swelling of both feet and ear infections.

Quality of Assessment and Treatment
The key results related to the quality of sick child-care in the areas of assessment, classification and treatment are presented in Table 4.In column 1, we present indicator tasks that were taken from CCM guideline used to train and assess WAJA.In column 2, we present the number of children for whom the task should have been performed based on the re-assessment of each sick child that was carried out by the IMCI trained clinician.In column 3, we present the proportion of the sick child consultations in which the WAJA carried out the same task (ie, Column 1) that the Expert Re-assessor carried out (ie, Column 2).In column 4, the 95% CI for each estimate.
The proportions shared in column 3, thus represents the proportion of cases in which the WAJA carried out the sick child consultation the same way as the Expert Re-assessor, as per each indicator task.We use this measure as a proxy for the level of correctness with which WAJA conducted care for sick children.It can be inferred that the proportion that remains after the proportion reported in column 3 is deducted from 100% is, indeed, the proportion of cases, per task, in which the necessary task was performed by the Expert Re-assessor only.
Notably, WAJA correctly assessed 90% of children for the presence of febrile, respiratory and diarrheal illnesses.Among the children with cough, 91% were correctly assessed for the presence of fast breathing by counting respiratory rate.Although 89% of children were correctly assessed for four general danger signs, a much lower proportion of them were correctly assessed for physical danger signs (39%).The physical danger signs most overlooked were malnutrition assessment using MUAC tape (assessed for 54% of children) swelling in both feet of the sick child (assessed for 55%), and chest in-drawing (assessed for 65%).
The illness classifications assigned by WAJA were consistent with those of the IMCI-trained medical professional who conducted the re-assessments for 73% of the sick children observed.The proportion rose to 83% when considering only classification of CCM treatable illnesses.Over threefourths (78%) of the 232 children with a CCM-treatable illness, including uncomplicated fever and malaria diagnosed through use of rapid diagnostic tests, cough and fast breathing as a proxy for pneumonia, and diarrhea, respectively, received the correct treatment from the WAJA for their illness(es).
A breakdown by type of illness shows that children with a febrile condition were more likely to receive correct treatment (84%).Out of the 86 children classified with malaria, 84% (72) received ACT correctly, while of the 58 children who were classified as having uncomplicated fever 84% (49) also received the recommended dose of paracetamol from WAJA.Out of the 162 children presenting with a fever, 18 (6%) were not administered an m-RDT because of stock-outs among the WAJA from whom they sought care (see Table 5).They, accordingly, were not classified as CCM-treatable for a febrile condition, and referred to a clinic.A total of 77 children enrolled in the assessment with persistent cough and fast breathing, of which 74% received correct antibiotic treatment from WAJA.The assessment identified 51 children with CCM-treatable forms of diarrhea, of which 36 (71%) received the correct regimen of ORS and zinc from the WAJA.Out of the five children with diarrhea that presented with bloody stool, three continued to receive treatment from WAJA, against protocols for CCM.Of note, of the 51 sick children with a CCM-treatable classification that did not receive correct treatment from WAJA, 36 (71%) were infants less than 12-months of age.35% (24) of the 68 children not classified with an illness, despite not needing treatment as specified in clinical guidelines, still received either an ACT, antibiotic, ORS or zinc from the WAJA.Two-thirds (37) of the 56 children presenting with danger signs received a referral a Child consultations in which the task in Column 1 was performed by the Expert Re-Assessor and should have been performed by WAJA.b % of the child consultations for which the task that was performed by the Expert Re-Assessor and should have been performed by WAJA (ie, Column 1) was performed by WAJA.c Includes those who were classified as having fever for less than 7 days without danger signs, positive results on m-RDT; excludes children with fever for more than 7 days to whom m-RDT was not administered.d Includes only those who tested negative for malaria using m-RDT.e Among those prescribed a CCM medication, irrespective of classification status.f Among those without any danger signs requiring referral; of those that did not have vaccination status checked 19 did not have a card available at time of visit.
from the WAJA.Table 5 presents the kappa values, which reflect the probability that inter-rater agreement presented in Table 4 can be ascribed to subject properties, ie, comparable adherence to the IMCI algorithm, or chance.Kappa values were calculated to assess the level of inter-agreement between WAJA and IMCI expert re-assessor for IMCI illness classification (all, CCM-related and specifically for uncomplicated malaria and fever, cough and fast breathing and diarrhea), treatment provision and instruction (uncomplicated malaria, cough and fast breathing and diarrhea).Kappa values ranged from 0.71-0.88(mean = 0.79), and six out of the eight values reflected 'almost perfect' between WAJA and IMCI expert or sick children's caregiver (See Table 5).

Quality of Counseling to Caregivers
Of the 203 caregivers whose children, sick with a CCMtreatable illness, received treatment from WAJA, 151 (74%) received information about the dose, frequency and duration of treatment; however, only 137 (67%) of the caregivers of sick children were able to correctly describe how to administer the treatment (Table 4).Of the 246 children who did not require a referral owing to presentation of danger signs, WAJA checked the vaccination status of 217 (88%).WAJA advised 35 out of the 51 caregivers of children with CCM-treatable diarrhea (69%) to give extra fluids and continue feeding their sick child.
Health Systems Supports to WAJA For the 3 months prior to the assessment (March-May, 2014), on average, each WAJA had two IMCI-related supervision exchanges with their health facility supervisor (Table 5).Two-thirds of WAJA report to routinely participate in village government meetings and collaborate with village governments, using their service delivery information to plan village health activities.About 85% of WAJA involved in the assessment had all the critical CCM drugs (ACT, amoxicillin, ORS and zinc, and paracetamol).Information on availability of key IMCI supplies is also provided.74% of WAJA enrolled had MUAC tape available during the assessment, and 86% had stock of malaria rapid diagnostic tests.See Table 6, which provides information on essential health systems supports that were available to WAJA during the time of the assessment.

Clinical Errors Pathways
Figures 1-3 presents an analysis of the clinical steps followed by WAJA during these episodes and pointing out their common errors.Availability of rapid diagnostic kits for testing malaria enabled the correct assessment and classification of most children presenting with fever (144/162).Of these children, 86 were correctly classified with malaria and 58 with uncomplicated fever.Of the 86 with malaria, 14 were incorrectly treated, 3 because of the WAJA were stocked out of ACT.The other 11 also presented with other CCMtreatable conditions and received other treatments.Of the 58 children testing negative for malaria and classified with uncomplicated fever, 49 received correct treatment from

Supervision
Average number of IMCI-related supervisions from health facility supervisor in the past quarter Average number of IMCI-related supervisions from village supervisor in the past quarter Proportion of WAJA who participate in village government meetings, including use of their health management information for planning 67%

Drug supply (WAJA had in stock on the day of the visit)
All critical CCM drugs (ACT, amoxicillin, ORS, zinc, paracetamol) 79%
WAJA (paracetamol and counseling to the caregiver).Of the 162 children presenting with fever, 18 (11%) could not be tested for malaria because WAJA were stocked out of malaria rapid diagnostic kits.A total of 77 children were classified with cough and fast breathing.Of these, 62 (81%) were correctly assessed.Mistakes (n = 15) occurred because WAJA failed to ask about time since onset of coughing, check for chest in-drawing, and count breaths.Of the 62 children assessed correctly, 6 were mistakenly classified as not having cough or fast breathing.These children also presented with cases of diarrhea, malaria, or both, together with their cough and rapid breathing.Of the 62 children with cough and fast breathing who were classified correctly, 57 went on to receive the correct antibiotic treatment for suspected pneumonia.Five who were assessed and classified correctly also presented with malaria and   diarrhea, and received treatment for those illnesses, but not for suspected pneumonia.One child that was assessed and classified incorrectly also presented with another CCMtreatable condition and was treated for it, but, again, not for suspected pneumonia.14 children who presented with cough and rapid breathing received no treatment for any illness, six of whom were classified correctly and eight incorrectly.Fifty-one children presented with diarrhea, of whom 40 were correctly and 11 were erroneously assessed by WAJA.Five of those mis-assessed had also been classified with suspected pneumonia or malaria, and 6 were mis-assessed because WAJA failed to assess thoroughly for dehydration (observe drink and abdomen skin folding back).Errors in classification of children with diarrhea (n = 11), were all due to mis-classifying their dehydration status.In total, 36 children received the correct treatment of ORS and zinc for diarrhea, while 12 were incorrectly treated, receiving either ORS or zinc, but not both.Three of the 51 children presenting with diarrhea were mis-assessed, mis-classified and incorrectly treated.They also presented with malaria.Of the cases in which ORS was provided without the zinc accompaniment (n = 9), six of the occurrences were amongst WAJA stocked out of zinc.

Discussion
Over 30 countries have adopted CCM as a strategy to increase access to and coverage of treatment interventions for childhood illness. 23This is, to the best of our knowledge, the first study to assess the quality of CCM in Tanzania.
The Connect Project carried out this intervention with the objective of demonstrating that community residents, having received the level of training and implementation support described in this paper, could provide care for sick children, comparably to their facility-based counterparts who, at the time of this study, were the only cadres authorized to do so.In our evaluation of this proposition, we found 78% of children with a confirmed CCM-treatable classification were treated correctly per CCM guidelines.This compares favorably with results from health facilities evaluated for quality of care for sick children during the roll out of facility-based IMCI in Tanzania, where it was found that 74% of children presenting with the same illnesses received correct treatment. 24This study demonstrated that WAJA correctly assessed 90% and correctly classified 73% of the sick children enrolled, as compared to results from IMCI assessments at facilities, which found that 90% and 63% of cases encountered by facility providers were correctly assessed and classified. 25To assess the influence of location and years of deployment on WAJA performance, we analyzed the prevalence of errors separately according these two variables (region where WAJA were deployed and years since completion of training and deployment).With this, we found no appreciable differences in error levels among WAJA deployed to different regions and WAJA that completed training in different cohorts.That this study employed re-examination of sick children by a medical professional trained in IMCI is a particular strength, and allows us to provide information on the quality of the care for specific, confirmed CCM classifications.Another strength of this study is that it was community-based, in contrast to other studies, which are facility-based use patient data, we observed WAJA in the villages where they ordinarily work.Acute respiratory infections (ARIs) are the most common illnesses in childhood, and pneumonia is responsible for 15% childhood mortality in sub-Saharan Africa 26 and Tanzania, specifically. 27According to the Demographic and Health Survey for Tanzania, published in 2016, roughly 40% of children with ARI receive appropriate treatment. 28This study and others reinforce that further effort is needed to improve its identification and treatment at the facility and communitylevel.New diagnostic technologies for pneumonia may be useful, 29 if they are affordable and embedded in functional logistics systems.Though our findings may question the assertion that CHW can rationally dispense antibiotics -35% of children not presenting with a CCM-treatable illness received treatment from WAJA -it bears note that 15 of those cases were of children who received ACTs from WAJA stocked out of m-RDT.However, the results presented question if m-RDT are sufficient to improve case management of febrile illness.Even among malaria cases identified with m-RDT, 84% received the correct ACT treatment from WAJA.This is slightly less than the 88% of children who were correctly treated for malaria at facilities during facility-based IMCI assessments.Lapses in the availability of m-RDT and implementation of effective supervision suggests that, as in other studies, weak health systems linkages undermine the potential of technologies and expanding the community health workforce to sustain improvements in IMCI. 30he fact that a quarter of WAJA observed lacked both ORS and zinc while attending to sick children underlies the gap between ideal and actual correct treatment rates for diarrhea.Further, it underscores the need for strong logistics systems for CCM.However, concerning suspected pneumonia, clinical exam errors were made in the counting of respiratory rates, despite the presence of timing devices, as observed in other assessments of CHW in sub-Saharan Africa. 12Over a quarter of basic diarrhea cases identified during the study were incorrectly treated by WAJA.In most of these cases sick children received ORS only, without zinc supplementation.Though related to challenge of stock outs, clinical errors, altogether, underscore the need for strengthening supervision systems and training facility-and community-based providers to correctly manage children presenting with multiple symptoms, recognizing the enhancing effect of zinc on ORS therapy for diarrheal disease and correct use of common CCM tools widely available, such as timers. 31his assessment has limitations that should be considered.The CHW and clusters chosen for the assessment are part of a pilot experiment of a program alternative being considered for scale up by the Government of Tanzania.Accordingly, the Connect Project provided support to local implementers, at times supporting them to procure and distribute essential medicines and supplies for WAJA when stock outs were inevitable and conduct independent and routine supervision of the WAJA when the means to accomplish this were not available.This, in turn, may have positively biased the results documented in this study.In addition, WAJA may have provided better care than normal because they were being observed, a phenomenon referred to as the Hawthorne effect. 32Finally, the exclusion of 43 sick from our study may introduced bias in our analysis.Among those removed, 12 were neonates and six exhibited urgent symptoms that necessitated immediate referral.This decision was consistent with the design of the WAJA service delivery package, which did not permit them to deliver care regimens including intravenous or rectal artesunate and antibiotic treatment, essential for sick neonates and severely ill children over 2 months.Nevertheless, it undermined our study's potential to discern if WAJA could help those children in the direst need of immediate medical attention.Of those removed, 15 had been attended to by the WAJA for the same condition during the period prior to data collection in their community.The removal of these children from our analysis prevented the study from identifying if errors in the WAJAs' treatment of these children and if this might explain their need to return to the WAJA for care for the same illness.
The purpose of this study was to inform decisions on the design of a national CHW program in Tanzania where policymakers were deliberating the content of a community-based primary healthcare policy and services package to take to scale.Saliently, WAJA seem to be fulfilling the expectation of bringing child healthcare closer to communities.The subset of WAJA observed with patient registers available saw a median of 66 sick children over the course of a quarter during which WAJA experienced temporary stock outs of essential medicines for CCM and received fewer than the recommended number of supportive supervisions.That Connect did not include components to address gaps in the implementation of IMCI at the facilities serving villages where WAJA were deployed, may be problematic for WAJA performance.Evidence from studies conducted in Tanzania shows that many health workers surveyed at facilities are not trained in, able to demonstrate understanding of IMCI, 33 or do not adhere to IMCI treatment and referral guidelines for children with severe illness. 34This study suggests that introducing CHWs in rural Tanzania, through the training and deployment model described in this paper, reaches children with needed services, the quality of which is, in the majority of cases, comparable to that of IMCI experts, when their prognoses are uncomplicated, rather than having children after they have become more severely ill.Roughly one-fifth of children (19%), presented with a danger sign that indicated a severe illness, and two-thirds of these children were referred after receiving pre-referral care from WAJA.Although this proportion may seem low, in some of these communities, referral to a health facility may not be feasible.Furthermore, the referral rates observed are comparable with those of IMCI trained health workers based in facilities in Tanzania and other countries in sub-Saharan Africa. 35,36Other findings from the Connect evaluation relate to implementation costs and contribute additional evidence pertaining to the feasibility of scaling up the WAJA model.Whereas the annual cost of training Clinical Officers, a threeyear training program, was measured at $731 per year, the cost of training WAJA, a one year training program, was $833.50 per year.In terms of the costs of sustaining WAJA after training, the Connect cost evaluation found that the per capita cost of deployment and routine management and support of this cadre was $1.16 per capita.This is compared with $2.72, the average annual per capita deployment cost of CHW from programs derived from a meta-analysis of CHW program costs in several countries in sub-Saharan Africa, suggesting that Connect cost estimates are an additional favorable feature of the WAJA model. 37pecific performance weaknesses observed among WAJA were recognition of physical danger signs, particularly malnutrition, correct assessment and classification of suspected pneumonia, and management of children presenting with multiple illnesses, including effective referrals.Although this is consistent with findings from other settings, poor WAJA performance on danger sign recognition, particularly through use of MUAC tape, is of serious concern as children with these symptoms are at greatest risk.This study underscores the importance of having a functional 'continuum of care' that ensures the routine availability of high quality referral-level services at the network of facilities to which CHW will refer sick children.This will prove essential for maintaining effective supervision structures and logistics connections between villages, where CHW will operate, and the formal health system.

Conclusion
This observation-based assessment of the quality of sick child-care provided by WAJA trained in IMCI and CCM in Tanzania demonstrates the degree to which CHW can perform lifesaving child survival interventions up to nationally accepted standards of quality, as modeled by IMCI-expert reassessors.The description of the intervention, including the discussion on health systems supports to the WAJA, delineate the processes that generated these encouraging results.Clinical errors emerged prevalently as the WAJA conducted their work, illuminating the effect of a weak health systems context, even in settings where a special project backstops CHW training and management.The Government of Tanzania, as it formulates its national program on CHW, and considers community IMCI and CCM for large-scale implementation, should consider the findings of this study: while training community members to deliver lifesaving interventions to children up to acceptable standards is feasible, it necessitates functionality, responsiveness and support from local health systems.As this study observed, CHW will struggle to achieve their potential unless their clinical skills, particularly those required for managing children with multiple conditions especially pneumonia, and identifying and referring severely ill children, are improved and better supported by reliable supervision and uninterrupted logistics systems.

Figure 2 .
Figure 2. Cough and Fast Breathing Clinical Errors Pathway.

Definition of CCM-treatable illnesses (ie, classification based on assessment)
WAJA takes the sick child's temperature with a thermometer and asks the caregiver about the presence or history of fever, or the caregiver offers this information; WAJA tests sick child's for presence of malarial parasites using rapid diagnostic tests (m-RDT).
malarial treatment is called Artemether Lumefantrine (Alu).During the first year of the program, the MOHSW changed the first line antibiotic for pneumonia from cotrimoxazole to amoxicillin.Also in line with adaptions to the national standard of care, in 2012, the project districts rolled out rapid diagnostic malarial test (m-RDT) kits for assessing febrile illness to health facilities and WAJA.that would prevent the WAJA from giving the sick child oral medications.In our study, these were convulsions, unusual sleepiness or unconsciousness, inability to drink or eat anything or vomiting everything.Children excluded from eligibility on account of their age or because they had a Box 1. Definitions of Correct Assessment and Illness Classifications Definition of correct assessment for CCM-treatable illness Assessment of uncomplicated fever:

Table 2 .
Characteristics of Sick Children and Their Caregivers (n = 300) a Abdominal pain

Table 3 .
Proportion and Number of Observed Clinical Interactions Where IMCI-Trained Medical Professional Determined Signs/Symptoms to Be Present (n = 300) a Abbreviations: CCM, community case management; m-RDT, rapid diagnostic malarial test; MUAC, mid-upper arm circumference.a Signs and symptoms are not mutually exclusive; one child can have more than one classification.

Table 4 .
WAJA Performance -Proportions of Children for Whom Specific Case Management Tasks, Necessary According to IMCI Expert Re-assessors, Were Performed by WAJA Abbreviations: WAJA, Wawezashaji wa Afya ya Jamii; IMCI, Integrated Management of Childhood Illness; CCM, community case management; ACT, artemisinin combination therapies; ORS, oral rehydration salts.

Table 5 .
Inter-rater Agreement of Classification and Treatment Performance Between WAJA and IMCI Expert Re-assessor Wawezashaji wa Afya ya Jamii; IMCI, Integrated Management of Childhood Illness; CCM, community case management.

Table 6 .
Health System Supports for CCM, as Reported by Sampled WAJA