Addressing Health Equity Through Action on the Social Determinants of Health: A Global Review of Policy Outcome Evaluation Methods

Background: Epidemiological evidence on the social determinants of health inequity is well-advanced, but considerably less attention has been given to evaluating the impact of public policies addressing those social determinants. Methodological challenges to produce evidence on policy outcomes present a significant barrier to mobilising policy actions for health equities. This review aims to examine methodological approaches to policy evaluation of health equity outcomes and identify promising approaches for future research. Methods: We conducted a systematic narrative review of literature critically evaluating policy impact on health equity, synthesizing information on the methodological approaches used. We searched and screened records from five electronic databases, using pre-defined protocols resulting in a total of 50 studies included for review. We coded the studies according to (1) type of policy analysed; (2) research design; (3) analytical techniques; (4) health outcomes; and (5) equity dimensions evaluated. Results: We found a growing number of a wide range of policies being evaluated for health equity outcomes using a variety of research designs. The majority of studies employed an observational research design, most of which were cross-sectional, however, other approaches included experimental designs, simulation modelling, and meta-analysis. Regression techniques dominated the analytical approaches, although a number of novel techniques were used which may offer advantages over traditional regression analysis for the study of distributional impacts of policy. Few studies made intra-national or cross-national comparisons or collected primary data. Despite longstanding challenges of attribution in policy outcome evaluation, the majority of the studies attributed change in physical or mental health outcomes to the policy being evaluated. Conclusion: Our review provides an overview of methodological approaches to health equity policy outcome evaluation, demonstrating what is most commonplace and opportunities from novel approaches. We found the number of studies evaluating the impacts of public policies on health equity are on the rise, but this area of policy evaluation still requires more attention given growing inequities.

geographical distribution of each population in need to measure geographical diversity of disadvantage populations; (3) measuring the equity of infrastructure provision using mean targeting ratios comparing the tertiles of clustering of the disadvantaged population by black and minority ethnic groups, children, socio-economic disadvantage and retired household.

Emery, Fleisch & McIntyre 2013
Legislated changes to federal pension income in Canada will adversely affect low income seniors' health

Policy
Federally-funded retirement benefits (raising age of eligibility from 65 to 67)

Policy Area
Social Protection and Welfare

Geographic Scope
Localised ( Inequity Dimension Education + Health -poor health + Sex Summary 1) Study measured the associations between employment rates and various degree of flexible, deregulated labour market; different relative generosity and entitlement of welfare benefits (decommodification); and greater investment and emphasis on active labour market policies by comparing the age-standardised employment rates between healthy population with those with longstanding illness that was not limiting; or those with limiting longstanding illness, stratified by sex, education level and countries Hu et al. 2016 The impact of tobacco control policies on smoking among socioeconomic groups in nine European countries, 1990European countries, -2007 Policy Different tobacco control policies

Analysis
Descriptive analysis: graphical analysis + statistical summaries; Inferential analysis: fixed effects regression model

Inequity Dimension Education + Occupation + Sex
Summary 1) Study compared the country trends in the price and non-price measures over time 2) Study compared the trends in age-standardised smoking prevalence by education in each country for men and women over time 3) Study measured the association between price (popular cigarette and cheapest cigarette price) and non-price related tobacco control policies, and smoking, comparing the odds ratio between man and women stratified by education level and occupation

McNamara 2015
Trade liberalization, social policies and health: An empirical case study

Policy
Social protective policies of welfare states and labour markets during trade liberalization (focus on employment changes of textile and clothing sector)

Summary
Study measured the association between the orientation of public gender equality policies and gender equalities in health by 1) comparing the prevalence ratios and age-standardised prevalence of poor self-perceived health among women and men, stratified by country and country typology of family policies (dual earner, traditional-central, traditional-southern, market oriented, contradictory) 2) comparing the prevalence ratio of poor self-perceived health according to gender, stratified by age, born in low-income country, education level, marriage status and employment status

Summary
Study measured the association between welfare reforms and the probability of tobacco smoking; binge alcohol consumption; mental health; medical access; affordability of medical access; and routine Pap test and mammography screening between single mothers and the "control" group (married mothers, single nonmothers, and married nonmothers); between single mothers and a synthetic control group; and between employed and unemployed single mothers Funding of tribal health programs linked to lower rates of hospitalization for conditions sensitive to ambulatory care

Policy Area
Health System Management/Health Insurance

Geographic Scope
Localised -California, US

Analysis
Descriptive analysis: statistical summaries; Inferential analysis: generalised estimating equations regression model + multilevel regression model

Outcome(s)
Long-term: Hospitalization rate for ambulatory-care sensitive conditions

Summary
Study measured the association between higher funding of Tribally Operated Health Programs and reduced rate ratio of the hospitalisations for ambulatory care sensitive conditions, comparing among federal funding and medical price index, eligible population, socioeconomic status and resources, disparities in occupation, education, and household differences between indigenous and white population, Indian gaming revenue

Summary
Study measured the association between health insurance reform and the likelihood of having insurance coverage at each time point (1989,1991,1993,1997)

Analysis
Descriptive analysis: graphical analysis + statistical summaries; Inferential analysis: fixed effects regression model + difference-in-difference-in-difference regression model

Outcome(s)
Intermediate: economic index (years of completed education, ratio of family income to the federal poverty line, and family wealth); health services utilization; medical debt Long-term: composite chronic condition index (high blood pressure, diabetes, heart disease/heart attack, and obesity)

Summary
(1) Study measured the association between exposure (duration and timing) to Medicaid during early childhood on economic attainment and adult health stratified by income and predicted participation (2)

Summary
(1) Study measured the association to which investment was allocated to need by comparing the investment amount per household to their income deprivation decile (2) Study measured the association between the mental and physical health and the level of investments by comparing the mean scores of mental and physical health with lower, medium and higher level of investments (3) Study measured the association of level of investment and change in the mean score of mental and physical health, comparing between the pre-and post-scores, stratified by 3 investment groups of lower, medium and higher level.

Summary
(1) Study measured the trend of association between public health care allocations and measures of disease burden (antenatal HIV prevalence and infant mortality), income per capital and race over time (2) Study measured the association between public health allocations and burden of disease (antenatal HIV prevalence, infant mortality and crude death rate),

Policy
Community-based primary health care (PHC) system (started early 80s, expanded in 90s)

Policy Area
Health System Management/Health Insurance

Analysis
Descriptive analysis: graphical analysis + statistical summaries + principal components analysis; Inferential analysis: chi-square analysis + hierarchical regression model + meta-regression model

Outcome(s)
Long-term: prevalence of dental trauma in 12-year-old schoolchildren

Summary
Study measured the differences in prevalence of dental traumas between areas that were demonstrated to be "betteroff" (top 50% of areas) after the Healthy Cities Project implementation relative to the areas that were demonstrated to be "worse-off" (bottom 50% of areas)

Analysis
Descriptive analysis: graphical analysis + statistical summaries; Inferential analysis: change-on-change regression model

Outcome(s)
Intermediate: frequency of bus travel, active travel, car travel; transport poverty; transport injury; assault; car dependence; access to education, training, independent travel; effects on older citizens Long-term: physical/mental health Inequity Dimension Age -youth + Income -deprivation + Race/Ethnicity

Summary
(1) Study compared bus, car, and active transport use; road injuries; and assaults pre-and post-policy intervention between youth and adults (2) Study compared distances travelled by youth according to journey purpose by decile of deprivation pre-and post-policy intervention (3)  Summary 1) Study measured the association between the cycling initiative interventions and change in prevalence of cycling; walking; driving; and taking public transport to work, comparing the pre-and post-intervention absolute and relative changes in the intervention group with matched comparison groups; unfunded comparison groups; non-London national comparison groups 2) Study measured the association between small-area deprivation and prevalence of cycling to work, comparing the percentage-point change and ratio of change in the intervention group with matched comparison groups; unfunded comparison groups; non-London national comparison groups stratified by deprivation indices

Kearns & Mason 2015 Policy
Housing investment, transformational area regeneration programme, and relocation due to demolition and rebuilding

Geographic Scope
Intra-National Comparison: Glasgow, UK

Summary
Study assigned samples into 3 groups based on their current location, residential circumstances and receiving regeneration in their living areas (since 2008): those living in non-regeneration area and not receiving any improvements (nonmovers); those who have remained living in the same dwellings but received regeneration (remainers); and those who have moved out of the regeneration areas into a social-rented dwellings (outmovers). Study compared the odd ratios of each outcomes in the 3 comparison groups: between remainers and outmovers relative to nonmovers; and outmovers relative to remainers, each comparison stratified by dwelling quality, neighbourhood quality, sex, education level, occupation, long-standing illness, and household types

Summary
(1) Study modelled the estimated changes in incidence, prevalence and case-death rates of 16 tobacco-related diseases from the 4 interventions (2) Study modelled the years of life saved, adjusting for disability from the change in disease rates (3) Study modelled the cost of cigarettes, factoring in travel cost by estimating change in distance and time travelling to reduced and location of retail outlets (3) Study measured change in disease rate factoring in price elasticities from changes in smoking behaviour (4) Study modelled QALY and cost-savings, comparing between business-as-usual-model and the 4 interventions, stratified by different population groups

Ogilvie et al. 2008
The harvest plot: a method for synthesising evidence about the differential effects of interventions.

Geographic Scope
Not specified

Dataset(s)
Studies which assessed the effects of any type of population-level tobacco control intervention and had reported effects stratified by at least one demographic or socio-economic characteristic irrespective of study design, methodological quality or outcomes measured (N=85)

Analysis
Descriptive analysis: graphical analysis + statistical summaries; Meta-analysis: Harvest plot

Outcome(s)
Intermediate: change in awareness, change in attitude, change in perceived availability Long-term: change in self-reported smoking status Inequity Dimension Age + Education + Income + Occupation + Race/Ethnicity + Sex

Summary
Study systematically reviewed the results of studies on the effects of population-level tobacco control interventions on changes in awareness, attitude, perceived availability and self-reported smoking status stratified by income, occupation, education, gender, ethnicity, and age