“Hearing from all sides” How legislative testimony influences state level policy-makers in the United States

Background: This paper investigates whether state legislators find testimony influential, to what extent testimony influences policy-makers’ decisions, and defines the features of testimony important in affecting policy-makers’ decisions. Methods: We used a mixed method approach to analyze responses from 862 state-level legislators in the United States (U.S.). Data were collected via a phone survey from January-October, 2012. Qualitative data were analyzed using a general inductive approach and codes were designed to capture the most prevalent themes. Descriptive statistics and cross tabulations were also completed on thematic and demographic data to identify additional themes. Results: Most legislators, regardless of political party and other common demographics, find testimony influential, albeit with various definitions of influence. While legislators reported that testimony influenced their awareness or encouraged them to take action like conducting additional research, only 6% reported that testimony changes their vote. Among those legislators who found testimony influential, characteristics of the presenter (e.g., credibility, knowledge of the subject) were the most important aspects of testimony. Legislators also noted several characteristics of testimony content as important, including use of credible, unbiased information and data. Conclusion: Findings from this study can be used by health advocates, researchers, and individuals to fine tune the delivery of materials and messages to influence policy-makers during legislative testimony. Increasing the likelihood that information from scholars will be used by policy-makers may lead to the adoption of more health policies that are informed by scientific and practice-based evidence.


Background
How do policy-makers make decisions? The question is of considerable interest to a wide variety of constituents including the media, lobbyists, interest and advocacy groups, the public, and academic scholars. Multiple models have been proposed for explaining how policy-makers utilize information; Weiss (1) described six models: the problemsolving model, the knowledge-driven model, the interactive model, the political model, the tactical model, and the enlightenment model. The most common concept of research utilization is the problem-solving model, which involves direct application of a specific study to a pending decision.

Implications for policy makers
• Given their many time constraints, policy-makers often use the easiest accessible information to make decisions. The results of this paper suggest that testimony may overcome some barriers encountered when using more accessible, but less direct, credible, or evidence-based forms of information. • Policy-makers should consider partnering more closely with those who develop and deliver testimony (e.g., advocates, researchers, public health officials) to produce testimony that is more likely to be influential, focusing on issues such evidence sources that are unbiased and presenters who are credible.
In the knowledge-driven model, the process is linear: applied research is conducted to test the findings of basic research for practical action and findings are implemented. The interactive model reflects an interactive search by policy-makers for knowledge and its process is not one of linear order, but a disorderly set of interconnections. The political model assumes that policy-makers already have made their decision (e.g., based on ideology or intellect) prior to receiving research and that research will only be used to supporting existing positions. The tactical model suggests that content of research is not as important as the fact that research is being conducted. In the enlightenment model, the concepts and theories from social science influence the policy-making process more than specific research studies (1). Furthermore, experience from numerous countries shows that the decision-making process is dynamic and complex and involves interaction among a variety of forces, including problems, politics, and policies (2)(3)(4)(5). It is often chaotic, and some scholars suggest a garbage can model of decision-making where policy-makers rummage around for problems, solutions, and choice opportunities often relying on the easiest accessible information to make decisions (6). Too often, the decision-making process relies heavily on the easiest accessible information, making it less likely that a solution will be well-matched with the most appropriate evidence (e.g., 'Garbage Can' logic) (7). Having a weak understanding of factors that influence the decisions of policy-makers can make it difficult for advocates and practitioners to affect policy change, particularly when working within a context of limited resources and multiple message framing and delivery options. Preparing and presenting testimony to legislative committees is one strategy used to communicate information and influence policy-makers. Testimony allows interested parties to present their positions and/or evidence on potential legislation and may influence legislative decision-making (8). Although policy-makers make some decisions outside of committee hearings (3,9), presenting testimony at hearings offers a unique opportunity for outsiders to publicly persuade policy-makers and potentially change policy. Schlozman and Tierney (10) conducted one of the most well-known studies of interest group activity to date. They found that testifying before a legislative committee is the tactic most frequently used by interest groups to inform and influence policy-makers. Their findings were confirmed by numerous other studies of federal interest group activity and among state level lobbyists and organizations (11,12). Testifying at committee hearings was the most frequent way interest groups interacted with state legislators (12). Through content analysis of written and oral testimony, the use of stories, scientific evidence, and format of testimony delivery are documented in peer-reviewed literature as effective tactics for interest groups to use when communicating with policymakers (13,14). However, there are conflicting findings about to what degree and in what capacity testimony influences policy-makers' decision-making. Some researchers found testimony from interest groups to be a "scripted show for the committee…" (15), that is often times ignored (16). Yet other literature suggests that testimony can influence and strengthen the language of a policy (16), and may be an important source of credible information for later floor debate (17), even shifting as many as 30-40 votes in the U.S. House of Representatives (18). How testimony influences decisions and what features of testimony are most helpful-according to the policy-makers themselves-is largely missing from the literature. Filling this gap could improve communications among academics, advocates, the public, and policy-makers and influence the types and format of testimony used during committee hearings. Increasing the likelihood that information from scholars will be used by policy-makers might lead to the adoption of more health policies that are informed by scientific and practice-based evidence. The aim of this study was to better understand: if and to what extent testimony influences policy-makers' decisions (e.g., testimony is not influential, somewhat influential, or very influential); at what point testimony influences decisionmaking along a continuum (e.g., awareness, opinion, decision-making, or voting); and what features of testimony are important in affecting policy-makers' decision-making process. Findings from this study can be used by health advocacy groups, institutions, universities, and individuals to fine tune the delivery of their material, messages, research, etc. to influence policy-makers during legislative testimony. Qualitative analysis Qualitative responses to the survey questions were analyzed using a general inductive approach in which the authors

A r c h i v e o f S I D www.SID.ir
sought to discover themes in the raw data unencumbered by a predefined structure or theory (12). This approach was particularly appropriate given the lack of existing research on the influence of testimony overall and what elements of testimony are most influential. The authors conducted initial exploration of the data, first grouping responses into specific themes (e.g., the presenter's use of visual aids affects testimony's influence) and then generating broader categories for coding based on the themes (e.g., presentation characteristics). Codes were designed to capture the most prevalent themes from the first round of analysis. Several rounds of testing were conducted using multiple raters to refine the codes developed by the authors.
Inter-rater reliability tests conducted in NVivo 10 achieved Kappa statistics ranging from 0.72-1. Given the limitations of using interviewer-recorded responses rather than wordfor-word transcription and the brief nature of the legislators' responses via the telephone survey, the statistics indicate an acceptable level of inter-rater reliability. Differences between coders were discussed and agreement reached for each round of testing. When all codes related to a survey question achieved a Kappa of 0.75 or higher, remaining coding was conducted with single raters. Only one code did not achieve this threshold for responses to "what about the testimony affects its influence". Double coders were used to code all responses to this question to ensure reliability. Coders discussed discrepancies in codes to come to agreement on the final classifications.

Quantitative analysis
The authors imported the thematic and demographic data into SPSS 20 for the quantitative analysis. Descriptive statistics and cross tabulations were computed for the characteristics of legislators, the type of influence of testimony, the action taken based on the testimony, and the features of testimony that the legislators found influential. The authors used the results of descriptive analysis to identify the most related themes and subthemes. Table 1 displays the characteristics of the legislators in the sample. Eight hundred and sixty-two policy-makers were interviewed as part of this study, with representation from a diverse group of policy-makers. Forty-six percent of the legislators identified as Democrat and 52.6% as Republican; 25.5% were women and 74.5% men; 35.8% had earned a college degree and 44.7% had a postgraduate degree; and 93% were over 40 years of age.

Influence
The majority (81.3%) of legislators implied that testimony at least somewhat influenced policy-making. We identified three categories within these responses reflecting different types of influence: Somewhat Influential (51.0%), Very Influential (11.9%), and Contingent Influence (18.3%). In addition, 2.2% of legislators suggested that testimony was Not Influential during policy-making and 16.5% did not clearly state whether or not testimony influences policy-making (e.g., some legislators' responses included contradictory statements). Table 2 provides a description for how each of the influence-related categories was coded along with sample responses.
Respondents who implied that testimony was Somewhat Influential mostly perceived testimony as one of many types of evidence incorporated into a decision. Largely, these responses suggested that testimony was a factor in policy-making, but not the main one, though more influential factors were rarely provided. Often, respondents suggested that they would use testimony to support their existing position on an issue. Some of these policy-makers also recognized that testimony could be anecdotal and subjective. One responded that testimony did not have as much influence as "...the information itself. I let the facts determine my vote". Additionally, many respondents implied that testimony was influential but this influence was contingent on factors related to and/or outside of the testimony. These factors included relevance to the issue, relevance to the legislator's interests and priorities, presenter credibility, presentation quality, alignment with existing community priorities, consistency with ideology, consistency with other research, and the legislator's previous position on the issue. For example, one legislator stated that the influence of testimony depended on "if I have my mind made up or not…. " Respondents that suggested testimony did not influence them were few, but these legislators were likely to state that they made their decisions prior to the testimony, based on their own research. Responses suggesting that testimony had a strong influence were somewhat more frequent, though most did not specify why they were strongly influenced by testimony, with the exception of a few responses citing personal contact.

Testimony and continuum of action
When analyzing responses to the following question: "How does testimony influence your decisions about policy?" themes emerged reflecting a continuum of action from increasing awareness about a policy issue to changing a legislator's vote on legislation. Table 3 provides codebook definitions of each of these stages of action and examples of the legislators' responses.
Increased awareness was the most common action resulting from listening to testimony. Legislators indicated that testimony increased their overall awareness on a policy issue and made them better informed about the different sides of an issue. This might involve asking questions of those presenting the testimony. Legislators' decision-making was the second most prevalent action affected by listening to testimony. Most legislators in this group specifically stated that testimony helped them to make a decision. Many legislators in this group reported taking an action that reflected a decision. The most common action taken was studying and thinking about the testimony. Writing or amending legislation to address the issue from the testimony were also reported. Other actions included discussing the testimony with colleagues and supporting or advocating for a policy after hearing influential testimony. Finally, some legislators stated that the testimony would cause them to conduct research on the policy issue discussed. This might involve seeking out experts or looking for additional, relevant evidence on the issue. Very few legislators in the

Testimony characteristics that influence legislators
All survey respondents answered the question, "What about testimony affects it influence on you?" The results below are reported for those legislators that found testimony at all influential (n= 701). Themes emerged from the analysis across three broad characteristics of testimony: 1) characteristics of the presenter; 2) characteristics of the testimony content; and 3) characteristics related to testimony presentation. Table 4 provides codebook definitions and examples of each of these characteristics.
Characteristics of the presenter (e.g., the presenter's background) were mentioned most often by respondents. Legislators found presenters that were credible or knowledgeable/an expert in their field enhanced the influence of testimony. The "credentialed expert" was described by the majority of respondents that reported presenter characteristics as influential; other kinds of presenters, such as those with a personal connection to the issue or constituents, were not mentioned as often by legislators. In most cases, legislators did not explicitly attribute a presenter's credibility or knowledge to the organization which they represent, and simply referenced the individual speaker. However, given the brevity of most responses, it is possible that the individual's credibility or knowledge may be a proxy for the credibility attributed to the speaker's organization or the organization's level of experience.
Characteristics of the testimony content (e.g., the nature and type of information presented) also affected the influence of testimony for nearly half of respondents. Legislators reported including credible content and sharing data enhanced the influence of testimony. Credible content was often described as content that was "objective", "unbiased", or "accurate". Sharing of data included "statistics" or "facts". Fewer legislators reported evidence-based content or personal stories as influential in testimony.
Characteristics of the presentation itself also affected the influence of testimony, though to a lesser degree than characteristics of the content or the presenter. Some legislators found that presentation design and delivery enhanced the influence of testimony. Design elements such as an organized format, visual aids, or take-home handouts were described as components of influential testimony. Legislators also described presenters with an engaging delivery, who were easy to understand or concise, as influential.

Discussion
This paper investigates whether state legislators find testimony influential, to what extent testimony influences policy-makers' actions, and what features of testimony are important in affecting policy-makers' decision-making processes. Testimony appears to be at least somewhat influential across legislator age, gender, political party, and other demographic characteristics. This is a promising finding for public health advocates seeking to influence policy-making in democratic governments where testimony is common. Moreover, testimony not only affects legislators' awareness of an issue but also, to a lesser extent, legislative decision-making. Importantly, some characteristics of Legislator responses coded to this category indicated that testimony was influential during policy-making, but this influence was contingent on a factor(s) related to and/or outside of the testimony.

A r c h i v e o f S I D www.SID.ir
"Anyone can defend any side of something that they want to. If the credibility of the presenter is high then the testimony has a large impact".

"Sometimes nothing, sometimes it makes a big difference, depends on what my constituency would agree with how I weigh the testimony, what I think is best for the state".
Not influential Legislator responses coded to this category indicated that testimony did not influence policy-making.

"The testimony does not influence me. I do my own research".
Somewhat influential Legislator responses coded to this category indicated that testimony influenced policy-making, but the strength of this influence (a little or very influential) was not specified.
"Every bit of it helps me to vote; both sides need to be presented in order for anyone to know the best thing for the state".
"It's one of many factors".

Very influential
Legislator responses coded to this category indicated that testimony had a large effect, or was a significant factor in policy-making.
"A large effect, testimony absolutely affects decisions".
"Testimony has a lot of influence on decisions". testimony are more influential than others -specifically the characteristics of the presenter (e.g., credibility, knowledge, expertise, etc.). These findings support several of the models of influence discussed by Weiss (1). Legislators most often responded that testimony affected their awareness of an issue. This supports the enlightenment model, where policy-makers assimilate information from a variety of sources to develop their theories about social issues. In this model, testimony is one source of information that contributes to a policy-maker's broader perspective, which affects decision-making. Additionally, many legislators cited testimony as directly influencing their decision-making, which reflects the dominant perspective of the problem-solving model. A proportion of legislators cited that testimony inspired them to conduct more research, which is congruent with the interactive model, and likewise, the opportunity for researchers to influence policy-making, by reminding policy-makers to become better informed on an issue. While some policy-makers stated only using testimony when it supported their existing position (i.e., the political model), this was not a major theme. Several lessons from this study are congruent with previous research from numerous countries. For instance, the analysis builds on Kasniunas's (15) research and supports the notion that testimony is not just a "scripted show", but rather is influential on legislators' decision-making. Research utilization literature from the U.S. and other countries also suggests that policy-makers use information that they find credible or that includes data and information presented by knowledgeable experts (19)(20)(21). The findings are also consistent with research completed in the U.S. that suggests presenter credibility has the most potential to influence legislators during the policy-making process (22)(23)(24)(25). Legislators actively assess the credibility of the person delivering the testimony, and as one legislator from the sample noted, "It doesn't take long to sort out the liars". Furthermore,

A r c h i v e o f S I D www.SID.ir
the finding that presenter characteristics are the most influential feature of testimony is consistent with research that personal contact (including testimony) is the most important facilitator of the use of evidence by policy-makers (25). Both of these findings suggest that policy-makers care a great deal about the person delivering the evidence, and potentially less about the evidence itself. While testifying before policymakers is an effective way for researchers and citizens to increase personal contact and directly influence policy decisions, these findings suggest that presenters need to be aware of how personal characteristics, particularly personal credibility and expertise, might influence whether or not their message is received. Additionally, the analysis confirms previous research findings that testimony can influence policy language (16), as legislators reported that testimony influences them to re-write legislation being drafted or to add amendments to existing legislation. This research has certain limitations. First, this study relies on responses from a sample of 11% of U.S. state-level legislators. Despite the size, the demographics of the sample are representative of the U.S. population of legislators for gender, age, education level, and political party (26). Although the survey did not include responses from legislators outside the U.S., the results may be applicable to countries with democratically-elected legislative branches where there is similar diversity among policy-makers and testimony is common. Another limitation is that survey responses may not reflect the complete response of each legislator because responses were summarized by the interviewer and not transcribed verbatim. The authors accounted for this possible limitation by coding responses conservatively. If a specific code could not explicitly be inferred from the response (e.g., not enough details were given), the code was not selected. Finally, all of the data used in this study are self-reported by legislators and so we are unable to verify the accuracy or their statements against actual legislative action. Understanding how influential and in what ways testimony impacts policy-maker decision-making is important to effectively design communication for health policy development. For example, the finding that testimony increases a legislator's awareness of an issue suggests that communicating timely research findings on a regular basis may increase the visibility of a topic on the policy agenda. Even more importantly, the finding that testimony sometimes influences legislators' decision-making offers an opportunity for researchers to bridge the disconnect between evidencebased science and policy-making through testimony (4,27,28). In addition, the finding that presenter credibility/knowledge is most influential, followed by the use of credible data in testimony, suggests strategies that researchers and advocates can employ to improve the effectiveness of testimony by considering the implications of who presents the testimony and what data are included. Previous research on credibility suggests that both the legislator's perception of common interests and expertise of the presenter matter more than a variety of other characteristics often associated with credibility, such as academic credentials or political party identification (29). The presenter should emphasize interests he/she shares with the committee or highlight the breadth of expertise instead of focusing on a specific academic degree as a strategy to enhance the presenter's credibility. For example, Lupia (29) suggests presenters of scientific testimony emphasize that their past work has been replicable, transparent, and reliable. The results of this study also suggest some areas for future research. For instance, while some legislators took action as a result of testimony, it is not yet clear how to influence what action is taken. A small proportion of legislators said that testimony influenced their vote, but it is still unknown If they are knowledgeable/an expert in the field "The person presenting information has a firm grip on the issues and a high level of expertise".
The presenter's credibility "Whether I trust the person giving the testimony". "Integrity of the deliverer, their lack of an agenda".

Characteristics of the testimony content
Content is credible "The most influential testimonies are the rare occasions when research is presented without specific interests behind it, unbiased". "I have to know the source of the information and if it's credible".
Testimony includes data/facts "Being provided valid data makes the biggest impact. You can typically tell if a person is presenting facts or just making a plea on behalf of their cause". "If it's factual, I'll use it. If not, I won't and the individual presenting is immediately discounted".
Evidence-based content "Real research, peer reviewed science". "Empirical or scientific evidence".
Stories "Constituents' personal experiences, how the situation impacted them positively or negatively". "Having someone who has been affected by the issue testify about the effect it has had on their life".

Presentation characteristics
Presentation design "Use of visual aids helps me to understand what the speaker is talking about". "Written testimonies are most important because I will go back many times and read over them".
Presenter's delivery "The presenter's ability to convey the information and capture my attention when talking about the issue". "Speakers who are short in delivery time, not reading the testimony, easy to understand".

A r c h i v e o f S I D www.SID.ir
how testimony can be tailored to impact this ultimate policy outcome. In addition, credibility of the presenter was an important characteristic of testimony. However, it is still unclear how credibility is defined by state legislators, and whether findings from previous credibility research (29) hold true for presenters of legislative testimony. Future research can build upon the existing literature on presenter credibility to identify the characteristics that make some presenters more credible than others in a state-level legislative setting, particularly for presenters sharing testimony other than scientific findings (e.g., public health researchers and practitioners). Such research could enhance the effectiveness of disseminating information for evidence-based policymaking. Finally, replication of this study with legislators in other countries where testimony is common would be useful in understanding how other legislative bodies are influenced by testimony. This paper highlights the important role of testimony in policy-making and presents several characteristics that make it influential. According to the garbage can model, policymakers use only the easiest accessible information to make decisions. The results of this paper suggest that testimony may overcome some barriers encountered when using other, less direct or immediately accessible forms of dissemination. Testimony can provide policy-makers with more and higher quality evidence to have on hand before a decision, and may actually cause legislators to make a decision on a policy issue and help to bridge the gap between health research and health policy (4,28).