Evidence-Informed Deliberative Processes for Legitimate Health Benefit Package Design − Part I: Conceptual Framework

Background: Countries around the world are increasingly rethinking the design of their health benefit packages to achieve universal health coverage (UHC). Health technology assessment (HTA) bodies support governments in these decisions, but employ value frameworks that do not sufficiently account for the intrinsically complex and value-laden political reality of benefit package design. Methods: Several years ago, evidence-informed deliberative processes (EDPs) were developed to address this issue. An EDP is a practical and stepwise approach for HTA bodies to enhance legitimate health benefit package design based on deliberation between stakeholders to identify, reflect and learn about the meaning and importance of values, and to interpret available evidence on these values. We further developed the conceptual framework and initial 2019 guidance based on academic knowledge exchange, analysing practices of HTA bodies, surveying HTA bodies and experts around the globe, and implementation of EDPs in several countries around the world. Results: EDPs stem from the general concept of legitimacy, which is translated into four elements – stakeholder involvement ideally operationalised through stakeholder participation with deliberation; evidence-informed evaluation; transparency; and appeal. The 2021 practical guidance distinguishes six practical steps of a HTA process and provides recommendations on how these elements can be implemented in each of these steps. Conclusion: There is an increased attention for legitimacy, deliberative processes for HTA and health benefit package design, but the development of theories and methods for such processes remain behind. The added value of EDPs lies in the operationalisation of the general concept of legitimacy into practical guidance for HTA bodies.


Background
Countries around the world are increasingly rethinking the design of their health benefit packages to support the progressive realisation of universal health coverage (UHC), ie, to select the appropriate set of services at fair levels of coverage and financial protection. [1][2][3] Many countries have established health technology assessment (HTA) bodies to support governments in these choices. 4 HTA assesses the value of a health technology and can inform decisions at different levels, eg, reimbursement decisions on a single health technology or regarding larger parts of the benefit package.
Decision-making on health technologies is an intrinsically complex and value-laden political process that takes place in an environment of diverging social values and interests. [5][6][7][8][9] However, value frameworks currently employed by HTA bodies do not sufficiently account for this complex reality. They are typically based on the use of 'substantive' criteria, which are believed to reflect the most important social values. This has led HTA bodies to use, for example, 'clinical benefit, ' 'safety' and 'cost-effectiveness' as important decision criteria. 4 These frameworks are ill fitted for considering the broad diversity of stakeholder values and may lead to insufficient sets of relevant information and sub-optimal decisions. [5][6][7]10,11 Several years ago, in response to these shortcomings, evidence-informed deliberative processes (EDPs) were developed. 12 An EDP is a practical and stepwise approach for HTA bodies to enhance legitimate health benefit package design based on deliberation between stakeholders to identify, reflect and learn about the meaning and importance of values, and to interpret available evidence on these values. Although the use of EDPs is relatively new, deliberative methods have been developed and used to some extent in the field of HTA since the 2000s. 13 This paper reports on the further development of EDPs with an emphasis on the key concept of stakeholder participation with deliberation. This is based on academic knowledge exchange, [14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30] analysing practices of eight HTA bodies, 31 surveying a total of 27 HTA bodies and 66 experts around the globe, 21,32 and experience with country level implementation of EDPs. With regard to the latter, EDPs are presently employed by national health authorities in Ghana, Iran, Moldova, Pakistan and Ukraine for revision of their health benefit packages, 33 and its principles were previously applied for similar use in Kazakhstan, Thailand, 34 the Netherlands 35 and Indonesia. 23 A companion paper reports on the development of a practical guide to support HTA bodies with the implementation of EDPs, including best practice examples from countries around the world -released at the HTAi meeting in June 2021. 31 Both papers target researchers, analysts and policymakers involved in benefit package design at Ministries of Health and/or related HTA bodies, and can together be read as a 'How to' guide for organising legitimate decision-making processes.
The paper starts with describing the conceptual framework of EDPs, including its elements and steps. It then focuses on the core element of EDPs, ie, stakeholder involvement ideally operationalised through stakeholder participation with deliberation. We conclude with several remarks on the added value of EDPs.
Conceptual Framework of Evidence-Informed Deliberative Processes Authorities take decisions in health benefit package design on behalf of the population they serve. They should provide stakeholders -specific population groups who bear the consequences of decisions -with well-justified, reasonable reasons to appreciate the decision-making process as fair and confer legitimacy to the process despite favouring a different decision-making outcome. 36,37 If this is not taken seriously by decision-makers, they risk losing their moral authority for making decisions. Subsequently, they may undermine the legitimacy of their own decision-making process.
Stakeholders are often categorised as one of the 7Ps. 38 These are: • Patients, the public and carer: patients are individuals with the lived experience of a disease or disorder who can provide information in the HTA process pertinent to that disease or disorder; public is an umbrella term which, in the context of the HTA process, incorporates all nonpatient, non-commercial, and nonprofessional stakeholders within the health sector; a carer or caregiver is usually recognized as holding the interests of the patient(s) in their care but may have additional needs and interests. 39 • Providers/health professionals: can bring in new perspectives, such as expert views on the effectiveness of technologies and the feasibility of alternative implementation options, including organisational aspects. • Purchasers: can provide information on the feasibility of alternative implementation options, including organisational aspects. • Payers: can provide generic non-disease specific perspectives and are often the public.  40 In pluralistic societies, stakeholders can be expected to disagree about what values can be used to include or exclude technologies from the package. 8 For example, in decisions on public funding of expensive cancer drugs, patients may argue that the best treatment should be made available, while other patients may argue that their treatment should not be displaced and taxpayers may reason that it is more important to use public resources efficiently. In comparison to all these other parties, health professionals may want to have access to the latest technological developments in their field. As a result, decisions are bound to be controversial because stakeholders likely disagree about what should be prioritised, who should benefit and who should not.
The accountability for reasonableness (A4R) framework recognises that stakeholders often justifiably disagree about the importance of specific social values in setting priorities and it argues that stakeholders are more likely to accept priorities that are the outcome of fair, legitimate processes. 41 In other words, stakeholders may agree with outcomes of a fair process even though they may have preferred another outcome. The A4R framework identifies four key conditions for organising fair processes: (i) all relevant values should be considered; (ii) transparency must be ensured; (iii) appeal opportunities must be organised; and (iv) these conditions must be regulated. However, there is little practical guidance as to how these conditions should be implemented.
EDPs respond to this and provide a practical stepwise approach for HTA bodies to implement the A4R conditions in their decision-making processes ( Figure 1). 12 As such, the use of EDPs, embodying fair processes, can improve the legitimacy of decision-making processes.

Elements of Evidence-Informed Deliberative Processes
An EDP integrates four elements ( Figure 1). First, the core element of EDPs is stakeholder involvement ideally operationalised through stakeholder participation with deliberation. Such stakeholder involvement ensures that all relevant values are considered, and is thereby a practical translation of the A4R relevance condition. By emphasizing the importance of stakeholder participation, EDPs go one step further than Daniels 41 who recommends to consult those who are affected by a decision. We argue that participation is a much more powerful approach to stakeholder involvement than consultation -this is discussed in more detail below. Second, evidence-informed evaluation which allows for the use of scientific evidence and contributions from stakeholders in terms of their experiences and judgments when further evidence is unavailable. This ensures that relevant evidence is considered and is also a practical translation of the A4R relevance condition. Third, transparency which ensures that the deliberative processes, including their objectives, modes of stakeholder involvement, the decision reached and its related argumentation, is explicitly described and made publicly available. Fourth, appeal which ensures that a decision can be challenged and revised if new information or insights become available. These elements are direct operationalisations of the related A4R conditions. As such, EDPs provide the best way to combine evidence, information, perspectives and values, while also allowing these aspects to be identified and openly discussed.

Practical Steps
We distinguish six practical steps of a HTA process based on existing HTA methods and tools, ( Figure 2) and provide recommendations on how elements of EDPs can be implemented in each of these steps. The latter is based on observed practices of HTA bodies around the world. We speak of 'HTA' when referring to the whole process, while 'hta' specifically refers to the evaluation of a single technology. In steps A-C, we offer advice on the installation of an advisory committee, including the organisation of stakeholder involvement; on how to define decision criteria, and on how to set up a process for identifying and selecting health technologies for hta. In steps D1-D3, we give advice on how to scope, assess and appraise a specific health technology.
In steps E-F, we provide guidance on communication and appeal, and monitoring and evaluation respectively. It is well recognised that, while the steps are presented as separate activities and in a linear fashion, in practice there may be iteration of them. 42 The development and organisation of the EDP practical steps is described in detail in the companion paper. 31 Overall, the practical recommendations on the implementation of EDPs in each step of the HTA process should not be considered as a blueprint, but rather as inspirational. Each country has a unique decision-making context determined by factors such as local values and traditions, level of HTA development, and available knowledge, skills and resources. From this follows that each country should introduce its own appropriate form of EDPs in its process, eg, mode of stakeholder involvement and transparency.
Organising Stakeholder Involvement Stakeholder involvement is formally defined as "an iterative

Legitimacy
Based on Accountability for Reasonableness framework

Elements of EDPs
Stakeholder participation, evidence-informed evaluation, transparency and appeal Practical steps Implementation of elements of EDPs in the steps of the decision-making process process of actively soliciting the knowledge, experience, judgment and values of individuals selected to represent a broad range of direct interest in a particular issue, for the dual purposes of creating a shared understanding; and making relevant, transparent, and effective decisions. " 43 It can improve the legitimacy of decision-making in three ways. 13 First, stakeholder involvement can serve to identify the full range of interests that a society has in relation to a particular decision. Second, it can improve understanding among stakeholders by explicating the interests of all parties involved. Third, stakeholder involvement can contribute to improving the quality of decisions, as stakeholders contribute specific knowledge (eg, about barriers to implementation or meaningful patient outcomes). Generally, three approaches to stakeholder involvement are distinguished: participation, consultation and communication, 44 and we discuss these below. Table  provides examples on how HTA bodies can implement these approaches in the different steps of the HTA process.

Stakeholder Participation
Stakeholder participation means that stakeholders are actively engaged in deliberations during each EDP step. A deliberative process is defined as "a series of coordinated activities allowing a group of people (or relevant stakeholders) to receive and exchange information, to critically examine an issue, and to come to an overall group judgement. " 45 Through this interaction and practical reasoning, stakeholders may deepen their understanding of their own preferences and those of others affected by decisions. They may replace uninformed opinions by views that are more rational and better supported by arguments and evidence, improving the quality of the decisions. There is good evidence that participants learn from deliberative engagement, including considering information that is contrary to their opinions and can change their opinions in line with this new information. 46 For these reasons, stakeholder participation can best realise the full benefits of stakeholder involvement and contribute to the legitimacy of decision-making. Accordingly, we strongly advise HTA bodies to consider implementing stakeholder participation in their decision-making process.
Stakeholder participation on coverage decisions can be organised through an advisory committee. Advisory committees are typically related to an HTA body and have the mandate to formulate recommendations to decisionmaking bodies, such as the Ministry of Health. Stakeholder participation in the advisory committee can be organised in two complementary ways.
• HTA bodies may choose to include specific stakeholders as formal members of their advisory committee. Such stakeholders typically represent the general interest of patients and sometimes industry, and not a specific interest regarding certain health technologies. These members have voting power. • HTA bodies can also organise stakeholder participation by inviting specific stakeholders to participate in their meetings. There stakeholders are not formal members of the advisory committee and are not granted voting power but can participate in deliberations. Such stakeholders typically represent interests or have specific expertise of the health technology being deliberated on. Nevertheless, this form of stakeholder participation does give some degree of influence and ownership of decisions to stakeholders. There are several approaches to structuring the decisionmaking process to facilitate stakeholder participation and deliberation in committee meetings. We advise using a systematic approach that ensures all participants and members express their preferences and considerations, such as Nominal Group Technique. Practical guidance on Nominal Group Technique is available elsewhere. 47 In addition, HTA bodies should take into account several general principles for stakeholder participation and deliberation. 48 These include: • Transparency: the deliberative processes -including the objectives, modes of stakeholder involvement, the decision reached and its related argumentation -should be explicitly described and made publicly available. • Inclusivity: all relevant values pertaining to decisions on a health technology should be taken into account. This requires that relevant stakeholders are meaningfully involved in the decision-making process. This means that barriers to effective participation should be removed. • Learning: stakeholders are ideally provided opportunities to participate and interact in deliberations as this likely improves the understanding of preferences, arguments and/or evidence, thereby improving the quality of the decision-making process. • Impartiality: the deliberative process used for each decision and those involved in it should be free from undue influences, both internal (eg, from the agency supporting the HTA process) and external (eg, from stakeholders with vested interests). The use of these principles in the organisation of stakeholder participation and deliberation is not without challenges. Ensuring transparency may be challenging in settings where there is little or no tradition of open decision-making. Ensuring inclusivity can be resource-intensive for HTA bodies who need to proactively identify affected stakeholders and involve them in decision-making processes and allocate adequately trained staff to the organisation of stakeholder participation. In addition, participating stakeholders may find it difficult to invest the time required to familiarise themselves with procedures of the HTA body. Ensuring learning takes place amongst stakeholders can be challenging for HTA bodies, as this requires multiple perspectives being effectively represented by participating stakeholders and that all perspectives are accounted for in the deliberations. Ensuring impartiality may also be challenging in contexts in institutions with less experience in managing conflicts of interest. Our guide provides practical guidance on how to deal with these challenges. 31 Stakeholder Consultation In many countries, meaningful stakeholder participation and actual deliberation between stakeholders is non-existent or in Step A Step B Step C Step D1 Step D2 Step D3 Step Publishing of appeals and relevant actions taken its infancy. Although not ideal, HTA bodies often rely on nondeliberative ways to involve stakeholder perspectives in their HTA processes such as through consultation. Consultation refers to a structured process for collecting feedback from groups of stakeholders on specific decisions without providing opportunities for meaningful deliberation with the HTA body's advisory committee. One example is the provision of oral or written patient testimonies to an advisory committee, as organised by the National Health Care Institute in the Netherlands, Canadian Agency for Drugs and Technologies in Health in Canada, the National Institute for Health and Care Excellence in the United Kingdom, and Pharmaceutical Benefits Advisory Committee in Australia. 49 Other examples include the use of surveys, stakeholder meetings and solicited feedback from stakeholders on HTA draft reports, including proposed recommendations. This latter approach is followed by the National Committee for Health Technology Incorporation in Brazil, and the Institute for Quality and Efficiency in Health Care in Germany among others. 49 The benefit of consultation is the large number of respondents that can be reached, however, there are some drawbacks. First, the timing and topic for input is predefined, which limits the scope of comments that stakeholders can provide. Second, consultation offers no opportunities for deliberation among stakeholders and the HTA body's advisory committee, which limits transparency and the facilitation of mutual learning. Third, the quantity of feedback can be significant, so time should be reserved for providing feedback. If the HTA body in question has limited capacity to do so, this can restrict the scope of public consultation and affect its ability to contribute to the overall legitimacy of the process.

Stakeholder Communication
Stakeholder communication refers to efforts by HTA bodies to inform stakeholders of their activities and results by using communication platforms. Communication can be achieved through public meetings, by preparing plain-language versions of reports to increase accessibility, by dissemination to high priority groups using patients' organisations or by using social media. Communication is characterised by a very low-level of stakeholder involvement. This one-way flow of information means there is little chance of stakeholders influencing the decision-making process. Note that stakeholder communication contributes to the requirement of transparency, the third element of legitimacy as defined in EDPs, and can thus be seen as a necessary activity of HTA bodies.
As an illustration, Box 1 describes the organisation of stakeholder involvement in the development of the UHC benefit package in Pakistan.

Conclusion
There is an increased attention for legitimacy, deliberative processes for HTA and health benefit package design, but the development of theories and methods for such processes remain behind. 10,26,50 This is in stark contrast with eg, the field of economic analysis of health technologies where the theory The project team designed an advisory committee structure including TWGs on four disease areas, a NAC, and a Steering Committee (step A). In this way, more than 100 disease experts and stakeholders from national, province and district level participated through deliberation in the committees. Next, the Ministry of Health conducted a survey among TWG and NAC members to assess the importance of decision criteria for the prioritisation of health technologies, and these members also participated in a meeting where decision criteria were communicated (step B). In addition, the Ministry of Health, together with key stakeholders, reviewed a DCP3 model benefit package on the technologies that should be assessed for inclusion in the UHC benefit package in Pakistan (step C). Subsequently, the project team collected evidence on these technologies. The TWGs then classified technologies into categories of 'high priority, ' 'medium priority' and 'low priority' using a deliberative process and majority voting as a closure mechanism. The meetings were led by a trained facilitator, and a rapporteur recorded the participants' arguments and votes. Next, the NAC reviewed TWG argumentation and recommendations and further prioritized the list of 'high priority' services taking into ac-count fiscal space, coverage and co-payment levels, and complementary investments in the health system (step D). The NAC developed recommendations for preferred packages, and these were presented to the Steering Committee for their approval upon consultation with an international advisory group.
Abbreviations: DCP3, Disease Control Priorities 3; UHC, universal health coverage; EDPs, evidence-informed deliberative processes; TWG, technical working group; NAC, National Advisory Committee. Box 1. Stakeholder Involvement in UHC Benefit Package Design in Pakistan of welfare economics and methods for cost-effectiveness analysis are well-developed. 26 This indicates the importance of this paper and related initiatives such as the 'Joint HTAi-ISPOR Task Force on Deliberative Processes for HTA. 51 Cuyler observes in his commentary on EDPs that the complexity and multifaceted nature of the topic makes it challenging to develop a "single unifying theory of deliberative processes. " 26 He argues that the design of deliberative processes also requires imagination and descriptive evidence. This is indeed reflected in our work: the conceptual framework of EDPs stems from the general concept of legitimacy, the definition of four elements is a practical translation of the A4R framework, 41 the definition of practical steps is based on existing HTA methods and tools, whereas related recommendations on best practices are inferred from observed practices of HTA bodies around the world.
We consider EDPs as complementary to other concepts and methods to support HTA bodies on health benefit package design such as reported in the publication What's in, what's out by the Center for Global Development, 3 the World Health Organization (WHO) Making Fair Choices report 52 and its handbook Strategizing national health in the 21st century. 53 Specific guidance on the use of HTA for benefit package design is provided in the HTA Toolkit by International Decision Support Initiative, 54 and the HTA roadmap by Management Sciences for Health. 55 All of these guides cover critical aspects of health benefit package design, including institutional set-up, required decision-making processes, the necessity of stakeholder involvement, collection of evidence and monitoring and evaluation aspects. The added value of EDPs lies in its explicit foundation in the concept of legitimacy and related elements (including stakeholder involvement ideally operationalised through stakeholder participation with deliberation), as well as in the provision of recommendations on how these elements can be implemented in each of the EDP steps.
The next step is to develop practical guidance for HTA bodies, on the basis of the conceptual framework as presented in this paper. We have made first efforts by analysing practices of eight HTA bodies around the world in terms of the six steps of EDPs and related elements of legitimacy, as described in the companion paper. 31

Ethical issues
Ethical approval was not necessary as no primary data was used.