Systems Thinking and Complexity Science Methods and the Policy Process in Non-communicable Disease Prevention: A Systematic Scoping Review

Background

The determinants of many non-communicable diseases (NCDs) are complex and inter-related, and policies designed to tackle them are made and unfold within dynamic realities. As such, the need for a ‘system-level’ approach to NCD prevention, which encompasses complex system properties, is increasingly recognised.¹ Systems thinking and complexity science (STCS) represents a multi-disciplinary field of established and emergent theories and methods² which may be applied to NCD prevention. While a differentiation has been drawn between STCS as distinct traditions,³ STCS theories and methods are broadly characterised by the idea that real-world phenomena exist within complex systems composed of dynamic actors, including people, organisations and other structures, which evolve in response to each other and their contexts.² These complex systems may have characteristics such as non-linear relationships, feedback loops and delays,^4-6 making their behaviour challenging to predict using more conventional methods for generating evidence.⁷

STCS theories and methods may lend themselves well to addressing complexity in policy-making for NCD prevention, an area characterised by a wide range of determinants, distributed responsibility for policy areas with direct and indirect impacts on NCD outcomes, and powerful non-governmental actors shaping environments that can support or undermine NCD prevention. In addition, many STCS methods are participatory in nature, facilitating stakeholder engagement and, in some cases, consensus-building,^8,9 both important attributes in policy-making. The potential of STCS methods for NCD prevention is evidenced by existing reports emphasising the importance of a systems perspective, such as the Lancet Report on the Global Syndemic of Obesity,¹⁰ the Foresight report¹¹ and the World Health Organization (WHO) Global Action Plan for Physical Activity,¹² which cites healthy systems as one of its explicit aims. Initiatives such as the Office of Behavioral and Social Sciences Research at the National Institutes of Health¹³ and the Australian Prevention Partnership Centre¹⁴ also highlight a commitment to a systems perspective in NCD prevention research and practice.

What Role Can Systems Thinking and Complexity Science Play in Public Health and Health Policy?

Methods informed by STCS have been applied in various contexts, and health researchers have explored their utility in solving seemingly intractable public health issues. These applications in public health are growing rapidly, with as many as 90% of published examples appearing in the past decade.¹⁵ Several reviews have documented existing approaches to applying STCS to methods and practice in public health and health policy, with most commenting on the relative paucity of studies documenting practical applications of such methods.^2,16-18

While examples of practical applications are limited, reviews have demonstrated the benefits of applying methods informed by STCS to health policy. A 2015 review on the application of system dynamics modelling to support health policy at any level of government reported that a key strength of the method was its ability to engage stakeholders and facilitate consensus-building.¹⁶ This was achieved by inviting their participation in developing a model, resulting in agreement over the optimal policy strategy to tackle a given health problem.¹⁶ A 2019 review of complex systems approaches to mental health commented on the potential applications of such methods to mental health policy. The review stated that they might be particularly useful in two processes: first, determining the potential impacts of ‘distal’ policies, where the policy was removed from its potential impacts in terms of time or causality; and second, assessing what conditions might be necessary for a policy to be successful.¹⁷

In addition to these advantages, STCS-informed methods have the potential to add robustness to evidence-informed policy-making. Despite the emphasis on evidence-based policy in public health,¹⁹ the role of research evidence in policy-making remains relatively limited,²⁰ with policy-makers often differing with researchers around what sort of evidence is ‘good’ and ‘useful.’^21,22 Further, evidence generated by researchers may only be incorporated at particular points in the policy process, with many parts of the policy process being a complex series of negotiations between different perspectives and interests. Given that policy-makers already operate in a complex and dynamic space, methods informed by STCS may support them in bringing greater rigour and transparency to the policy process, potentially leading to the implementation of more evidence-informed policy.

While many working in policy have expressed an interest in the insights yielded by STCS methods, a recent study of policy evaluators concluded that the methods were in limited use, and that the pragmatic framing of these methods should be seen as a priority to ensure their greater penetration into the policy process.²³ Evaluators described a number of reasons why they did not use STCS-informed methods, including the perception that simple policies and evaluations could be effective despite complex contexts; the idea that complexity was implicitly drawn out in policy and evaluation processes without the need for explicit methods; and the lack of relevant capacity and skills.²³

Aims and Scope

While there has been substantial discussion and theoretical development related to the application of STCS in the policy process,^24-26 well-documented examples of how STCS approaches can be applied to this arena, particularly at the national level and in a global context, are less common. A comprehensive and systematic review of the application of STCS-informed methods in the policy process is needed, with particular attention to specific methods used to support NCD prevention.

Further, a gap exists in determining which of these methods are useful to particular processes and practical for practitioners with different needs and levels of resource, and in making these distinctions accessible to potential users. Scholars of complex systems have previously emphasised the importance of increasing the use of methods informed by STCS in public policy processes, and the responsibility held by researchers to effectively translate their knowledge and methods to encourage their adoption in the policy process.^27,28 A review of existing practice which documents clear examples of how these methods can be applied in this context, as well as under what conditions a certain approach might be most useful, is an important part of this process of knowledge translation.

Therefore, the objective of this review was to systematically identify and summarise existing applications of STCS-informed methods in NCD prevention policy, documenting key methodological elements and identifying which domains of the policy process they have been applied to.

Methods

We conducted a systematic scoping review of peer-reviewed literature documenting the application of methods informed by STCS to the policy process in NCD prevention. This review was not registered but is based on a previously published protocol.²⁹ Results are reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-2020) statement.³⁰

The systematic scoping review was conducted according to guidelines published by Arksey and O’Malley and refined by Levac and colleagues,^31-33 which emphasise an iterative approach for exploratory research questions.³¹

Stage 1: Identifying the Research Questions

Informed by our study objective, our central research questions were:

1. How have methods informed by STCS been applied in the policy process in NCD prevention? Which domains of the policy process and areas of NCD prevention policy have methods informed by STCS been applied to?

2. What practical considerations, such as advantages, limitations, barriers and facilitators, have been described in applying STCS-informed methods to NCD prevention policy?

By policy we refer to public policy, defined as ‘a set of interrelated decisions taken by a political actor or group of actors concerning the selection of goals and the means of achieving them within a specified situation where those decisions should, in principle, be within the power of those actors to achieve.’³⁴ We understand policy as being ultimately in the hands of government, although we recognise that a number of limitations constrain the policy options available to government, including other domestic and international actors.^34,35 For the purposes of this review, we extended the definition of government to include regional (eg, continent-wide) and global governing bodies, as well as local and national government.

Following Howlett and Cashore’s conceptualizations of public policy, we characterised the policy process as one which moves from broader ‘goals’ to concrete ‘means’: specifically, on-the-ground policy measures designed to achieve the stated goals.³⁵ We used the definition of the domains of the policy process developed by the Centers for Disease Control and Prevention (CDC) (see Table 1).³⁶

Stage 2: Identifying Relevant Studies

We systematically searched electronic databases (Medline, Scopus, Web of Science, EMBASE). The search strategy was informed by the main concepts in our research question using the Population Concept Context Framework recommended by the Joanna Briggs Institute for use in scoping reviews^37,38:

• Population: Whole population approach to NCD prevention;

• Concept: Methods and approaches informed by STCS;

• Context: Policy-making and different domains of the policy process at different levels of government, including local, national and supranational.

We conducted searches on October 13, 2020, including studies published at any previous point in time. Search terms for the Concept included specific methods informed by STCS (eg, network analysis, causal loop diagram, group model building) as well as broader terms indicating a systems or complexity approach taken to different types of research (eg, complexity, complex system, adaptive system, system lens) (see Supplementary file 1 for detailed search).

Stage 3: Study Selection

We collated and screened records identified through the searches using the online platform Covidence.³⁹ Studies were included where they met all of the following criteria:

1. Primary study from any country or region, available in English;

2. Self-identify as taking an approach informed by STCS;

3. Report empirical findings from a piece of research done during a specific point in the policy process (ie, problem identification, policy analysis, strategy and policy development, policy enactment, policy implementation, evaluation, stakeholder engagement and education)³⁶; and

4. Focus on a subject related to NCD prevention.

Titles and abstracts were initially screened by one researcher (CCA) against these inclusion criteria. Full texts of potentially relevant records were then independently assessed against these criteria by two researchers (CCA, along with TP or EM). Discrepancies were resolved by discussion with reference to the inclusion criteria.

Stage 4: Charting the Data

We conducted data charting, a process which parallels data extraction in Arksey and O’Malley’s method for scoping reviews,³¹ using a data charting form designed to identify the information required to answer the research questions (see Supplementary file 2). As recommended, we piloted the data charting form with ten records to ensure that it was consistent with the research questions.^31-33 One researcher undertook data extraction (JC), and a second researcher checked the extracted data (CCA or KML). Discrepancies were resolved through discussion.

Stage 5: Collating, Summarising and Reporting the Results

We undertook quality assessment of the included studies using a novel quality assessment tool informed by the approach developed by Dixon-Woods and colleagues as part of their critical interpretive synthesis, where studies are excluded if they are identified as ‘fatally flawed’ in the first instance, and included studies are assessed based on their quality and relevance.⁴⁰ To develop this quality assessment tool, two researchers (CCA and KML) outlined criteria based on Dixon-Woods’ considerations of quality and relevance⁴⁰ (see Supplementary file 3 for full quality assessment tool). Both researchers conducted quality assessment on an initial set of ten papers. Ratings were discussed, allowing discrepancies to be resolved through discussion and criteria to be iteratively refined. This process was repeated four times, until reasonable inter-rater reliability was reached. In total, 40 papers (36% of included papers) were assessed by both researchers, with 98% concordance reached at the final round. These papers were then revisited by both researchers to ensure that rating was consistent. The remaining papers were assessed by either CCA or KML.

We used this approach to assess the diverse body of literature included in this review, including both qualitative and quantitative methods, many different study designs, as well as studies authored by researchers working in different disciplines with different approaches to reporting. In order to be inclusive of this range of literature, we only excluded papers as being ‘fatally flawed’ if their relevance was low, meaning that they did not meet the inclusion criteria.

We analysed the charted data, presented a descriptive summary of the included studies in table form and identified intersections between STCS methods and domains of the policy process. We also conducted a thematic analysis of a subset of the included articles in order to identify what needs these methods have met and the resources they require, and what challenges were encountered in applying the methods. A random sample of the included articles was coded until new themes ceased to be identified. A purposive sample of additional articles was then coded, focusing on articles rated as having a ‘high’ relevance, as well as articles identified through the data charting process as having characteristics that were uncommon within the sample. Thematic analysis was conducted using the approach described by Braun and Clarke where the focus is guided by the researcher’s analytic interests,⁴¹ with four overarching themes chosen as an a priori coding framework: (1) value added or rationale for using STCS; (2) limitations of STCS methods; (3) challenges or barriers to using an STCS approach; and (4) characteristics that enabled or facilitated the use of an STCS approach. These four themes were chosen in light of our research questions and previous familiarisation with the included articles during the process of article selection, data extraction and quality assessment. Sub-themes were subsequently identified through close reading and coding of the included articles. Thematic analysis was coded by one researcher (CCA) using the qualitative data analysis software Dedoose⁴² and discussed with co-authors (KML and TLP).

Results

After removing duplicates, our searches identified a total of 4681 records. After screening titles and abstracts, we considered 298 records for full-text review. In total, we included 112 papers in this review (Figure 1).³⁰ Papers which concern potential risk factors for NCDs but focus on non-NCD outcomes (such as alcohol consumption as a risk factor for road traffic accidents or inter-personal violence) were excluded.

Close

Figure 1. PRISMA 2020 Flow Diagram. Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Use of STCS Methods in NCD Prevention Policy

The characteristics of the included studies are presented in Table 2. Studies focused on a number of different risk factors or health outcomes within NCD prevention, with the most common being tobacco (n = 18), obesity (n = 17) and physical activity (n = 12). Most studies were conducted in high-income countries, with a substantial number from the United States (n = 39), Australia (n = 20), Canada (n = 11), and New Zealand (n = 8). In addition, most studies had been published relatively recently, with 54% (n = 61) published after 2015 and 84% (n = 100) published after 2010 (Figure 2).

Close

Figure 2. Papers Included in Review by Publication Year.

Table 3 summarises the key STCS methods identified in the review.

We identified recurring intersections between methods and policy process domains (Figure 3). System dynamics modelling and agent-based modelling, both methods typically used for quantitative modelling, were used more frequently for policy analysis. Network analysis, mainly focusing on relationships between individuals and organisations, was predominantly used for policy development. Concept mapping and qualitative research with a ‘systems lens’ were used consistently across the policy process. Meanwhile, group model building was used consistently for all domains of the policy process except implementation, while causal loop diagrams were used for problem identification and policy development.

Close

Figure 3. STCS Methods Used Across the Policy Process, Weighted by Number of Papers. Abbreviation: STCS, Systems thinking and complexity science.

Studies focused on policy-making at different levels. Most studies were conducted in a local government context (n = 62) or at the national level (n = 35). A smaller number examined supra-national policy processes (n = 7). These studies included analyses of the role of international networks in European smoke-free policy^135,136 and ‘policy diffusion’ across countries in the context of country-level implementation of the global Framework Convention on Tobacco Control.¹⁵⁰ The remaining studies considered the policy process at multiple levels (n = 8), for example developing models,⁴⁷ tools¹⁴⁰ or frameworks¹⁴⁴ that could be applied to inform or support policy-making in different contexts.

Most of the included studies relied on stakeholder input (n = 87), including survey questionnaires, semi-structured interviews and participatory workshops. These stakeholders included policy-makers from several sectors, including health, transport, planning and education, as well as non-policy stakeholders such as community members, clinical practitioners, industry representatives and academics. The remaining studies used existing data sets or published literature in their analyses.

Quality Assessment

The quality of included studies was high overall (see Supplementary file 4). Most studies were rated as high in terms of clearly stating their aims (n = 97), clearly identifying and justifying their research design (n = 102), and presenting their results clearly and comprehensively (n = 109). High ratings were slightly less frequent in terms of the reporting of the research process (n = 89) and analysis (n = 91). In terms of relevance, the studies were mixed, with just under half of studies having a high relevance (n = 50), indicating that the research was embedded in the policy process (commissioned, co-designed or conducted in partnership with government, or involving participatory workshops with stakeholders, see Supplementary file 3). The remaining papers (n = 62) were rated medium for relevance, meaning the studies were focused on understanding the policy process from an outside perspective.

Added Value of Using STCS Methods

Authors cited a number of advantages to applying STCS methods to the policy process. Some advantages were mentioned across studies using different methods in different contexts. The participatory process inherent in some STCS methods was seen as a key feature, providing opportunities for building consensus among stakeholders from different sectors^59,146 incorporating multiple perspectives when making decisions or trying to understand a problem; ^{55,87,104,118}; and building capacity in terms of systems understanding among the policy or community stakeholders who participate in the research, for example through methods such as community-based system dynamics modelling.⁶¹ Other frequently noted advantages of STCS methods included facilitating evidence-informed decision-making;^45,92,152 identifying ‘leverage points,’ or points within a system where interventions might be expected to be most impactful;^106,144,153 integrating different forms of evidence;^{57,63,101,145} and tackling ‘wicked’ problems, where little progress had been made using other methods despite concerted efforts.^{87,106,111,154}

Other advantages were recognised as being specific to individual STCS methods. For example, the benefits of concept mapping included transparency of process and interpretability of results, making the method useful for engaging stakeholders with different areas of expertise.^59,146 In turn, system dynamics modelling was seen as useful for modelling impacts on multiple outcomes^91,92; surfacing mental models of systems⁷²; and providing ‘dynamic decision support’ for policy decisions, offering a ‘what if’ tool to simulate different policies and explore their costs and benefits.¹⁰¹

Finally, some advantages were frequently cited in studies focusing on specific domains of the policy process. In problem identification, involving clarifying and framing an issue in terms of its effect on population health, the suitability of STCS methods to modelling complex systems was noted.^45,47,57,58 Systems that drive NCD-related practices such as smoking, physical activity and diet were understood as being complex and sometimes adaptive, being characterised by properties such as feedback loops and emergent behaviours. STCS methods are appropriate for modelling these types of systems and understanding the different drivers and outcomes of these practices.

In policy analysis, which involves identifying and evaluating different policy options, STCS methods were useful in terms of enabling comparison of these different options through different types of modelling^{81,92,155,156}; identifying potential unintended consequences^76,80,155; and modelling impacts over different time scales.^63,80,81,93 In addition, modelling different policy options was seen as a resource-efficient way to obtain information about their potential impacts, compared to implementing and evaluating policies.^63,81,155

In policy development, involving identifying a strategy for getting policies adopted, many authors used network analyses to understand how individuals and organisations relate to each other, influence each other, collaborate and share knowledge. In this context, STCS methods were useful in addressing multi-sectoral problems,^{104,118,119,153} and identifying disconnects between sectors or boundary-spanning organisations and individuals with the potential to connect disparate stakeholders.^88,119

Finally, in policy implementation, and particularly in evaluating policies that had been implemented, the potential of STCS methods to facilitate the incorporation of multiple perspectives was noted.^140,142 This provided a more comprehensive understanding of how a policy operated within a complex system.

Limitations of STCS Methods

Discussion of the limitations of STCS methods emphasised that, while these methods aim to incorporate more complexity than conventional methods, they still represent simplified versions of reality, and this must be acknowledged in interpreting findings.^64,75 For example, the authors of a modelling study acknowledged model assumptions around potential policies having the same level of impact as reported in the literature.⁸⁵ This ignored possible variation in impact caused by inadequate funding to implement policies and different approaches to monitoring and enforcement.⁸⁵

Authors also commented on the limitations of participatory methods: where different stakeholders were invited to jointly participate in workshops, their presence could have a silencing effect on other participants, for example senior and junior staff of an organisation.¹⁴⁸ While some authors solved this issue by keeping these stakeholder groups separate,¹³⁰ this may result in some of the advantages of a participatory method such as consensus- and capacity-building being lost.

Challenges or Barriers to Using an STCS Approach

Authors also encountered challenges while applying STCS-informed methods to NCD prevention policy. Along with challenges encountered in applying research methods of any kind, such as a lack of appropriate data^{47,63,76,81,93} and high requirements in terms of technical skill,^81,93,142 studies also noted particular challenges in applying STCS methods, mainly relating to engaging stakeholders in the research process.

First, although the participatory processes involved in some STCS methods were noted as a key source of added value, implementing these processes was also challenging. Challenges included the resource-intensiveness of participatory processes, with both time required of participants and travel costs being high^{51,89,118,142}; maintaining communication and engagement with participants throughout a sometimes lengthy research process^87,101; and different expectations and priorities among participants.^101,142

Second, researchers also noted a lack of trust in models as a source of evidence, with modelling evidence not always seen as strong enough to act on.^72,101 To some extent, participatory modelling was seen as a way to add credibility to models, as stakeholders were embedded in the modelling process and developed a better understanding of how models incorporate and generate evidence.

Third, authors also noted that in some cases both processes and results were too complex to engage some stakeholder groups, and stakeholders struggled to interpret them.^47,76,81,140 Some authors noted a tension around model complexity, wanting models to be both complex enough to adequately represent the system in question, and simple enough to lend themselves more readily to interpretation.^47,76,81

Characteristics That Enabled or Facilitated the Use of an STCS Approach

While authors noted a number of challenges which were encountered in applying STCS methods to the policy process, they also provided many examples of how to make these methods work in practice.

First, many studies adapted methods to make them less resource-intensive, particularly in terms of demanding less of their participants. For example, a number of studies used online platforms to engage participants and collect data, allowing geographically dispersed participants to participate without the need to travel.⁸⁹ Some studies also enabled participants to contribute asynchronously, for example through qualitative interviews, which were then integrated with other perspectives by the research team.⁴⁸

Second, authors noted the importance of presenting processes and results in an accessible way, allowing different groups of stakeholders to engage with and learn from them. For example, one study reporting system dynamics modelling of diabetes in pregnancy described the use of individual ‘life stories’ as a means of communicating model structure and results: hypothetical patient journeys were described in light of the model, with patients being born with particular risk profiles, ageing over the course of their lives, gaining and losing weight, undergoing clinical procedures or changes in health-related practices, and so on.⁷² This helped make the modelling process and findings relevant to different stakeholder groups. Other approaches included the use of ‘rich pictures’ to communicate complex findings,¹⁴² or interactive user interfaces allowing stakeholders to model different policy interventions for themselves.^{69,73,77,93,101} Authors also cited progress in modelling software as an enabling factor, making it more user-friendly for different stakeholders.^63,71

Third, authors emphasised the importance of fostering stakeholder ownership of the research process.^72,101 This involved ensuring that stakeholders identified, or at least acknowledged, the problem under consideration as important, as well as in some cases being an issue that ‘conventional’ approaches had failed to address. Another key aspect was recruiting participants who had credibility, which facilitated buy-in from stakeholders who were not involved in the research process, as participants acted as ‘ambassadors’ for the process. Stakeholder ownership could also be encouraged through the facilitation process, for example by highlighting participants’ contributions to different parts of the model, allowing them to see how their input was incorporated.

A number of other characteristics that enabled these methods to be put to use, including ensuring expert facilitation of any workshops^72,101; transparency and acknowledgement around the limitations of STCS methods,^72,101 with authors acknowledging that, even though STCS methods seek to incorporate a greater degree of complexity, complexity is still lost in the modelling process⁷⁵; and insider access to relevant stakeholder groups.¹¹¹

Discussion

Implications for Research and Practice

This study presents a comprehensive review of how methods informed by STCS have been applied in the policy process in NCD prevention. The findings can inform researchers and policy-makers around which methods are most frequently applied to both facilitate and understand different domains of the policy process (Figure 3), as well as typical applications of these methods in NCD prevention policy (Table 3). This overview of current practice may be helpful when selecting the most appropriate STCS method to address a particular research objective. The study also sheds light on the practical dimensions of applying these methods, including considerations of the added value of STCS methods in the policy process; the limitations of these methods; the challenges that have been encountered in applying these methods; and the circumstances that have facilitated their application. A particular finding for practice is the importance of stakeholder engagement when using STCS methods in policy-making. As several of the methods identified in this review involve some form of system modelling, particularly in comparing scenarios at the policy analysis stage, authors highlight stakeholders’ reluctance to use models and their evidence to support policy decisions. However, stakeholder engagement in model conceptualisation and development emerges as key to building trust in models.

Comparison With Existing Literature

To our knowledge, this is the first systematic scoping review identifying and analysing the range of STCS-informed methods applied in the policy process for NCD prevention. In line with previous work, we found that STCS-informed methods were valued as a way of building consensus between different stakeholders in the policy context,¹⁶ and changing policy-makers’ perspectives on their work.²⁴ Previous work also identified a lack of trust in models as a form of evidence as a barrier to use.¹⁶ In turn, our findings present approaches to building trust, providing examples of how processes and results can be communicated in user-friendly ways and emphasising the importance of stakeholder ownership. This latter finding echoes Rouwette’s review of group model building case studies in a wide range of settings, which emphasises ownership of the model and trust in the modeller as an important outcome of the group model building process.⁹

In contrast to existing reviews around system dynamics modelling in health policy¹⁶ and systems thinking in public health more broadly,^2,18 which noted a paucity of practical applications and the need for additional applied work, we identified a substantial number of examples, reflecting our broader scope in terms of methods as well as the increasing emphasis on these methods in the literature in recent years (see Figure 2). The numerous applied examples identified in this review supported us in developing insights around the added value and limitations of, as well as barriers and facilitators to, using STCS methods in NCD prevention policy. However, in common with previous work on the topic,^16,18 we identified a lack of evaluations of effectiveness of policy decisions made as a result of STCS-informed methods. While previous reviews highlighted that system dynamics modelling studies often claimed a greater validity due to being based on systems principles, without providing evidence of this validity in the form of, for example, greater predictive power than conventional models,^16,18 we found this pattern repeated across the range of STCS methods included in this review. While researchers and funders may hesitate to use both conventional and STCS methods to answer the same question, particularly due to the often larger resource requirements of STCS studies, such comparative work would help clarify where STCS methods are useful and where they are not. This remains an important gap in the evidence base.

Strengths and Limitations of the Review

This review could only identify examples of methods which have previously been applied in the policy domain, and where this application was documented. We hope this will increase the value of our findings for practitioners, but as a result, methods that have not been applied, or only applied in other fields, could not be identified in this review.

Further, in an attempt to include the wide range of STCS methods in use, we searched for and included studies using specific methods previously identified as being informed by STCS (eg, concept mapping, system dynamics modelling, network analysis),^16,17,157 as well as studies whose authors characterised their work as being informed by STCS. As a result, studies that neither employed methods identified by existing literature as being informed by STCS, nor ‘self-defined’ as using methods informed by STCS, were not included. Anzola and colleagues highlight the existence of ‘analogical’ uses of terms relating to complexity, where central characteristics of STCS are employed or implicitly referred to without being explicitly linked to the relevant theory and methods.¹⁵⁸ In the absence of shared terminology, such usage may be difficult to systematically identify in the literature. However, given that the aim of this review was to identify the applications of specific methods, we aimed to be inclusive of relevant theories and methods. In addition, the inclusion of additional terms relating to NCD prevention, such as ‘population health,’ may have yielded additional relevant examples.

Finally, this review focused on peer-reviewed literature in order to identify ‘best practice,’ methodologically robust descriptions of the specific and distinct methods that are in use. As a result, applications of methods informed by STCS which are documented in the grey literature were not identified, potentially excluding methods used in non-academic settings, such as government institutions, non-government organisations, and public health practice.

In conducting this review, we developed a quality assessment tool informed by an existing approach to assessing and synthesising complex, multi-disciplinary bodies of literature. Both authors who conducted quality assessment have backgrounds in public health and epidemiology, and may have been biased towards their own disciplinary standards of reporting. As a result, some studies, while presenting interesting ideas and arguments, ranked low in terms of quality. In order not to exclude papers due to disciplinary differences, all papers were included other than those that ranked low in terms of relevance to our research questions, meaning they did not meet our inclusion criteria. The ‘relevance’ criterion was used to inform our analysis of included papers, as described in the methods. This is an initial attempt at quality assessment in this body of literature, but future research may refine this process further by considering disciplinary differences in reporting.

Unanswered Questions and Future Research

This study identifies some of the ways in which, from the perspective of study authors, methods informed by STCS add value throughout the policy process. However, more formal analysis of the added value of these methods could be an important next step in determining their usefulness. While the effectiveness of some systems methods has been reviewed, review authors cite a lack of uniformity in approaches to determining ‘effectiveness.’⁹ Contributing to our understanding of effectiveness would require some consideration of what stakeholders expect of these methods. For example, the focus might be on outcomes, considering whether policies incorporating STCS into their design and implementation might be more effective in achieving desired outcomes. On the other hand, the impact on the policy process itself might be deemed important, with participatory STCS methods perhaps being more inclusive of multiple perspectives.

Our findings also highlight contexts in which methods informed by STCS have seen more limited application in policy-making for NCD prevention, particularly in low- and middle-income countries and in policy-making at the supra-national level. Future research could seek to apply these methods in these contexts and document the process, identifying whether sources of value added, challenges encountered or facilitating characteristics are similar or different.

Conclusion

This systematic scoping review aimed to identify existing applications of STCS methods to the policy process in NCD prevention. A substantial number of studies were identified, exhibiting diversity in terms of method, level of government, area of NCD prevention, and policy process domain. The findings of this review can inform researchers and policy-makers around which methods are most frequently applied to both facilitate and understand different domains of the policy process, as well as providing practical insights around their application. Future research in this area could focus on applying these methods in policy-making in low- and middle-income countries and at the supra-national level, as well as generating additional evidence around the added value of using these methods in NCD prevention policy.

Ethical issues

This study involved a synthesis of existing peer-reviewed literature and no primary data were collected. Ethical approval was therefore not required.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

Conception and design: TLP, CCA, EM, and ME. Acquisition of data: CCA, TLP, and EM. Analysis and interpretation of data: CCA, KML, and JC. Drafting of the manuscript: CCA. Critical revision of the manuscript for important intellectual content: KML, EM, ME, RM, HR, AH, KW, and TLP. Obtaining funding: TLP.

Funding

CCA, TLP, and JC acknowledge internal research support from York University, Toronto, Canada. CCA and TLP acknowledge funding from the WHO European Office for Prevention and Control of Noncommunicable Diseases. EM and ME were supported by the National Institute for Health Research (NIHR) School for Public Health Research (SPHR), Grant Reference Number PD-SPH-2015.

Supplementary files

References

1. Lich KH, Ginexi EM, Osgood ND, Mabry PL. A call to address complexity in prevention science research. Prev Sci 2013; 14(3):279-289. doi: 10.1007/s11121-012-0285-2 [Crossref] [ Google Scholar]
2. Rusoja E, Haynie D, Sievers J. Thinking about complexity in health: a systematic review of the key systems thinking and complexity ideas in health. J Eval Clin Pract 2018; 24(3):600-606. doi: 10.1111/jep.12856 [Crossref] [ Google Scholar]
3. Gates EF. Making sense of the emerging conversation in evaluation about systems thinking and complexity science. Eval Program Plann 2016; 59:62-73. doi: 10.1016/j.evalprogplan.2016.08.004 [Crossref] [ Google Scholar]
4. Meadows DH. Thinking in Systems: A Primer. Chelsea Green Publishing; 2008.
5. Sterman JD. Learning from evidence in a complex world. Am J Public Health 2006; 96(3):505-514. doi: 10.2105/ajph.2005.066043 [Crossref] [ Google Scholar]
6. de Savigny D, Adam T. Systems Thinking for Health Systems Strengthening. Alliance for Health Policy and Systems Research, World Health Organization; 2009.
7. Rutter H, Savona N, Glonti K. The need for a complex systems model of evidence for public health. Lancet 2017; 390(10112):2602-2604. doi: 10.1016/s0140-6736(17)31267-9 [Crossref] [ Google Scholar]
8. Vennix JA, Akkermans HA, Rouwette EA. Group model-building to facilitate organizational change: an exploratory study. Syst Dyn Rev 1996; 12(1):39-58. doi: 10.1002/(sici)1099-1727(199621)12:1<39::aidsdr94>3.0.co;2-k [Crossref] [ Google Scholar]
9. Rouwette EA, Vennix JA, van Mullekom T. Group model building effectiveness: a review of assessment studies. Syst Dyn Rev 2002; 18(1):5-45. doi: 10.1002/sdr.229 [Crossref] [ Google Scholar]
10. Swinburn BA, Kraak VI, Allender S. The global syndemic of obesity, undernutrition, and climate change: the Lancet Commission report. Lancet 2019; 393(10173):791-846. doi: 10.1016/s0140-6736(18)32822-8 [Crossref] [ Google Scholar]
11. Butland B, Jebb S, Kopelman P, et al. Tackling Obesities: Future Choices-Project Report. 2nd ed. London: Department of Innovation, Universities and Skills; 2007:164. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/287937/07-1184x-tackling-obesities-future-choices-report.pdf.
12. World Health Organization. Global Action Plan on Physical Activity 2018-2030: More Active People for a Healthier World. WHO; 2018:104. https://apps.who.int/iris/bitstream/handle/10665/272722/9789241514187-eng.pdf.
13. Office of Behavioral and Social Sciences Research. https://obssr.od.nih.gov/.
14. Wutzke S, Rowbotham S, Haynes A. Knowledge mobilisation for chronic disease prevention: the case of the Australian Prevention Partnership Centre. Health Res Policy Syst 2018; 16(1):109. doi: 10.1186/s12961-018-0379-9 [Crossref] [ Google Scholar]
15. Atkinson JA, Page A, Prodan A, McDonnell G, Osgood N. Systems modelling tools to support policy and planning. Lancet 2018; 391(10126):1158-1159. doi: 10.1016/s0140-6736(18)30302-7 [Crossref] [ Google Scholar]
16. Atkinson JA, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Health Res Pract 2015; 25(3):e2531531. doi: 10.17061/phrp2531531 [Crossref] [ Google Scholar]
17. Langellier BA, Yang Y, Purtle J, Nelson KL, Stankov I, Diez Roux AV. Complex systems approaches to understand drivers of mental health and inform mental health policy: a systematic review. Adm Policy Ment Health 2019; 46(2):128-144. doi: 10.1007/s10488-018-0887-5 [Crossref] [ Google Scholar]
18. Carey G, Malbon E, Carey N, Joyce A, Crammond B, Carey A. Systems science and systems thinking for public health: a systematic review of the field. BMJ Open 2015; 5(12):e009002. doi: 10.1136/bmjopen-2015-009002 [Crossref] [ Google Scholar]
19. World Health Organization (WHO). World Health Assembly Resolution WHA51. 12 Health Promotion. WHO; 1998. https://www.who.int/healthpromotion/wha51-12/en/.
20. Bambra C. The primacy of politics: the rise and fall of evidence-based public health policy?. J Public Health (Oxf) 2013; 35(4):486-487. doi: 10.1093/pubmed/fdt113 [Crossref] [ Google Scholar]
21. Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health 2004; 58(10):811-816. doi: 10.1136/jech.2003.015289 [Crossref] [ Google Scholar]
22. McGill E, Egan M, Petticrew M. Trading quality for relevance: non-health decision-makers’ use of evidence on the social determinants of health. BMJ Open 2015; 5(4):e007053. doi: 10.1136/bmjopen-2014-007053 [Crossref] [ Google Scholar]
23. Barbrook-Johnson P, Proctor A, Giorgi S, Phillipson J. How do policy evaluators understand complexity?. Evaluation 2020; 26(3):315-332. doi: 10.1177/1356389020930053 [Crossref] [ Google Scholar]
24. Haynes A, Garvey K, Davidson S, Milat A. What can policy-makers get out of systems thinking? Policy partners’ experiences of a systems-focused research collaboration in preventive health. Int J Health Policy Manag 2020; 9(2):65-76. doi: 10.15171/ijhpm.2019.86 [Crossref] [ Google Scholar]
25. Knai C, Petticrew M, Mays N. Systems thinking as a framework for analyzing commercial determinants of health. Milbank Q 2018; 96(3):472-498. doi: 10.1111/1468-0009.12339 [Crossref] [ Google Scholar]
26. Canty-Waldron J. Using systems thinking to create more impactful social policy. J Futures Stud 2014; 19(2):61-86. [ Google Scholar]
27. Forrester JW. System dynamics—the next fifty years. Syst Dyn Rev 2007; 23(2-3):359-370. doi: 10.1002/sdr.381 [Crossref] [ Google Scholar]
28. Ghaffarzadegan N, Lyneis J, Richardson GP. How small system dynamics models can help the public policy process. Syst Dyn Rev 2011; 27(1):22-44. doi: 10.1002/sdr.442 [Crossref] [ Google Scholar]
29. Clifford Astbury C, McGill E, Egan M, Penney TL. Systems thinking and complexity science methods and the policy process in non-communicable disease prevention: a systematic scoping review protocol. BMJ Open 2021; 11(9):e049878. doi: 10.1136/bmjopen-2021-049878 [Crossref] [ Google Scholar]
30. Page MJ, McKenzie JE, Bossuyt PM. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372:n71. doi: 10.1136/bmj.n71 [Crossref] [ Google Scholar]
31. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8(1):19-32. doi: 10.1080/1364557032000119616 [Crossref] [ Google Scholar]
32. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010; 5:69. doi: 10.1186/1748-5908-5-69 [Crossref] [ Google Scholar]
33. Colquhoun HL, Levac D, O’Brien KK. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014; 67(12):1291-1294. doi: 10.1016/j.jclinepi.2014.03.013 [Crossref] [ Google Scholar]
34. Jenkins WI. Policy Analysis: A Political and Organisational Perspective. Wiley-Blackwell; 1978.
35. Howlett M, Cashore B. Conceptualizing public policy. In: Engeli I, Allison CR, eds. Comparative Policy Studies: Conceptual and Methodological Challenges. London: Palgrave Macmillan; 2014:17-33. 10.1057/9781137314154_2.
36. Centers for Disease Control and Prevention. Overview of CDC’s Policy Process. 2012. https://www.cdc.gov/policy/analysis/process/docs/CDCPolicyProcess.pdf.
37. Aromataris E, Munn Z. JBI Manual for Evidence Synthesis. The Joanna Briggs Institute; 2020. 10.46658/jbimes-20-01.
38. Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. JBI Evid Implement 2015; 13(3):141-146. doi: 10.1097/xeb.0000000000000050 [Crossref] [ Google Scholar]
39. Covidence - Better Systematic Review Management. https://www.covidence.org/home.
40. Dixon-Woods M, Cavers D, Agarwal S. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 2006; 6:35. doi: 10.1186/1471-2288-6-35 [Crossref] [ Google Scholar]
41. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3(2):77-101. doi: 10.1191/1478088706qp063oa [Crossref] [ Google Scholar]
42. Dedoose Version 8.3.47, web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, LLC; 2021.
43. Willis C, Kernoghan A, Riley B, Popp J, Best A, Milward HB. Outcomes of interorganizational networks in Canada for chronic disease prevention: insights from a concept mapping study, 2015. Prev Chronic Dis 2015; 12:E199. doi: 10.5888/pcd12.150297 [Crossref] [ Google Scholar]
44. Stankov I, Howard NJ, Daniel M, Cargo M. Policy, research and residents’ perspectives on built environments implicated in heart disease: a concept mapping approach. Int J Environ Res Public Health 2017; 14(2):170. doi: 10.3390/ijerph14020170 [Crossref] [ Google Scholar]
45. Crespo R, Alvarez C, Hernandez I, García C. A spatially explicit analysis of chronic diseases in small areas: a case study of diabetes in Santiago, Chile. Int J Health Geogr 2020; 19(1):24. doi: 10.1186/s12942-020-00217-1 [Crossref] [ Google Scholar]
46. Giles BG, Findlay CS, Haas G, LaFrance B, Laughing W, Pembleton S. Integrating conventional science and aboriginal perspectives on diabetes using fuzzy cognitive maps. Soc Sci Med 2007; 64(3):562-576. doi: 10.1016/j.socscimed.2006.09.007 [Crossref] [ Google Scholar]
47. Auchincloss AH, Riolo RL, Brown DG, Cook J, Diez Roux AV. An agent-based model of income inequalities in diet in the context of residential segregation. Am J Prev Med 2011; 40(3):303-311. doi: 10.1016/j.amepre.2010.10.033 [Crossref] [ Google Scholar]
48. Gerritsen S, Harré S, Swinburn B. Systemic barriers and equitable interventions to improve vegetable and fruit intake in children: interviews with national food system actors. Int J Environ Res Public Health 2019; 16(8):1387. doi: 10.3390/ijerph16081387 [Crossref] [ Google Scholar]
49. Guariguata L, Rouwette EA, Murphy MM. Using group model building to describe the system driving unhealthy eating and identify intervention points: a participatory, stakeholder engagement approach in the Caribbean. Nutrients 2020; 12(2):384. doi: 10.3390/nu12020384 [Crossref] [ Google Scholar]
50. Mazzocchi G, Marino D. Rome, a policy without politics: the participatory process for a metropolitan scale food policy. Int J Environ Res Public Health 2020; 17(2):479. doi: 10.3390/ijerph17020479 [Crossref] [ Google Scholar]
51. Baugh Littlejohns L, Baum F, Lawless A, Freeman T. The value of a causal loop diagram in exploring the complex interplay of factors that influence health promotion in a multisectoral health system in Australia. Health Res Policy Syst 2018; 16(1):126. doi: 10.1186/s12961-018-0394-x [Crossref] [ Google Scholar]
52. Witter S, Zou G, Diaconu K. Opportunities and challenges for delivering non-communicable disease management and services in fragile and post-conflict settings: perceptions of policy-makers and health providers in Sierra Leone. Confl Health 2020; 14:3. doi: 10.1186/s13031-019-0248-3 [Crossref] [ Google Scholar]
53. Roblin L, Truscott R, Boddy MR. The development of a provincial food and nutrition strategy through cross-sector collaboration. Can J Diet Pract Res 2018; 79(1):28-34. doi: 10.3148/cjdpr-2017-033 [Crossref] [ Google Scholar]
54. Nelson DA, Simenz CJ, OʼConnor SP. Using group model building to understand factors that influence childhood obesity in an urban environment. J Public Health ManagPract 2015; 21 Suppl 3:S74-78. doi: 10.1097/phh.0000000000000219 [Crossref] [ Google Scholar]
55. Nau T, Lee K, Smith BJ. Toward whole-of-system action to promote physical activity: a cross-sectoral analysis of physical activity policy in Australia. J Phys Act Health 2019; 16(11):1029-1038. doi: 10.1123/jpah.2019-0122 [Crossref] [ Google Scholar]
56. Malhi L, Karanfil O, Merth T, Acheson M, Palmer A, Finegood DT. Places to intervene to make complex food systems more healthy, green, fair, and affordable. J Hunger Environ Nutr 2009; 4(3-4):466-476. doi: 10.1080/19320240903346448 [Crossref] [ Google Scholar]
57. Buck C, Loyen A, Foraita R. Factors influencing sedentary behaviour: a system based analysis using Bayesian networks within DEDIPAC. PLoS One 2019; 14(1):e0211546. doi: 10.1371/journal.pone.0211546 [Crossref] [ Google Scholar]
58. Chao D, Hashimoto H, Kondo N. Dynamic impact of social stratification and social influence on smoking prevalence by gender: an agent-based model. Soc Sci Med 2015; 147:280-287. doi: 10.1016/j.socscimed.2015.08.041 [Crossref] [ Google Scholar]
59. Stillman F, Hoang M, Linton R, Ritthiphakdee B, Trochim W. Mapping tobacco industry strategies in South East Asia for action planning and surveillance. Tob Control 2008; 17(1):e1. doi: 10.1136/tc.2006.017988 [Crossref] [ Google Scholar]
60. Tan DT, Siri JG, Gong Y. Systems approaches for localising the SDGs: co-production of place-based case studies. Global Health 2019; 15(1):85. doi: 10.1186/s12992-019-0527-1 [Crossref] [ Google Scholar]
61. Langellier BA, Kuhlberg JA, Ballard EA. Using community-based system dynamics modeling to understand the complex systems that influence health in cities: the SALURBAL study. Health Place 2019; 60:102215. doi: 10.1016/j.healthplace.2019.102215 [Crossref] [ Google Scholar]
62. Yang Y, Diez-Roux AV. Using an agent-based model to simulate children’s active travel to school. Int J BehavNutr Phys Act 2013; 10:67. doi: 10.1186/1479-5868-10-67 [Crossref] [ Google Scholar]
63. Atkinson JA, Knowles D, Wiggers J. Harnessing advances in computer simulation to inform policy and planning to reduce alcohol-related harms. Int J Public Health 2018; 63(4):537-546. doi: 10.1007/s00038-017-1041-y [Crossref] [ Google Scholar]
64. Castillo-Carniglia A, Pear VA, Tracy M, Keyes KM, Cerdá M. Limiting alcohol outlet density to prevent alcohol use and violence: estimating policy interventions through agent-based modeling. Am J Epidemiol 2019; 188(4):694-702. doi: 10.1093/aje/kwy289 [Crossref] [ Google Scholar]
65. Holder HD, Blose JO. Reduction of community alcohol problems: computer simulation experiments in three counties. J Stud Alcohol 1987; 48(2):124-135. doi: 10.15288/jsa.1987.48.124 [Crossref] [ Google Scholar]
66. Kang H, Nembhard HB, Ghahramani N, Curry W. A system dynamics approach to planning and evaluating interventions for chronic disease management. J Oper Res Soc 2018; 69(7):987-1005. doi: 10.1057/s41274-017-0279-3 [Crossref] [ Google Scholar]
67. Hirsch G, Homer J, Evans E, Zielinski A. A system dynamics model for planning cardiovascular disease interventions. Am J Public Health 2010; 100(4):616-622. doi: 10.2105/ajph.2009.159434 [Crossref] [ Google Scholar]
68. Li Y, Kong N, Lawley M, Weiss L, Pagán JA. Advancing the use of evidence-based decision-making in local health departments with systems science methodologies. Am J Public Health 2015; 105(Suppl 2):S217-222. doi: 10.2105/ajph.2014.302077 [Crossref] [ Google Scholar]
69. Loyo HK, Batcher C, Wile K, Huang P, Orenstein D, Milstein B. From model to action: using a system dynamics model of chronic disease risks to align community action. Health PromotPract 2013; 14(1):53-61. doi: 10.1177/1524839910390305 [Crossref] [ Google Scholar]
70. Yarnoff B, Bradley C, Honeycutt AA, Soler RE, Orenstein D. Estimating the relative impact of clinical and preventive community-based interventions: an example based on the community transformation grant program. Prev Chronic Dis 2019; 16:E87. doi: 10.5888/pcd16.180594 [Crossref] [ Google Scholar]
71. Freebairn L, Atkinson JA, Qin Y. ‘Turning the tide’ on hyperglycemia in pregnancy: insights from multiscale dynamic simulation modeling. BMJ Open Diabetes Res Care 2020; 8(1):e000975. doi: 10.1136/bmjdrc-2019-000975 [Crossref] [ Google Scholar]
72. Freebairn L, Atkinson JA, Osgood ND, Kelly PM, McDonnell G, Rychetnik L. Turning conceptual systems maps into dynamic simulation models: an Australian case study for diabetes in pregnancy. PLoS One 2019; 14(6):e0218875. doi: 10.1371/journal.pone.0218875 [Crossref] [ Google Scholar]
73. Li Y, Padrón NA, Mangla AT, Russo PG, Schlenker T, Pagán JA. Using systems science to inform population health strategies in local health departments: a case study in San Antonio, Texas. Public Health Rep 2017; 132(5):549-555. doi: 10.1177/0033354917722149 [Crossref] [ Google Scholar]
74. Li Y, Zhang D, Thapa JR. Assessing the role of access and price on the consumption of fruits and vegetables across New York City using agent-based modeling. Prev Med 2018; 106:73-78. doi: 10.1016/j.ypmed.2017.10.014 [Crossref] [ Google Scholar]
75. Widener MJ, Metcalf SS, Bar-Yam Y. Agent-based modeling of policies to improve urban food access for low-income populations. Appl Geogr 2013; 40:1-10. doi: 10.1016/j.apgeog.2013.01.003 [Crossref] [ Google Scholar]
76. Abdollahiasl A, Kebriaeezadeh A, Dinarvand R. A system dynamics model for national drug policy. Daru 2014; 22(1):34. doi: 10.1186/2008-2231-22-34 [Crossref] [ Google Scholar]
77. Honeycutt AA, Wile K, Dove C, Hawkins J, Orenstein D. Strategic planning for chronic disease prevention in rural America: looking through a PRISM lens. J Public Health ManagPract 2015; 21(4):392-399. doi: 10.1097/phh.0000000000000062 [Crossref] [ Google Scholar]
78. Signal LN, Walton MD, Ni Mhurchu C. Tackling ‘wicked’ health promotion problems: a New Zealand case study. Health Promot Int 2013; 28(1):84-94. doi: 10.1093/heapro/das006 [Crossref] [ Google Scholar]
79. Zhang D, Giabbanelli PJ, Arah OA, Zimmerman FJ. Impact of different policies on unhealthy dietary behaviors in an urban adult population: an agent-based simulation model. Am J Public Health 2014; 104(7):1217-1222. doi: 10.2105/ajph.2014.301934 [Crossref] [ Google Scholar]
80. Carrete L, Arroyo P, Villaseñor R. A socioecological view toward an understanding of how to prevent overweight in children. J Consum Mark 2017; 34(2):156-168. doi: 10.1108/jcm-01-2016-1660 [Crossref] [ Google Scholar]
81. El-Sayed AM, Seemann L, Scarborough P, Galea S. Are network-based interventions a useful antiobesity strategy? An application of simulation models for causal inference in epidemiology. Am J Epidemiol 2013; 178(2):287-295. doi: 10.1093/aje/kws455 [Crossref] [ Google Scholar]
82. Johnston LM, Matteson CL, Finegood DT. Systems science and obesity policy: a novel framework for analyzing and rethinking population-level planning. Am J Public Health 2014; 104(7):1270-1278. doi: 10.2105/ajph.2014.301884 [Crossref] [ Google Scholar]
83. Liu S, Osgood N, Gao Q, Xue H, Wang Y. Systems simulation model for assessing the sustainability and synergistic impacts of sugar-sweetened beverages tax and revenue recycling on childhood obesity prevention. J Oper Res Soc 2016; 67(5):708-721. doi: 10.1057/jors.2015.99 [Crossref] [ Google Scholar]
84. Orr MG, Kaplan GA, Galea S. Neighbourhood food, physical activity, and educational environments and Black/White disparities in obesity: a complex systems simulation analysis. J Epidemiol Community Health 2016; 70(9):862-867. doi: 10.1136/jech-2015-205621 [Crossref] [ Google Scholar]
85. Powell KE, Kibbe DL, Ferencik R. Systems thinking and simulation modeling to inform childhood obesity policy and practice. Public Health Rep 2017; 132(2_Suppl):33S-38S. doi: 10.1177/0033354917723601 [Crossref] [ Google Scholar]
86. Roberts N, Li V, Atkinson J-A. Can the target set for reducing childhood overweight and obesity be met? A system dynamics modelling study in New South Wales, Australia. Syst Res Behav Sci 2019; 36(1):36-52. doi: 10.1002/sres.2555 [Crossref] [ Google Scholar]
87. Bellew W, Smith BJ, Nau T, Lee K, Reece L, Bauman A. Whole of systems approaches to physical activity policy and practice in Australia: the ASAPa project overview and initial systems map. J Phys Act Health 2020; 17(1):68-73. doi: 10.1123/jpah.2019-0121 [Crossref] [ Google Scholar]
88. Browne J, de Leeuw E, Gleeson D, Adams K, Atkinson P, Hayes R. A network approach to policy framing: a case study of the National Aboriginal and Torres Strait Islander Health Plan. Soc Sci Med 2017; 172:10-18. doi: 10.1016/j.socscimed.2016.11.011 [Crossref] [ Google Scholar]
89. Brennan LK, Brownson RC, Kelly C, Ivey MK, Leviton LC. Concept mapping: priority community strategies to create changes to support active living. Am J Prev Med 2012; 43(5 Suppl 4):S337-350. doi: 10.1016/j.amepre.2012.07.015 [Crossref] [ Google Scholar]
90. Yang Y, Auchincloss AH, Rodriguez DA, Brown DG, Riolo R, Diez-Roux AV. Modeling spatial segregation and travel cost influences on utilitarian walking: towards policy intervention. Comput Environ Urban Syst 2015; 51:59-69. doi: 10.1016/j.compenvurbsys.2015.01.007 [Crossref] [ Google Scholar]
91. Ahmad S, Franz GA. Raising taxes to reduce smoking prevalence in the US: a simulation of the anticipated health and economic impacts. Public Health 2008; 122(1):3-10. doi: 10.1016/j.puhe.2007.02.020 [Crossref] [ Google Scholar]
92. Cavana RY, Tobias M. Integrative system dynamics: analysis of policy options for tobacco control in New Zealand. Syst Res Behav Sci 2008; 25(5):675-694. doi: 10.1002/sres.934 [Crossref] [ Google Scholar]
93. Cavana RY, Clifford LV. Demonstrating the utility of system dynamics for public policy analysis in New Zealand: the case of excise tax policy on tobacco. Syst Dyn Rev 2006; 22(4):321-348. doi: 10.1002/sdr.347 [Crossref] [ Google Scholar]
94. Hammond RA, Combs TB, Mack-Crane A. Development of a computational modeling laboratory for examining tobacco control policies: Tobacco Town. Health Place 2020; 61:102256. doi: 10.1016/j.healthplace.2019.102256 [Crossref] [ Google Scholar]
95. Roberts EB, Homer J, Kasabian A, Varrell M. A systems view of the smoking problem: perspective and limitations of the role of science in decision-making. Int J Biomed Comput 1982; 13(1):69-86. doi: 10.1016/0020-7101(82)90051-4 [Crossref] [ Google Scholar]
96. Tengs TO, Ahmad S, Moore R, Gage E. Federal policy mandating safer cigarettes: a hypothetical simulation of the anticipated population health gains or losses. J Policy Anal Manage 2004; 23(4):857-872. doi: 10.1002/pam.20051 [Crossref] [ Google Scholar]
97. Tengs TO, Ahmad S, Savage JM, Moore R, Gage E. The AMA proposal to mandate nicotine reduction in cigarettes: a simulation of the population health impacts. Prev Med 2005; 40(2):170-180. doi: 10.1016/j.ypmed.2004.05.017 [Crossref] [ Google Scholar]
98. Tobias MI, Cavana RY, Bloomfield A. Application of a system dynamics model to inform investment in smoking cessation services in New Zealand. Am J Public Health 2010; 100(7):1274-1281. doi: 10.2105/ajph.2009.171165 [Crossref] [ Google Scholar]
99. Zwald ML, Eyler AA, Haire-Joshu D. Network influences on the development and implementation of active transportation policies in six US cities. Prev Med 2019; 118:176-183. doi: 10.1016/j.ypmed.2018.10.026 [Crossref] [ Google Scholar]
100. Peters D, Verweij S, Grêaux K, Stronks K, Harting J. Conditions for addressing environmental determinants of health behavior in intersectoral policy networks: a fuzzy set qualitative comparative analysis. Soc Sci Med 2017; 195:34-41. doi: 10.1016/j.socscimed.2017.08.036 [Crossref] [ Google Scholar]
101. Freebairn L, Rychetnik L, Atkinson JA. Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling. Health Res Policy Syst 2017; 15(1):83. doi: 10.1186/s12961-017-0245-1 [Crossref] [ Google Scholar]
102. McGetrick JA, Raine KD, Wild TC, Nykiforuk CIJ. Advancing strategies for agenda setting by health policy coalitions: a network analysis of the Canadian chronic disease prevention survey. Health Commun 2019; 34(11):1303-1312. doi: 10.1080/10410236.2018.1484267 [Crossref] [ Google Scholar]
103. Garney WR, Patterson MS, Garcia K, Muraleetharan D, McLeroy K. Interorganizational network findings from a nationwide cardiovascular disease prevention initiative. Eval Program Plann 2020; 79:101771. doi: 10.1016/j.evalprogplan.2019.101771 [Crossref] [ Google Scholar]
104. Beaton A, Manuel C, Tapsell J, Foote J, Oetzel JG, Hudson M. He Pikinga Waiora: supporting Māori health organisations to respond to pre-diabetes. Int J Equity Health 2019; 18(1):3. doi: 10.1186/s12939-018-0904-z [Crossref] [ Google Scholar]
105. Baker P, Brown AD, Wingrove K. Generating political commitment for ending malnutrition in all its forms: a system dynamics approach for strengthening nutrition actor networks. Obes Rev 2019; 20 Suppl 2:30-44. doi: 10.1111/obr.12871 [Crossref] [ Google Scholar]
106. Cullerton K, Donnet T, Lee A, Gallegos D. Joining the dots: the role of brokers in nutrition policy in Australia. BMC Public Health 2017; 17(1):307. doi: 10.1186/s12889-017-4217-8 [Crossref] [ Google Scholar]
107. Peters D, Raab J, Grêaux KM, Stronks K, Harting J. Structural integration and performance of inter-sectoral public health-related policy networks: an analysis across policy phases. Health Policy 2017; 121(12):1296-1302. doi: 10.1016/j.healthpol.2017.10.001 [Crossref] [ Google Scholar]
108. Heo HH, Jeong W, Che XH, Chung H. A stakeholder analysis of community-led collaboration to reduce health inequity in a deprived neighbourhood in South Korea. Glob Health Promot 2020; 27(2):35-44. doi: 10.1177/1757975918791517 [Crossref] [ Google Scholar]
109. Scheele CE, Little I, Diderichsen F. Governing health equity in Scandinavian municipalities: the inter-sectorial challenge. Scand J Public Health 2018; 46(1):57-67. doi: 10.1177/1403494816685538 [Crossref] [ Google Scholar]
110. Clarke B, Swinburn B, Sacks G. Understanding health promotion policy processes: a study of the government adoption of the Achievement Program in Victoria, Australia. Int J Environ Res Public Health 2018; 15(11):2393. doi: 10.3390/ijerph15112393 [Crossref] [ Google Scholar]
111. Clarke B, Swinburn B, Sacks G. Understanding the LiveLighter® obesity prevention policy processes: an investigation using political science and systems thinking. Soc Sci Med 2020; 246:112757. doi: 10.1016/j.socscimed.2019.112757 [Crossref] [ Google Scholar]
112. Pérez-Escamilla R, Lutter CK, Rabadan-Diehl C. Prevention of childhood obesity and food policies in Latin America: from research to practice. Obes Rev 2017; 18 Suppl 2:28-38. doi: 10.1111/obr.12574 [Crossref] [ Google Scholar]
113. Waqa G, Moodie M, Snowdon W. Exploring the dynamics of food-related policymaking processes and evidence use in Fiji using systems thinking. Health Res Policy Syst 2017; 15(1):74. doi: 10.1186/s12961-017-0240-6 [Crossref] [ Google Scholar]
114. Sturgiss E, Luig T, Campbell-Scherer DL, Lewanczuk R, Green LA. Using concept maps to compare obesity knowledge between policy makers and primary care researchers in Canada. BMC Res Notes 2019; 12(1):23. doi: 10.1186/s13104-018-4042-x [Crossref] [ Google Scholar]
115. de Bruin WE, Stayner C, Lange M, Taylor RW. Who are the key players involved with shaping public opinion and policies on obesity and diabetes in New Zealand?. Nutrients 2018; 10(11):1592. doi: 10.3390/nu10111592 [Crossref] [ Google Scholar]
116. Loitz CC, Stearns JA, Fraser SN, Storey K, Spence JC. Network analysis of inter-organizational relationships and policy use among active living organizations in Alberta, Canada. BMC Public Health 2017; 17(1):649. doi: 10.1186/s12889-017-4661-5 [Crossref] [ Google Scholar]
117. Barnes M, Maclean J, Cousens L. Understanding the structure of community collaboration: the case of one Canadian health promotion network. Health Promot Int 2010; 25(2):238-247. doi: 10.1093/heapro/daq002 [Crossref] [ Google Scholar]
118. Bergeron K, Lévesque L. Designing active communities: a coordinated action framework for planners and public health professionals. J Phys Act Health 2014; 11(5):1041-1051. doi: 10.1123/jpah.2012-0178 [Crossref] [ Google Scholar]
119. Buchthal OV, Taniguchi N, Iskandar L, Maddock J. Assessing state-level active living promotion using network analysis. J Phys Act Health 2013; 10(1):19-32. doi: 10.1123/jpah.10.1.19 [Crossref] [ Google Scholar]
120. Noël Racine A, Garbarino JM, Corrion K, D’Arripe-Longueville F, Massiera B, Vuillemin A. Perceptions of barriers and levers of health-enhancing physical activity policies in mid-size French municipalities. Health Res Policy Syst 2020; 18(1):62. doi: 10.1186/s12961-020-00575-z [Crossref] [ Google Scholar]
121. Spitters HP, Lau CJ, Sandu P. Unravelling networks in local public health policymaking in three European countries - a systems analysis. Health Res Policy Syst 2017; 15(1):5. doi: 10.1186/s12961-016-0168-2 [Crossref] [ Google Scholar]
122. Leppin AL, Okamoto JM, Organick PW. Applying social network analysis to evaluate implementation of a multisector population health collaborative that uses a bridging hub organization. Front Public Health 2018; 6:315. doi: 10.3389/fpubh.2018.00315 [Crossref] [ Google Scholar]
123. Fisher M, Milos D, Baum F, Friel S. Social determinants in an Australian urban region: a ‘complexity’ lens. Health Promot Int 2016; 31(1):163-174. doi: 10.1093/heapro/dau071 [Crossref] [ Google Scholar]
124. Hoeijmakers M, De Leeuw E, Kenis P, De Vries NK. Local health policy development processes in the Netherlands: an expanded toolbox for health promotion. Health Promot Int 2007; 22(2):112-121. doi: 10.1093/heapro/dam009 [Crossref] [ Google Scholar]
125. Leider JP, Castrucci BC, Harris JK, Hearne S. The relationship of policymaking and networking characteristics among leaders of large urban health departments. Int J Environ Res Public Health 2015; 12(8):9169-9180. doi: 10.3390/ijerph120809169 [Crossref] [ Google Scholar]
126. Merrill J, Keeling JW, Carley KM. A comparative study of 11 local health department organizational networks. J Public Health ManagPract 2010; 16(6):564-576. doi: 10.1097/PHH.0b013e3181e31cee [Crossref] [ Google Scholar]
127. Oliver K, Everett M, Verma A, de Vocht F. The human factor: re-organisations in public health policy. Health Policy 2012; 106(1):97-103. doi: 10.1016/j.healthpol.2012.03.009 [Crossref] [ Google Scholar]
128. Oliver K, de Vocht F, Money A, Everett M. Who runs public health? A mixed-methods study combining qualitative and network analyses. J Public Health (Oxf) 2013; 35(3):453-459. doi: 10.1093/pubmed/fdt039 [Crossref] [ Google Scholar]
129. Pineo H, Zimmermann N, Davies M. Integrating health into the complex urban planning policy and decision-making context: a systems thinking analysis. Palgrave Commun 2020; 6(1):21. doi: 10.1057/s41599-020-0398-3 [Crossref] [ Google Scholar]
130. van den Driessen Mareeuw F, Vaandrager L, Klerkx L, Naaldenberg J, Koelen M. Beyond bridging the know-do gap: a qualitative study of systemic interaction to foster knowledge exchange in the public health sector in the Netherlands. BMC Public Health 2015; 15:922. doi: 10.1186/s12889-015-2271-7 [Crossref] [ Google Scholar]
131. Harris JK. Communication ties across the national network of local health departments. Am J Prev Med 2013; 44(3):247-253. doi: 10.1016/j.amepre.2012.10.028 [Crossref] [ Google Scholar]
132. Harris JK, Luke DA, Burke RC, Mueller NB. Seeing the forest and the trees: using network analysis to develop an organizational blueprint of state tobacco control systems. Soc Sci Med 2008; 67(11):1669-1678. doi: 10.1016/j.socscimed.2008.07.013 [Crossref] [ Google Scholar]
133. Luke DA, Wald LM, Carothers BJ, Bach LE, Harris JK. Network influences on dissemination of evidence-based guidelines in state tobacco control programs. Health Educ Behav 2013; 40(1 Suppl):33S-42S. doi: 10.1177/1090198113492760 [Crossref] [ Google Scholar]
134. Moreland-Russell S, Carothers BJ. An examination of two policy networks involved in advancing smokefree policy initiatives. Int J Environ Res Public Health 2015; 12(9):11117-11131. doi: 10.3390/ijerph120911117 [Crossref] [ Google Scholar]
135. Weishaar H, Amos A, Collin J. Capturing complexity: mixing methods in the analysis of a European tobacco control policy network. Int J Soc Res Methodol 2015; 18(2):175-192. doi: 10.1080/13645579.2014.897851 [Crossref] [ Google Scholar]
136. Weishaar H, Amos A, Collin J. Best of enemies: using social network analysis to explore a policy network in European smoke-free policy. Soc Sci Med 2015; 133:85-92. doi: 10.1016/j.socscimed.2015.03.045 [Crossref] [ Google Scholar]
137. Macmillan A, Smith M, Witten K. Suburb-level changes for active transport to meet the SDGs: causal theory and a New Zealand case study. Sci Total Environ 2020; 714:136678. doi: 10.1016/j.scitotenv.2020.136678 [Crossref] [ Google Scholar]
138. Kokkinen L, Freiler A, Muntaner C, Shankardass K. How and why do win-win strategies work in engaging policy-makers to implement Health in All Policies? A multiple-case study of six state- and national-level governments. Health Res Policy Syst 2019; 17(1):102. doi: 10.1186/s12961-019-0509-z [Crossref] [ Google Scholar]
139. Shankardass K, Muntaner C, Kokkinen L. The implementation of Health in All Policies initiatives: a systems framework for government action. Health Res Policy Syst 2018; 16(1):26. doi: 10.1186/s12961-018-0295-z [Crossref] [ Google Scholar]
140. Cambon L, Minary L, Ridde V, Alla F. A tool to analyze the transferability of health promotion interventions. BMC Public Health 2013; 13:1184. doi: 10.1186/1471-2458-13-1184 [Crossref] [ Google Scholar]
141. Macdiarmid JI, Loe J, Douglas F, Ludbrook A, Comerford C, McNeill G. Developing a timeline for evaluating public health nutrition policy interventions. What are the outcomes and when should we expect to see them? Public Health Nutr 2011; 14(4):729-739. doi: 10.1017/s1368980010002168 [Crossref] [ Google Scholar]
142. Conte KP, Davidson S. Using a ‘rich picture’ to facilitate systems thinking in research coproduction. Health Res Policy Syst 2020; 18(1):14. doi: 10.1186/s12961-019-0514-2 [Crossref] [ Google Scholar]
143. Roussy V, Riley T, Livingstone C, Russell G. A system dynamic perspective of stop-start prevention interventions in Australia. Health Promot Int 2020; 35(5):1015-1025. doi: 10.1093/heapro/daz098 [Crossref] [ Google Scholar]
144. Beets MW, Webster C, Saunders R, Huberty JL. Translating policies into practice: a framework to prevent childhood obesity in afterschool programs. Health PromotPract 2013; 14(2):228-237. doi: 10.1177/1524839912446320 [Crossref] [ Google Scholar]
145. Knai C, Petticrew M, Douglas N. The public health responsibility deal: using a systems-level analysis to understand the lack of impact on alcohol, food, physical activity, and workplace health sub-systems. Int J Environ Res Public Health 2018; 15(12):2895. doi: 10.3390/ijerph15122895 [Crossref] [ Google Scholar]
146. Kuunders TJM, van Bon-Martens MJH, van de Goor IAM, Paulussen T, van Oers HAM. Towards local implementation of Dutch health policy guidelines: a concept-mapping approach. Health Promot Int 2018; 33(4):635-647. doi: 10.1093/heapro/dax003 [Crossref] [ Google Scholar]
147. Pagliccia N, Spiegel J, Alegret M, Bonet M, Martinez B, Yassi A. Network analysis as a tool to assess the intersectoral management of health determinants at the local level: a report from an exploratory study of two Cuban municipalities. Soc Sci Med 2010; 71(2):394-399. doi: 10.1016/j.socscimed.2010.03.041 [Crossref] [ Google Scholar]
148. van Roode T, Pauly BM, Marcellus L. Values are not enough: qualitative study identifying critical elements for prioritization of health equity in health systems. Int J Equity Health 2020; 19(1):162. doi: 10.1186/s12939-020-01276-3 [Crossref] [ Google Scholar]
149. Terpstra JL, Best A, Saul J, Leischow SJ. The complexity of institutionalizing evaluation as a best practice in North American quitlines. Am J Eval 2013; 34(3):356-371. doi: 10.1177/1098214013487426 [Crossref] [ Google Scholar]
150. Valente TW, Pitts S, Wipfli H, Vega Yon GG. Network influences on policy implementation: evidence from a global health treaty. Soc Sci Med 2019; 222:188-197. doi: 10.1016/j.socscimed.2019.01.008 [Crossref] [ Google Scholar]
151. Wen W, Zhang Z, Li Z. Public reactions to the cigarette control regulation on a Chinese microblogging platform: empirical analysis. J Med Internet Res 2020; 22(4):e14660. doi: 10.2196/14660 [Crossref] [ Google Scholar]
152. Cullerton K, Donnet T, Lee A, Gallegos D. Exploring power and influence in nutrition policy in Australia. Obes Rev 2016; 17(12):1218-1225. doi: 10.1111/obr.12459 [Crossref] [ Google Scholar]
153. Fisher M, Milos D, Baum F, Friel S. Social determinants in an Australian urban region: a ‘complexity’ lens. Health Promot Int 2016; 31(1):163-174. doi: 10.1093/heapro/dau071 [Crossref] [ Google Scholar]
154. Signal LN, Walton MD, Ni Mhurchu C. Tackling ‘wicked’ health promotion problems: a New Zealand case study. Health Promot Int 2013; 28(1):84-94. doi: 10.1093/heapro/das006 [Crossref] [ Google Scholar]
155. Castillo-Carandang NT, Buenaventura RD, Chia YC. Moving towards optimized noncommunicable disease management in the ASEAN region: recommendations from a review and multidisciplinary expert panel. Risk ManagHealthc Policy 2020; 13:803-819. doi: 10.2147/rmhp.s256165 [Crossref] [ Google Scholar]
156. Brown V, Barr A, Scheurer J, Magnus A, Zapata-Diomedi B, Bentley R. Better transport accessibility, better health: a health economic impact assessment study for Melbourne, Australia. Int J BehavNutr Phys Act 2019; 16(1):89. doi: 10.1186/s12966-019-0853-y [Crossref] [ Google Scholar]
157. McGill E, Er V, Penney T. Evaluation of public health interventions from a complex systems perspective: a research methods review. Soc Sci Med 2021; 272:113697. doi: 10.1016/j.socscimed.2021.113697 [Crossref] [ Google Scholar]
158. Anzola D, Barbrook-Johnson P, Cano JI. Self-organization and social science. Comput Math Organ Theory 2017; 23(2):221-257. doi: 10.1007/s10588-016-9224-2 [Crossref] [ Google Scholar]