You are here

Article Text

Article menu
Health services research
Original research
A search for relevant contextual factors in intervention studies: a stepwise approach with online information
  1. Michael Van der Elst1,2,
  2. Birgitte Schoenmakers1,
  3. Eva Dierckx3,
  4. Liesbeth De Donder4,
  5. Ellen De Roeck3,5,
  6. Daan Duppen4,
  7. Bram Fret4,
  8. Jos M G A Schols2,6,
  9. Gertrudis I J M Kempen2,
  10. Jan De Lepeleire1
  1. 1Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
  2. 2Department of Health Services Research, Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
  3. 3Department of Clinical & Lifespan Psychology, Vrije Universiteit Brussel (VUB), Brussels, Belgium
  4. 4Department of Educational Sciences, Vrije Universiteit Brussel (VUB), Brussels, Belgium
  5. 5Laboratory of Neurochemistry and Behavior, University of Antwerp, Antwerpen, Belgium
  6. 6Department of General Practice, School for Public Health and Primary Care (CAPHRI), Maastricht University, Maastricht, Netherlands
  1. Correspondence to Mr Michael Van der Elst; michael.vanderelst{at}kuleuven.be

Abstract

Objective The aim of the present study is to describe a stepwise approach to study which contextual factors might moderate the effect of healthcare interventions and to test feasibility of this approach within the D-SCOPE project.

Design Exploratory case study.

Setting In the D-SCOPE project, a complex intervention by means of home visits was set up to improve access to tailored care in three municipalities (Ghent, Knokke-Heist and Tienen).

Methods One designed and tested an approach including five steps: (1) a theoretical/conceptual discussion of relevant contextual factor domains was held; (2) a search was done to find appropriate web-based public datasets which covered these topics with standardised information; (3) a list of all identified contextual factors was made (inventory); (4) to reduce the long list of contextual factors, a concise list of most relevant contextual factors was developed based on the opinion of two independent reviewers and (5) a nominal grouping technique (NGT) was applied.

Results Three public web-based datasets were found resulting in an inventory of 157 contextual factors. After the selection by two independent reviewers, 41 contextual factors were left over and presented in a NGT which selected 10 contextual factors. The NGT included seven researchers, all familiar with the D-SCOPE intervention, with various educational backgrounds and expertise and lasted approximately 1 hour.

Conclusion The present study shows that a five-step approach is feasible to determine relevant contextual factors that might affect the results of an intervention study. Such information may be used to correct for in the statistical analyses and for interpretation of the outcomes of intervention studies.

NCT03168204

  • PUBLIC HEALTH
  • HEALTH SERVICES ADMINISTRATION & MANAGEMENT
  • PRIMARY CARE

Data availability statement

Data are available on reasonable request. Data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2021-057048

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The role of context within intervention studies is often ignored.

    • The World Wide Web offers an opportunity for to study the setting of an intervention.

    • The present study offers a uniform and standardised way based on five steps.

    • An analysis of the local context using online databases is feasible.

    • The present approach only presents a fraction of context and not the full context of a study.

    Introduction

    Randomised controlled trials (RCTs) are widely regarded as gold standard to identify causal relations between interventions and their predetermined outcomes. Some critics argue that, with respect to randomised trials of complex public health interventions, researchers fail to address the interaction of intervention components with each other and with the local context.1–3 In literature, the concept ‘context’ refers to spatial and institutional locations of social situations, with the inherent norms, values and interrelationships and describes those features of the conditions in which programmes are introduced.1 3 Key features of complex interventions are: (1) the number of interacting components (the number and complexity of behaviours required by those delivering or receiving the intervention), (2) the number of groups or organisational levels targeted by the intervention, (3) the number and variability of outcomes and (4) the degree of flexibility or tailoring of the intervention permitted.4 As interventions are almost always introduced into diverse contexts (eg, municipalities, neighbourhoods, clinics), the mechanisms activated by an intervention will vary according to the saliently different context conditions. Because of relevant variations in context and mechanisms activated by an intervention, its result is liable to have mixed outcome patterns.1 In RCTs of complex interventions, one often ignores the role of implementers, the local context and other factors that may moderate the effect of an intervention.2 5 Some authors argue that certain contexts are supportive to the intervention and some are not.1 The need for including contextually relevant factors was also highlighted in ‘The National Care For Elderly Programme’ (2008–2016), a countrywide government-funded programme in the Netherlands. Its goal was to develop a more proactive, integrated healthcare system for older adults. One conducted more than 70 scientific projects, including nine large-scale trials. None of these nine proactive primary-care programmes demonstrated clinically relevant effects on daily functioning. After a process evaluation, the authors concluded that in research, more attention should be given towards contextual factors and the need to develop a uniform methodology to study the local context in a standardised way.6 Currently, more attention is given to the importance of context and the understanding of context in complex interventions.7 8 Several guidances exist to support researchers during the design of a complex intervention and to take context into account.7–10 A wide range of research methods can be used to gain a better understanding of context in which an intervention operates, although the focus is on qualitative methods and less on quantitative methods.7 8 Nowadays, a significant amount of information can be found online, which was not available or difficult to find in the past. The World Wide Web could offer an opportunity for researchers to study the setting of an intervention. However, it is unknown whether the information available online is useful to study and compare local contexts.

    The present study is part of the Detection, Support and Care for Older People: Prevention and Empowerment (D-SCOPE) project, which features an organised trial that aimed to enable older adults to age well in place in three municipalities in Flanders (Belgium). After the baseline assessment, a professional from the social service of the municipality contacted participants assigned to the experimental for a home visit. During the home visit, the professional explored the older adult’s competences, needs and preferences. The professional proposed a type of intervention based on the results of the baseline assessment and home visit. In consultation with the participant, decisions were made with regard to tailored care and support. The intervention depended on the availability of care and support services in the municipality and could be formal (eg, home care) or informal (eg, activities of an older adult’s association). A professional from the social service of the municipality monitored which care the participant received. A professional of the municipality contacted every month all participants in the experimental group by phone. The aim of the contact was: (1) to verify whether the extra care and support was initiated and still ongoing, (2) to identify new care needs and (3) to assess the participants’ satisfaction of the given care and support.11 As a part of the D-SCOPE project, we wanted to know which contextual factors might interact/moderate the effect of a home visit and its related tailored care and support. This information can be useful in explaining the results of the D-SCOPE intervention study and provide insight regarding which context might be supportive for a home visit and which might not.

    In the present study, we describe an approach to study which contextual factors might moderate the effect of healthcare interventions and to apply this approach for the D-SCOPE intervention. As web-based public data are generally easily obtainable, we focus on context data from such resources. To determine feasibility to analyse local context, following research questions are answered: (1) are there relevant standardised web-based public data available in these three municipalities? and (2) how can the contextual factors most likely to interact with the intervention and moderate its outcomes be determined?

    Methods

    Design

    To test feasibility of determining relevant contextual factors in a RCT, one conducted an exploratory case study of the three municipalities within the D-SCOPE project.11 The participating municipalities in the D-SCOPE trial are Ghent, Knokke-Heist and Thienen, in Flanders (see online supplemental file 1: Map of Flanders). Therefore, only contextual factors of these three municipalities were considered. In what follows, one describes the different steps of the approach:

    Five-step approach

    Because of the complex nature of its intervention and depending on the availability of care and support services in the municipality, the effect may be context-sensitive.12–14 To determine relevant contextual factors within the D-SCOPE project, five steps were taken (see figure 1).

    Figure 1
    Figure 1

    Flow chart of the five-step approach to determine assumedly the most relevant contextual factors.

    Step 1: Theoretical/conceptual discussion of relevant contextual factor domains

    The authors (first, second and last) organised a meeting to discuss topics that should be covered with regard to the D-SCOPE intervention, meaning which features the data should fulfil to be included. The motivation to organise the meeting was based on the results of the meta-analysis of Van der Elst et al5 and professional experience of the two coauthors (the second and last author). In preparation of this meeting, the first author searched for scientific approaches to take into account the context in an intervention study and studies concerning contextual factors. Based on this literature, several inclusion and exclusion criteria were formulated such as the exclusion of factors only related to children, such as childcare or crèches.5

    Step 2: Explorative search for public databases

    To find appropriate and relevant public web-based datasets, the first author did an explorative search online. To be appropriate, public web-based databases had to include data concerning the topics as described in step one and meet the inclusion criteria. In the search of databases, we focused on governmental websites and scientific research institutes related to the Belgian/Flemish government (eg, KCE, WIV). Afterwards, the first author did a google search using terms like official statistics, local data(bases), data(bases) municipalities.

    Step 3: Inventory of the retrieved contextual factors

    In step three, after determining the appropriate public web-based databases, the first author made an inventory of contextual factors retrieved from the public databases. Thereby each municipality was a column and each variable was a row (see table 1). Contextual factors were separately categorised within a topic (eg, sociodemographic, socioeconomic). Regarding the availability of services, the inventory was based on the frameworks of official organisations. We used Microsoft Excel and the technique of mind mapping to construct the inventory. Mind mapping was used to structure and compare the available services in the three municipalities (see online supplemental file 2: Mind mapping).

    View this table:
    Table 1

    Inventory list.

    Step 4: Critical selection

    To reduce the number of contextual factors, two experienced clinicians in primary care (the second and last author) made a first (critical) selection. Both received the inventory with contextual factors and its distributions. They assigned each contextual factor a green, orange or red score, independently of each other. A green score indicated that a contextual factor might moderate the effect of the D-SCOPE intervention. An orange score reflected the opinion that one was not sure if a contextual factor might moderate the effect of the D-SCOPE intervention. A red score indicated that a contextual factor was not considered able to moderate the effect of the D-SCOPE intervention. Contextual factors assigned a green score by both reviewers were included in the fifth step; those factors with only red scores were automatically excluded. The first author organised a meeting with both authors to reach consensus regarding all other contextual factors.

    Step 5: Nominal grouping technique

    In order to determine the most relevant contextual factors, the first author organised a nominal grouping technique (NGT).15 The NGT included seven researchers of the D-SCOPE Consortium, all familiar with the D-SCOPE intervention, with various educational backgrounds and expertise (eg, nurse, psychologist, educational scientist) and lasted approximately 1 hour. NGT is a highly structured method in decision-making and contains five parts (see figure 2): (1) generating ideas: the participants received the inventory of contextual factors and its distributions. Each participant was asked to write down the contextual factors that might influence the outcome of a home visit and had to motivate why these factors were chosen. To keep it concise, the participants were asked to limit the number of factors up to 10. The participants registered them without discussion; (2) recording ideas: the participants then shared their ideas and motivations with the group, without discussion; (3) discussing/clarifying ideas: in this phase, the participants discussed the contextual factors and the motivations of choosing them; (4) voting/rating ideas: after discussion, every participant was asked to register those contextual factors (maximum of 10) that might influence the results of a home visit and rank them and (5) summing the ratings: a list of the 10 highest ranked contextual factors was made. The NGT method overcomes the problem of reluctance in participants who might be less willing to suggest ideas because of concerns of being criticised or creating conflict in groups.16–18

    Figure 2
    Figure 2

    Flow chart nominal grouping technique.

    Patient and public involvement

    The study presents analysis of secondary data. There was no patient and public involvement.

    Results

    In what follows, one presents the results of the five-step approach applied within the D-SCOPE project.

    Step 1: Theoretical/conceptual discussion of relevant contextual factor domains

    The aim of the intervention was to detect frail older people, improve their access to tailored care and support and facilitate ageing well in place. Therefore, the research team decided that the retrieved information should cover sociodemographic, socioeconomic contextual factors, factors related to care supply/availability or care use and factors related to the local government. Moreover, one determined that: (1) contextual factors should focus on older adults (aged 60 years and older) and (2) the public web-based dataset should use standardised data (eg, official statistics) of the three municipalities of the D-SCOPE trial.

    Step 2: Explorative search for public datasets

    Three suitable online public web-based datasets were identified in the selected municipalities: (1) the ‘InterMutualistic Agency’ database, (2) the ‘Local Statistics’ database and (3) the ‘Social Map’ database. The ‘InterMutualistic Agency’ database collects the data of seven Belgian health insurance institutions. The ‘Local Statistics’ database is a portal site in which all types of statistics regarding the local and provincial administrations have been collected. The ‘Social Map’ database collects data from healthcare organisations (broad interpretation) in a structured database. Additional information regarding the databases can be found in online supplemental file 3: Databases.

    Step 3: Inventory of the retrieved contextual factors

    The inventory included 157 contextual factors, retrieved from the aforementioned datasets: 70 contextual factors derived from the ‘InterMutualistic Agency’ database, 36 contextual factors derived from the ‘Local Statistics’ database and 51 contextual factors were derived from the ‘Social Map’ database. These contextual factors covered a broad range of information regarding the municipalities, including sociodemographic, socioeconomic, local governmental information and data on care supply/availability. Microsoft Excel was used to enlist contextual factors and its distributions. Since the ‘Social Map’ lists all organisations and describes the services they offer, the technique of mind mapping was used to structure and compare the available services in the municipalities (online supplemental file 2: Mind Mapping). To categorise the availability of care and support in the municipality, the framework of the agency ‘Zorg en Gezondheid’ (Care and Health) was used. This framework includes 12 domains, such as home care, geriatric care and hospitals as well as several subdomains of each domain. The agency ‘Zorg en Gezondheid’ was founded by the Flemish authorities and its main task is the organisation of care and support.19

    Step 4: Critical selection

    In total, two reviewers (the second and last author) independently selected 41 of the 157 contextual factors that were presented during the NGT. Eighty-five contextual factors received a red score (do not moderate the effect of the intervention) by both reviewers, while 28 were assigned a green score (might moderate the effect of the intervention) by the reviewers. All other factors were discussed (between the first, second and last author) until consensus was reached (online supplemental file 4: Critical selection). The final inventory of contextual factors included 9 factors of the ‘InterMutualistic Agency’ database, 7 from the ‘Local Statistics’ database and 25 of the ‘Social Map’ database.

    Step 5: Nominal grouping technique

    During the NGT, the list of the remaining contextual factors (see step 4) was presented. First, all participants had 10 min to go through the list of contextual factors and their distribution and to indicate the most relevant factors according to their opinions including motivating why. Second, all participants shared their most relevant factors and motivation, without any discussion. This task required 15 min. Third, the participants held a discussion of approximately 30 min. Fourth, the participants voted and afterwards the results were counted (step 5). In total, 20 of the 41 contextual factors presented in the NGT received votes. Within the D-SCOPE project, the aim was to retrieve a concise list of contextual factors. Therefore, table 2 presents those contextual factors with the highest scores (10) after voting in the NGT, together with the data of the three municipalities (derived from the three aforementioned databases). According to the participants of the NGT, those 10 contextual factors were likely the most important moderators of the D-SCOPE intervention. The number of contextual factors on the list is purely meant to illustrate the approach; further research should determine whether the selected contextual factors are moderating the D-SCOPE trial. The dependency ratio (age 65+/20–64) had the highest score of all the contextual factors.

    View this table:
    Table 2

    Ten contextual factors and their distribution after nominal grouping technique*

    Discussion

    In RCTs of complex interventions, one often ignores the role of the local context which may moderate the effect of an intervention. Therefore, more attention should be given to contextual factors in the design and analysis of complex interventions. However, it remained unclear whether it is feasible to explore and analyse the local context with online information. The present study shows that based, on a five-step approach an analysis of the context using online data(bases) is possible. The results show that a large amount of standardised data (contextual factors) is accessible on public web-based datasets. The five-step approach seems useful to collect and select relevant contextual factors that might influence the outcome of an intervention applied in a specific context.

    A first key finding is the large amount of standardised public information/data currently available online (eg, official statistics) which offers an opportunity for researchers. These web-based datasets cover a broad range of topics, such as sociodemographic data, socioeconomic data, information related to the availability of care support services (these data were considered important in the context of the D-SCOPE programme). The adopted approach in the present study makes it possible for future research to have a more comprehensive understanding of the setting in which a healthcare intervention is implemented. However, the amount and type of information identified may differ depending on country/region and topic of study. For instance, in the D-SCOPE project, the inventory contextual factors consisted of 157 factors.

    Since a large amount of online information is available, one can assume that not all of this information is useful. Therefore, a systematic approach is essential to construct a concise list of contextual factors. A second result of the present study, therefore, is the five-step approach as described in the methods that was used to identify relevant contextual factors. The discussion section within the NGT (step 5) can be used to formulate hypotheses and may help to explain the final results of the intervention. For instance, during the discussion in the NGT it was argued that the availability of a community centre would have a moderating effect in the D-SCOPE intervention because it is important for social participation and organising activities, but it also provides information, educational activities, meals and helps people to refer to other care and support services (‘snowball-effect’). The lack of a community centre in Knokke-Heist made it impossible for the professional of the social service centre to refer participants towards other care and support services.

    Third, as a result of the five-step approach, it was revealed that in the D-SCOPE programme, large differences were found between the three municipalities (Ghent, Knokke-Heist and Thienen). Sociodemographically, Knokke-Heist had the oldest population, with a dependency ratio (65+/20–64 years) of 63.1% compared with 27.03% in Ghent and 36.21% in Thienen. In Knokke-Heist, the percentage of adults older than 80 years of age was almost twice as high compared with Ghent, while the total resources of community social security in euros per inhabitant in the year 2013 was only half of the budget in Ghent. These differences in contextual settings between the three municipalities may moderate the effect of the D-SCOPE intervention on its outcomes and emphasises the relevance of context. For instance, a previous systematic review by Stuck et al concluded that preventive home visits reduce mortality in a younger study population (mean age <80 years) but not in older populations.20

    Strengths and limitations

    The present study has several strengths. First, the present study gives a systematic approach to investigate the local context in an easy-to-apply way. Second, previous studies have shown that the NGT is a valid method in decision-making, based on the expertise of experienced researchers.16 17 The NGT made it feasible to reduce a long inventory of contextual factors to a short and concise list with the assumedly most relevant ones.

    Our study also has some limitations. First, according to the socioecological model, context can be divided into various layers: microsystem, mesosystem, exosystem and macrosystem. The present study solely focuses on the level of the municipality and not on the individual or cultural level. For example, no information is found regarding relevant contextual factors, such as the level of coordination between and within services/institutions, or the norms and values within/between municipalities.21 Second, the present information was retrieved from three public web-based datasets. The correctness of analysis depends on the correctness and accuracy of those datasets (eg, for many contextual factors the latest update was in 2014–2015, although the intervention study started in 2017). Third, regardless of the large amount of information that can be found online, it is plausible that a significant amount of relevant information is still missing. For instance, we are aware that Knokke-Heist does not have a community centre; however, no information is available regarding the activities organised by local organisations or other initiatives organised by the municipality that could function as an alternative for a community centre. Fourth, several aspects of the 5-step approach are based on experts’ opinions (eg, part four and five). This indicates the assumption that the D-SCOPE trial can interact with the selected contextual factors. However, further evidence-based research is needed.

    Implications and future research

    New innovations and technologies offer opportunities for contemporary and future scientists. Before the existence of the World Wide Web, constructing an inventory of contextual factors in different communities would be a considerable and time-consuming challenge. Today, a substantial amount of information can be found in online-standardised datasets. This enables future intervention studies to take the local context into account. For instance, the present results can be useful to explain differences in effect of the D-SCOPE intervention in the three municipalities and provide insight regarding contexts that might be supportive for a home visit and those that are not. For instance, older adults in need of extra social contact and participation could not be referred to a community centre in Knokke-Heist, while this is possible in Ghent and Thienen, where a community centre is available. The lack of a community centre in Knokke-Heist could impact how the D-SCOPE intervention affected its outcomes. Based on these insights of the present study, new (theory-driven) hypotheses can be formulated that can be tested, giving a better understanding of mechanisms related to an intervention. Therefore, we would advise researchers to perform an analysis of context before the start of an intervention to avoid posthoc data-driven analysis in urge to explain the results. In case an intervention study includes many municipalities, a contextual factor can also be used as moderator in the statistical model. Within the D-SCOPE project the availability of a community centre could be an independent dummy variable in the statistical analysis: the value 0=not available in the municipality and the value 1=available in the municipality. Contextual factors can also be changed into an ordinal scale. We illustrate this with the variable ‘total resources of the community social security in euros per inhabitant’ which can be ordered as 1=municipality with the lowest resources per capita (Knokke-Heist); 2=municipality with the mid value (Thienen) and 3=municipality with the highest resources per capita (Ghent).

    Because of the proposed five-step approach, future RCTs could meet the criticism of lack of attention to context when evaluating an intervention.1 This five-step approach can also be used for interventions with other topics (eg, economic research, criminology) or research for other purposes; for instance, the risk stratification of areas whereby characteristics (eg, sociodemographic, socioeconomic, care supply) of a village, municipality or city are assessed and compared with macro-level data to determine local (health) needs and challenges.22 23

    Conclusion

    Some authors argue that certain contexts are supportive for the implementation of an intervention and some are not, although the role of context is often ignored in RCTs.1 The present study shows that it is feasible to perform an analysis of contextual factors that could impact outcomes in a RCT. A significant amount of information is available online and an easy-to-apply five-step approach can determine the assumedly most relevant contextual factors. With this five-step approach, future intervention studies can consider the local context when examining the effect of an intervention and formulate theory-driven hypotheses in RCTs. This should give us a better understanding of the effects of an intervention and the mechanisms related to the intervention.

    Data availability statement

    Data are available on reasonable request. Data are available.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. Pawson R,
      2. Tilley N
      . Realist evaluation. monograph prepared for British cabinet office, 2004. Available: https://www.communitymatters.com.au/RE_chapter.pdf
    2. ↵
      1. Moore G,
      2. Audrey S,
      3. Barker M
      . (MRC) guidance on process evaluation of complex interventions UK medical Research Council (MRC) guidance, 2015. Available: https://www.mrc.ac.uk/documents/pdf/mrc-phsrn-process-evaluation-guidance-final/
    3. ↵
      1. Pawson R,
      2. Tilley N
      . Realistic evaluation. Sage Publications, Inc, 1997.
    4. ↵
      1. Craig P,
      2. Dieppe P,
      3. Macintyre S, et al
      . Developing and evaluating complex interventions: the new medical Research Council guidance. BMJ 2008;337:a1655.doi:10.1136/bmj.a1655pmid:http://www.ncbi.nlm.nih.gov/pubmed/18824488
      OpenUrlFREE Full Text
    5. ↵
      1. Van der Elst M,
      2. Schoenmakers B,
      3. Duppen D, et al
      . Interventions for frail community-dwelling older adults have no significant effect on adverse outcomes: a systematic review and meta-analysis. BMC Geriatr 2018;18): :249.doi:10.1186/s12877-018-0936-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/30342479
      OpenUrlCrossRefPubMed
    6. ↵
      1. Smit LC,
      2. Schuurmans MJ,
      3. Blom JW, et al
      . Unravelling complex primary-care programs to maintain independent living in older people: a systematic overview. J Clin Epidemiol 2018;96: :110–9.doi:10.1016/j.jclinepi.2017.12.013pmid:http://www.ncbi.nlm.nih.gov/pubmed/29289764
      OpenUrlPubMed
    7. ↵
      1. O'Cathain A,
      2. Croot L,
      3. Duncan E, et al
      . Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open 2019;9:e029954.doi:10.1136/bmjopen-2019-029954pmid:http://www.ncbi.nlm.nih.gov/pubmed/31420394
      OpenUrlCrossRefPubMed
    8. ↵
      1. Craig P,
      2. Di Ruggiero E,
      3. Frohlich KL
      . Taking account of context in population health intervention research: guidance for producers, users and funders of research. NIHR Evaluation, Trials and Studies Coordinating Centre: Southampton, 2018. https://www.ncbi.nlm.nih.gov/books/NBK498645/pdf/Bookshelf_NBK498645.pdf
    9. ↵
      1. Moore GF,
      2. Evans RE,
      3. Hawkins J, et al
      . From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation 2019;25:23–45.doi:10.1177/1356389018803219pmid:http://www.ncbi.nlm.nih.gov/pubmed/30705608
      OpenUrlCrossRefPubMed
    10. ↵
      1. Campbell M,
      2. Katikireddi SV,
      3. Hoffmann T, et al
      . TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ 2018;361:k1079.doi:10.1136/bmj.k1079pmid:http://www.ncbi.nlm.nih.gov/pubmed/29769210
      OpenUrlFREE Full Text
    11. ↵
      1. Lambotte D,
      2. De Donder L,
      3. De Roeck EE, et al
      . Randomized controlled trial to evaluate a prevention program for frail community-dwelling older adults: a D-SCOPE protocol. BMC Geriatr 2018;18:194.doi:10.1186/s12877-018-0875-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/30149798
      OpenUrlPubMed
    12. ↵
      1. Gittell JH,
      2. Weiss L
      . Coordination networks within and across organizations: a multi-level Framework*. Journal of Management Studies 2004;41:127–53.doi:10.1111/j.1467-6486.2004.00424.x
      OpenUrl
    13. ↵
      1. Moore GF,
      2. Audrey S,
      3. Barker M, et al
      . Process evaluation of complex interventions: medical Research Council guidance. BMJ 2015;350:h1258.doi:10.1136/bmj.h1258pmid:http://www.ncbi.nlm.nih.gov/pubmed/25791983
      OpenUrlFREE Full Text
    14. ↵
      1. Holland JH
      . Complex adaptive systems. Daedalus 1992:17–30.
    15. ↵
      1. Delbecq AL,
      2. Van de Ven AH
      . A group process model for problem identification and program planning. J Appl Behav Sci 1971;7:466–92.doi:10.1177/002188637100700404
      OpenUrlCrossRefWeb of Science
    16. ↵
      1. Brahm C,
      2. Kleiner BH
      . Advantages and disadvantages of group decision‐ making approaches. Team Performance Management: An International Journal 1996;2:30–5.doi:10.1108/13527599610105538
      OpenUrl
    17. ↵
      1. Jorn SD,
      2. Williamson JW
      . Statistical methods for reliability and validity testing: an application to nominal group judgments in health care. Med Care 1977;15:922–8.doi:10.1097/00005650-197711000-00006pmid:http://www.ncbi.nlm.nih.gov/pubmed/926874
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Fink A,
      2. Kosecoff J,
      3. Chassin M, et al
      . Consensus methods: characteristics and guidelines for use. Am J Public Health 1984;74:979–83.doi:10.2105/AJPH.74.9.979pmid:http://www.ncbi.nlm.nih.gov/pubmed/6380323
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Agentschap zorg en gezondheid
      , 2018. Available: https://www.zorg-en-gezondheid.be/per-domein
    20. ↵
      1. Stuck AE,
      2. Egger M,
      3. Hammer A, et al
      . Home visits to prevent nursing home admission and functional decline in elderly people: systematic review and meta-regression analysis. JAMA 2002;287:1022–8.doi:10.1001/jama.287.8.1022pmid:http://www.ncbi.nlm.nih.gov/pubmed/11866651
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Bronfenbenner U
      . The ecology of human development; experiments by nature and design. Harvard University Press, 1979.
    22. ↵
      Agenstschap voor Zorg en Gezondheid, Zorgregiodecreet van 23 mei 2003 betreffende de indeling in zorgregio’s en betreffende de samenwerking en programmatie van gezondheidsvoorzieningen en welzijnsvoorzieningen. Available: https://www.integreo.be/sites/default/files/public/content/praktische_fiche_risicostratificatie_en_middelencartografie_nl.pdf
    23. ↵
      Integrio, praktische fiche component 13 : risicostratificatie van de populatie en cartografie van de middelen. Available: https://www.integreo.be/sites/default/files/public/content/praktische_fiche_risicostratificatie_en_middelencartografie_nl.pdf

    Supplementary materials

    Footnotes

    • Contributors All authors contributed to the design of the D-SCOPE intervention. The present study within the D-SCOPE trial was conceived by MVdE, JMGAS, GIJMK, JDL and BS. MVdE developed the inventory and wrote the first draft of the manuscript. LDD, BF, EDR and DD participated in the NGT and contributed to the data-analysis. All authors critically revised the manuscript and reviewed and approved the final manuscript. MVdE is responsible for the overall content as guarantor. All authors comply with the conditions of authorship according to the ICMJE.

    • Funding This work was supported by a grant from the Flemish government agency for Innovation by Science and Technology (IWT-140027 SBO).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Author note The D-SCOPE consortium is an international research consortium and is composed of researchers from Vrije Universiteit Brussel, Belgium (Dr A-.S. Smetcoren, Dr S. Dury, Professor Dr L. De Donder, Professor Dr E. Dierckx, Dr D. Lambotte, Dr B. Fret, Dr D. Duppen, Professor Dr M. Kardol, Professor Dr D. Verté); College University Ghent, Belgium (Dr L. J. Hoeyberghs, Professor Dr N. De Witte); Universiteit Antwerpen, Belgium (Dr E. E. De Roeck, Professor Dr S. Engelborghs, Professor Dr P. P. De Deyn); Katholieke Universiteit Leuven, Belgium (M. Van der Elst, Professor Dr J. De Lepeleire, Professor Dr B. Schoenmakers) and Maastricht University, The Netherlands (M. Van der Elst, Dr A. van der Vorst, Dr G.A.R. Zijlstra, Professor Dr G .I. J. M. Kempen, Professor Dr J. M. G. A. Schols).

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.