Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey

Joanna C Thorn; Charlotte F Davies; Sara T Brookes; Sian M Noble; Melina Dritsaki; Ewan Gray; Dyfrig A Hughes; Borislava Mihaylova; Stavros Petrou; Colin Ridyard; Tracey Sach; Edward C F Wilson; Sarah Wordsworth; William Hollingworth

doi:10.1016/j.jval.2020.10.002

Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey

Joanna C Thorn, Charlotte F Davies, Sara T Brookes, Sian M Noble, Melina Dritsaki, Ewan Gray, Dyfrig A Hughes, Borislava Mihaylova, Stavros Petrou, Colin Ridyard, Tracey Sach, Edward C F Wilson, Sarah Wordsworth, William Hollingworth

Research output: Contribution to journal › Article (Academic Journal) › peer-review

28 Citations (Scopus)

121 Downloads (Pure)

Abstract

ObjectivesHealth economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.
MethodsA list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant’s own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.
Results62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.
ConclusionsVia expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.

Original language	English
Pages (from-to)	539-547
Number of pages	9
Journal	Value in Health
Volume	24
Issue number	4
Early online date	28 Nov 2020
DOIs	https://doi.org/10.1016/j.jval.2020.10.002
Publication status	Published - 24 Apr 2021

Bibliographical note

Funding Information:
Author Contributions: Concept and design: Thorn, Davies, Brookes, Noble, Hughes, Mihaylova, Petrou, Ridyard, Wordsworth, Hollingworth, Acquisition of data: Thorn, Davies, Brookes, Dritsaki, Hughes, Ridyard, Wordsworth, Hollingworth, Analysis and interpretation of data: Thorn, Davies, Brookes, Dritsaki, Gray, Hughes, Mihaylova, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Drafting of the manuscript: Thorn, Davies, Brookes, Dritsaki, Gray, Petrou, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Critical revision of the paper for important intellectual content: Thorn, Brookes, Noble, Dritsaki, Gray, Hughes, Mihaylova, Petrou, Sach, Wilson, Wordsworth, Hollingworth, Obtaining funding: Thorn, Brookes, Noble, Hughes, Mihaylova, Wordsworth, Hollingworth, Administrative, technical, or logistic support: Davies, Supervision: Thorn, Noble, Hollingworth, Conflict of Interest Disclosures: Dr Thorn reported receiving grants from Medical Research Council during the conduct of the study. Dr Noble reported receiving grants from MRC HTMR during the conduct of the study. Dr Hughes reported receiving grants from MRC NWHTMR (North West Hub for Trials Methodology Research) and grants from MRC Network of Hubs for Trials Methodology Research during the conduct of the study. Dr Mihaylova reported receiving grants from UK Medical Research Council Hubs for Trials Methodology Research during the conduct of the study; she is also an editor of Value in Health and had no role in the peer review process of this article. Dr Sach reported that she is a member of the NIHR Health Technology Assessment commissioning committee at funds studies which may include economic evaluations for which it would be useful to write a HEAP; she is also the lead health economist on a number of studies funded by the NIHR Health Technology Assessment programme and has experience of writing HEAPs in this context. Dr Hollingworth reported receiving grants from Medical Research Council during the conduct of the study. No other disclosures were reported. Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1), and the MRC Network of Hubs for Trials Methodology Research (MR/L004933/1-N65). BM and SW acknowledge support by the National Institute for Health Research Oxford Biomedical Research Centre. Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Acknowledgment: We would like to thank the participants for their time spent completing the 2 Delphi questionnaires whose expert participation enabled us to conduct this study, and David Turner, Rita Faria, and the audience for a helpful discussion of the work at the Health Economists? Study Group meeting in Bristol, June 2018. We would also like to thank Chris Rogers and Lucy Culliford for their contribution to the final item selection meeting, Carrol Gamble for advice on statistical analysis plans, and Alison Horne for help with the REDCap software. We would like to thank the individuals who shared unpublished analysis plans with us, contributing to the examples used.

Funding Information:
Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1 ), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1 ), and the MRC Network of Hubs for Trials Methodology Research ( MR/L004933/1-N65 ). BM and SW acknowledge support by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC).

Publisher Copyright:
© 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research

Research Groups and Themes

Bristol Population Health Science Institute
HEHP@Bristol

Keywords

analysis plans
bias
economic evaluation

Access to Document

10.1016/j.jval.2020.10.002

Full-text PDF (final published version)
This is the final published version of the article (version of record). It first appeared online via International Society for Pharmacoeconomics and Outcomes Research at https://www.valueinhealthjournal.com/article/S1098-3015(20)34444-2/fulltext . Please refer to any applicable terms of use of the publisher.
Final published version, 1.04 MBLicence: CC BY

Handle.net

Persistent link

Embargoed Document

Full-text PDF (author’s accepted manuscript)
Accepted author manuscript, 346 KB
Request copy

Embargoed Document

Supplementary information PDF
Accepted author manuscript, 256 KB
Request copy

8071 - HEAPS 2 HTMR Network Impact Award: 'Health Economics Analysis Plans: developing content guidance through consensus' (CLOSED)
Thorn, J. C. (Principal Investigator)
1/09/18 → 31/05/19
Project: Research
Health Economics Analysis Plans: devising content guidance through consensus
Thorn, J. C. (Principal Investigator)
1/06/17 → 31/05/18
Project: Research

Cite this

Thorn, J. C., Davies, C. F., Brookes, S. T., Noble, S. M., Dritsaki, M., Gray, E., Hughes, D. A., Mihaylova, B., Petrou, S., Ridyard, C., Sach, T., Wilson, E. C. F., Wordsworth, S., & Hollingworth, W. (2021). Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey. Value in Health, 24(4), 539-547. https://doi.org/10.1016/j.jval.2020.10.002

@article{7e62cd979cde4c6f9b4ac91ac65eee3d,

title = "Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey",

abstract = "ObjectivesHealth economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.MethodsA list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant{\textquoteright}s own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.Results62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.ConclusionsVia expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.",

keywords = "analysis plans, bias, economic evaluation",

author = "Thorn, {Joanna C} and Davies, {Charlotte F} and Brookes, {Sara T} and Noble, {Sian M} and Melina Dritsaki and Ewan Gray and Hughes, {Dyfrig A} and Borislava Mihaylova and Stavros Petrou and Colin Ridyard and Tracey Sach and Wilson, {Edward C F} and Sarah Wordsworth and William Hollingworth",

note = "Funding Information: Author Contributions: Concept and design: Thorn, Davies, Brookes, Noble, Hughes, Mihaylova, Petrou, Ridyard, Wordsworth, Hollingworth, Acquisition of data: Thorn, Davies, Brookes, Dritsaki, Hughes, Ridyard, Wordsworth, Hollingworth, Analysis and interpretation of data: Thorn, Davies, Brookes, Dritsaki, Gray, Hughes, Mihaylova, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Drafting of the manuscript: Thorn, Davies, Brookes, Dritsaki, Gray, Petrou, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Critical revision of the paper for important intellectual content: Thorn, Brookes, Noble, Dritsaki, Gray, Hughes, Mihaylova, Petrou, Sach, Wilson, Wordsworth, Hollingworth, Obtaining funding: Thorn, Brookes, Noble, Hughes, Mihaylova, Wordsworth, Hollingworth, Administrative, technical, or logistic support: Davies, Supervision: Thorn, Noble, Hollingworth, Conflict of Interest Disclosures: Dr Thorn reported receiving grants from Medical Research Council during the conduct of the study. Dr Noble reported receiving grants from MRC HTMR during the conduct of the study. Dr Hughes reported receiving grants from MRC NWHTMR (North West Hub for Trials Methodology Research) and grants from MRC Network of Hubs for Trials Methodology Research during the conduct of the study. Dr Mihaylova reported receiving grants from UK Medical Research Council Hubs for Trials Methodology Research during the conduct of the study; she is also an editor of Value in Health and had no role in the peer review process of this article. Dr Sach reported that she is a member of the NIHR Health Technology Assessment commissioning committee at funds studies which may include economic evaluations for which it would be useful to write a HEAP; she is also the lead health economist on a number of studies funded by the NIHR Health Technology Assessment programme and has experience of writing HEAPs in this context. Dr Hollingworth reported receiving grants from Medical Research Council during the conduct of the study. No other disclosures were reported. Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1), and the MRC Network of Hubs for Trials Methodology Research (MR/L004933/1-N65). BM and SW acknowledge support by the National Institute for Health Research Oxford Biomedical Research Centre. Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Acknowledgment: We would like to thank the participants for their time spent completing the 2 Delphi questionnaires whose expert participation enabled us to conduct this study, and David Turner, Rita Faria, and the audience for a helpful discussion of the work at the Health Economists? Study Group meeting in Bristol, June 2018. We would also like to thank Chris Rogers and Lucy Culliford for their contribution to the final item selection meeting, Carrol Gamble for advice on statistical analysis plans, and Alison Horne for help with the REDCap software. We would like to thank the individuals who shared unpublished analysis plans with us, contributing to the examples used. Funding Information: Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1 ), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1 ), and the MRC Network of Hubs for Trials Methodology Research ( MR/L004933/1-N65 ). BM and SW acknowledge support by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC). Publisher Copyright: {\textcopyright} 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research",

year = "2021",

month = apr,

day = "24",

doi = "10.1016/j.jval.2020.10.002",

language = "English",

volume = "24",

pages = "539--547",

journal = "Value in Health",

issn = "1098-3015",

publisher = "Elsevier Inc.",

number = "4",

}

Thorn, JC , Davies, CF, Brookes, ST, Noble, SM, Dritsaki, M, Gray, E, Hughes, DA, Mihaylova, B, Petrou, S, Ridyard, C, Sach, T, Wilson, ECF, Wordsworth, S & Hollingworth, W 2021, 'Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey', Value in Health, vol. 24, no. 4, pp. 539-547. https://doi.org/10.1016/j.jval.2020.10.002

TY - JOUR

T1 - Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations

T2 - expert Delphi consensus survey

AU - Thorn, Joanna C

AU - Davies, Charlotte F

AU - Brookes, Sara T

AU - Noble, Sian M

AU - Dritsaki, Melina

AU - Gray, Ewan

AU - Hughes, Dyfrig A

AU - Mihaylova, Borislava

AU - Petrou, Stavros

AU - Ridyard, Colin

AU - Sach, Tracey

AU - Wilson, Edward C F

AU - Wordsworth, Sarah

AU - Hollingworth, William

N1 - Funding Information: Author Contributions: Concept and design: Thorn, Davies, Brookes, Noble, Hughes, Mihaylova, Petrou, Ridyard, Wordsworth, Hollingworth, Acquisition of data: Thorn, Davies, Brookes, Dritsaki, Hughes, Ridyard, Wordsworth, Hollingworth, Analysis and interpretation of data: Thorn, Davies, Brookes, Dritsaki, Gray, Hughes, Mihaylova, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Drafting of the manuscript: Thorn, Davies, Brookes, Dritsaki, Gray, Petrou, Ridyard, Sach, Wilson, Wordsworth, Hollingworth, Critical revision of the paper for important intellectual content: Thorn, Brookes, Noble, Dritsaki, Gray, Hughes, Mihaylova, Petrou, Sach, Wilson, Wordsworth, Hollingworth, Obtaining funding: Thorn, Brookes, Noble, Hughes, Mihaylova, Wordsworth, Hollingworth, Administrative, technical, or logistic support: Davies, Supervision: Thorn, Noble, Hollingworth, Conflict of Interest Disclosures: Dr Thorn reported receiving grants from Medical Research Council during the conduct of the study. Dr Noble reported receiving grants from MRC HTMR during the conduct of the study. Dr Hughes reported receiving grants from MRC NWHTMR (North West Hub for Trials Methodology Research) and grants from MRC Network of Hubs for Trials Methodology Research during the conduct of the study. Dr Mihaylova reported receiving grants from UK Medical Research Council Hubs for Trials Methodology Research during the conduct of the study; she is also an editor of Value in Health and had no role in the peer review process of this article. Dr Sach reported that she is a member of the NIHR Health Technology Assessment commissioning committee at funds studies which may include economic evaluations for which it would be useful to write a HEAP; she is also the lead health economist on a number of studies funded by the NIHR Health Technology Assessment programme and has experience of writing HEAPs in this context. Dr Hollingworth reported receiving grants from Medical Research Council during the conduct of the study. No other disclosures were reported. Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1), and the MRC Network of Hubs for Trials Methodology Research (MR/L004933/1-N65). BM and SW acknowledge support by the National Institute for Health Research Oxford Biomedical Research Centre. Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Acknowledgment: We would like to thank the participants for their time spent completing the 2 Delphi questionnaires whose expert participation enabled us to conduct this study, and David Turner, Rita Faria, and the audience for a helpful discussion of the work at the Health Economists? Study Group meeting in Bristol, June 2018. We would also like to thank Chris Rogers and Lucy Culliford for their contribution to the final item selection meeting, Carrol Gamble for advice on statistical analysis plans, and Alison Horne for help with the REDCap software. We would like to thank the individuals who shared unpublished analysis plans with us, contributing to the examples used. Funding Information: Funding/Support: This work was undertaken with the support of the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures - MR/K025643/1 ), the MRC NWHTMR (North West Hub for Trials Methodology Research - MR/K025635/1 ), and the MRC Network of Hubs for Trials Methodology Research ( MR/L004933/1-N65 ). BM and SW acknowledge support by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC). Publisher Copyright: © 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research

PY - 2021/4/24

Y1 - 2021/4/24

N2 - ObjectivesHealth economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.MethodsA list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant’s own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.Results62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.ConclusionsVia expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.

AB - ObjectivesHealth economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.MethodsA list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant’s own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.Results62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.ConclusionsVia expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.

KW - analysis plans

KW - bias

KW - economic evaluation

U2 - 10.1016/j.jval.2020.10.002

DO - 10.1016/j.jval.2020.10.002

M3 - Article (Academic Journal)

C2 - 33840432

SN - 1098-3015

VL - 24

SP - 539

EP - 547

JO - Value in Health

JF - Value in Health

IS - 4

ER -

Content of Health Economics Analysis Plans (HEAPs) for trial-based economic evaluations: expert Delphi consensus survey

Abstract

Bibliographical note

Research Groups and Themes

Keywords

Access to Document

Handle.net

Embargoed Document

Embargoed Document

Fingerprint

Projects

8071 - HEAPS 2 HTMR Network Impact Award: 'Health Economics Analysis Plans: developing content guidance through consensus' (CLOSED)

Health Economics Analysis Plans: devising content guidance through consensus

Cite this