A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

Angus G K McNair; Christin Hoffmann; Rhiannon C Macefield; Daisy Elliott; Jane Blazeby; Kerry N L Avery; Shelley Potter

doi:10.1016/j.jclinepi.2022.10.006

A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

Research output: Contribution to journal › Article (Academic Journal) › peer-review

2 Citations (Scopus)

194 Downloads (Pure)

Abstract

Objective
During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators’ experiences (how professionals feel undertaking the innovation) drives this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators’ experience of surgical innovation.
Study design and setting
There were three phases: 1) Literature reviews identified measures of operators’ experience and concepts measured were extracted and grouped into domains. 2) Quality appraisal was conducted to assess content validity of identified instruments, and was supported by COSMIN methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COSMIN criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. 3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multi-disciplinary professionals and a focus group.
Results
Literature reviews identified 16 measurement instruments from 243 studies. Most assessed ‘psychological’ experiences and ‘usability’. No instrument was specifically validated for innovative surgery. Three instruments were rated ‘sufficient’ (SURG-TLX) or ‘indeterminate’ (STAI,ISAT). Twenty professionals were interviewed (7 female; 15 specialties; 6 countries) and the focus group included 10 participants (4 professionals, 6 researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument.
Conclusion
The SURG-TLX is preliminarily recommended to measure operators’ experiences of innovation. Further work exploring its role and impact on surgical innovation is required.

Original language	English
Pages (from-to)	55-65
Number of pages	11
Journal	Journal of Clinical Epidemiology
Volume	153
DOIs	https://doi.org/10.1016/j.jclinepi.2022.10.006
Publication status	Published - 10 Oct 2022

Bibliographical note

Funding Information:
Sources of funding: This study was funded by an National Institute for Health and Care Research (NIHR) Clinician Scientists Fellowship award to A.M. (NIHR CS-2017-17-010). This work was further supported by the NIHR Biomedical Research Center (BRC) at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol (BRC-1215-20,011). The views expressed in this publication are those of the authors and not necessarily those of the NIHR. S.P. is an NIHR Clinician Scientist (NIHR CS-2016-16-019). J.B. is an NIHR Senior Investigator.

Publisher Copyright:
© 2022 The Authors

Access to Document

10.1016/j.jclinepi.2022.10.006Licence: CC BY

Full-text PDF (final published version)
This is the final published version of the article (version of record). It first appeared online via Elsevier at https://doi.org/10.1016/j.jclinepi.2022.10.006 . Please refer to any applicable terms of use of the publisher.
Final published version, 888 KBLicence: CC BY

Handle.net

Persistent link

Cite this

@article{ee2fae202dcc498697ccb3e95040658c,

title = "A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions",

abstract = "Objective During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators{\textquoteright} experiences (how professionals feel undertaking the innovation) drives this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators{\textquoteright} experience of surgical innovation. Study design and setting There were three phases: 1) Literature reviews identified measures of operators{\textquoteright} experience and concepts measured were extracted and grouped into domains. 2) Quality appraisal was conducted to assess content validity of identified instruments, and was supported by COSMIN methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COSMIN criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. 3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multi-disciplinary professionals and a focus group. Results Literature reviews identified 16 measurement instruments from 243 studies. Most assessed {\textquoteleft}psychological{\textquoteright} experiences and {\textquoteleft}usability{\textquoteright}. No instrument was specifically validated for innovative surgery. Three instruments were rated {\textquoteleft}sufficient{\textquoteright} (SURG-TLX) or {\textquoteleft}indeterminate{\textquoteright} (STAI,ISAT). Twenty professionals were interviewed (7 female; 15 specialties; 6 countries) and the focus group included 10 participants (4 professionals, 6 researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument. Conclusion The SURG-TLX is preliminarily recommended to measure operators{\textquoteright} experiences of innovation. Further work exploring its role and impact on surgical innovation is required.",

author = "McNair, \{Angus G K\} and Christin Hoffmann and Macefield, \{Rhiannon C\} and Daisy Elliott and Jane Blazeby and Avery, \{Kerry N L\} and Shelley Potter",

note = "Funding Information: Sources of funding: This study was funded by an National Institute for Health and Care Research (NIHR) Clinician Scientists Fellowship award to A.M. (NIHR CS-2017-17-010). This work was further supported by the NIHR Biomedical Research Center (BRC) at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol (BRC-1215-20,011). The views expressed in this publication are those of the authors and not necessarily those of the NIHR. S.P. is an NIHR Clinician Scientist (NIHR CS-2016-16-019). J.B. is an NIHR Senior Investigator. Publisher Copyright: {\textcopyright} 2022 The Authors",

year = "2022",

month = oct,

day = "10",

doi = "10.1016/j.jclinepi.2022.10.006",

language = "English",

volume = "153",

pages = "55--65",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

}

TY - JOUR

T1 - A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

AU - McNair, Angus G K

AU - Hoffmann, Christin

AU - Macefield, Rhiannon C

AU - Elliott, Daisy

AU - Blazeby, Jane

AU - Avery, Kerry N L

AU - Potter, Shelley

N1 - Funding Information: Sources of funding: This study was funded by an National Institute for Health and Care Research (NIHR) Clinician Scientists Fellowship award to A.M. (NIHR CS-2017-17-010). This work was further supported by the NIHR Biomedical Research Center (BRC) at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol (BRC-1215-20,011). The views expressed in this publication are those of the authors and not necessarily those of the NIHR. S.P. is an NIHR Clinician Scientist (NIHR CS-2016-16-019). J.B. is an NIHR Senior Investigator. Publisher Copyright: © 2022 The Authors

PY - 2022/10/10

Y1 - 2022/10/10

N2 - Objective During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators’ experiences (how professionals feel undertaking the innovation) drives this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators’ experience of surgical innovation. Study design and setting There were three phases: 1) Literature reviews identified measures of operators’ experience and concepts measured were extracted and grouped into domains. 2) Quality appraisal was conducted to assess content validity of identified instruments, and was supported by COSMIN methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COSMIN criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. 3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multi-disciplinary professionals and a focus group. Results Literature reviews identified 16 measurement instruments from 243 studies. Most assessed ‘psychological’ experiences and ‘usability’. No instrument was specifically validated for innovative surgery. Three instruments were rated ‘sufficient’ (SURG-TLX) or ‘indeterminate’ (STAI,ISAT). Twenty professionals were interviewed (7 female; 15 specialties; 6 countries) and the focus group included 10 participants (4 professionals, 6 researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument. Conclusion The SURG-TLX is preliminarily recommended to measure operators’ experiences of innovation. Further work exploring its role and impact on surgical innovation is required.

AB - Objective During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators’ experiences (how professionals feel undertaking the innovation) drives this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators’ experience of surgical innovation. Study design and setting There were three phases: 1) Literature reviews identified measures of operators’ experience and concepts measured were extracted and grouped into domains. 2) Quality appraisal was conducted to assess content validity of identified instruments, and was supported by COSMIN methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COSMIN criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. 3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multi-disciplinary professionals and a focus group. Results Literature reviews identified 16 measurement instruments from 243 studies. Most assessed ‘psychological’ experiences and ‘usability’. No instrument was specifically validated for innovative surgery. Three instruments were rated ‘sufficient’ (SURG-TLX) or ‘indeterminate’ (STAI,ISAT). Twenty professionals were interviewed (7 female; 15 specialties; 6 countries) and the focus group included 10 participants (4 professionals, 6 researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument. Conclusion The SURG-TLX is preliminarily recommended to measure operators’ experiences of innovation. Further work exploring its role and impact on surgical innovation is required.

U2 - 10.1016/j.jclinepi.2022.10.006

DO - 10.1016/j.jclinepi.2022.10.006

M3 - Article (Academic Journal)

C2 - 36228972

SN - 0895-4356

VL - 153

SP - 55

EP - 65

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

Abstract

Bibliographical note

Access to Document

Handle.net

Fingerprint

Cite this