VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction

Harvey Stedman; Basaran Bahadir Kocer; Mirko Kovac; Vijay M. Pawar

doi:10.1109/ISMAR-Adjunct57072.2022.00029

VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction

Harvey Stedman^*
, Basaran Bahadir Kocer
, Mirko Kovac
, Vijay M. Pawar

^*Corresponding author for this work

School of Civil, Aerospace and Design Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

12 Citations (Scopus)

Abstract

The availability of consumer virtual reality peripherals has led to a growing interest in developing immersive teleoperation interfaces that feature online 3D environment reconstruction of unknown environments. However, there are many unanswered questions around how such systems are used within deployed teleoperation missions and how operators navigate and explore the 3D data presented to them. We present VRTAB-Map, a configurable immersive teleoperation framework built using the RTAB-Map SLAM library. The framework can be configured with different sensor inputs allowing for it to be used as an experimental baseline for understanding operator behaviours under different teleoperation conditions and offering a methodology for real time immersive analytics in teleoperation missions. We analyse its performance during operation and, building upon our previous work in UAV teleoperation, showcase how it can be integrated in a realised teleoperation system featuring a UAV-UGV team. We then explore opportunities for future work that can be achieved with the presented framework including understanding human factors in teleoperation missions, analysing how variable environment representation effects teleoperation performance, and investigating how immersive interface design complements the operator's mental model of complex environments.

Original language	English
Title of host publication	Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	104-110
Number of pages	7
ISBN (Electronic)	9781665453653
DOIs	https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00029
Publication status	Published - 15 Dec 2022
Event	21st IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022 - Singapore, Singapore Duration: 17 Oct 2022 → 21 Oct 2022

Publication series

Name	IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct
ISSN (Print)	2771-1196
ISSN (Electronic)	2771-120X

Conference

Conference	21st IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022
Country/Territory	Singapore
City	Singapore
Period	17/10/22 → 21/10/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

Artificial intelligence
Computer systems organization
Computer vision
Computer vision problems
Computing methodologies
Embedded and cyber-physical systems
External interfaces for robotics
Human computer interaction (HCI)
Human machine interaction (HMI)
Human-centered computing
Human-computer interaction (HCI)
Interaction paradigms
Reconstruction
Robotics
Simultaneous localisation and mapping (SLAM)
Teleoperation
Telerobotics
Virtual Reality

Access to Document

10.1109/ISMAR-Adjunct57072.2022.00029Licence: Unspecified

https://discovery.ucl.ac.uk/id/eprint/10168226/

Handle.net

Persistent link

Cite this

Stedman, H., Kocer, B. B., Kovac, M., & Pawar, V. M. (2022). VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction. In Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022 (pp. 104-110). (IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct ). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00029

Stedman, Harvey ; Kocer, Basaran Bahadir ; Kovac, Mirko et al. / VRTAB-Map : A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction. Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022. Institute of Electrical and Electronics Engineers (IEEE), 2022. pp. 104-110 (IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct ).

@inproceedings{f29f2e13d7784f7480142d532b53598e,

title = "VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction",

abstract = "The availability of consumer virtual reality peripherals has led to a growing interest in developing immersive teleoperation interfaces that feature online 3D environment reconstruction of unknown environments. However, there are many unanswered questions around how such systems are used within deployed teleoperation missions and how operators navigate and explore the 3D data presented to them. We present VRTAB-Map, a configurable immersive teleoperation framework built using the RTAB-Map SLAM library. The framework can be configured with different sensor inputs allowing for it to be used as an experimental baseline for understanding operator behaviours under different teleoperation conditions and offering a methodology for real time immersive analytics in teleoperation missions. We analyse its performance during operation and, building upon our previous work in UAV teleoperation, showcase how it can be integrated in a realised teleoperation system featuring a UAV-UGV team. We then explore opportunities for future work that can be achieved with the presented framework including understanding human factors in teleoperation missions, analysing how variable environment representation effects teleoperation performance, and investigating how immersive interface design complements the operator's mental model of complex environments.",

keywords = "Artificial intelligence, Computer systems organization, Computer vision, Computer vision problems, Computing methodologies, Embedded and cyber-physical systems, External interfaces for robotics, Human computer interaction (HCI), Human machine interaction (HMI), Human-centered computing, Human-computer interaction (HCI), Interaction paradigms, Reconstruction, Robotics, Simultaneous localisation and mapping (SLAM), Teleoperation, Telerobotics, Virtual Reality",

author = "Harvey Stedman and Kocer, \{Basaran Bahadir\} and Mirko Kovac and Pawar, \{Vijay M.\}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 21st IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022 ; Conference date: 17-10-2022 Through 21-10-2022",

year = "2022",

month = dec,

day = "15",

doi = "10.1109/ISMAR-Adjunct57072.2022.00029",

language = "English",

series = "IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct ",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "104--110",

booktitle = "Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022",

address = "United States",

}

Stedman, H, Kocer, BB, Kovac, M & Pawar, VM 2022, VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction. in Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022. IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct , Institute of Electrical and Electronics Engineers (IEEE), pp. 104-110, 21st IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022, Singapore, Singapore, 17/10/22. https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00029

VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction. / Stedman, Harvey; Kocer, Basaran Bahadir; Kovac, Mirko et al.
Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022. Institute of Electrical and Electronics Engineers (IEEE), 2022. p. 104-110 (IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct ).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - VRTAB-Map

T2 - 21st IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022

AU - Stedman, Harvey

AU - Kocer, Basaran Bahadir

AU - Kovac, Mirko

AU - Pawar, Vijay M.

PY - 2022/12/15

Y1 - 2022/12/15

N2 - The availability of consumer virtual reality peripherals has led to a growing interest in developing immersive teleoperation interfaces that feature online 3D environment reconstruction of unknown environments. However, there are many unanswered questions around how such systems are used within deployed teleoperation missions and how operators navigate and explore the 3D data presented to them. We present VRTAB-Map, a configurable immersive teleoperation framework built using the RTAB-Map SLAM library. The framework can be configured with different sensor inputs allowing for it to be used as an experimental baseline for understanding operator behaviours under different teleoperation conditions and offering a methodology for real time immersive analytics in teleoperation missions. We analyse its performance during operation and, building upon our previous work in UAV teleoperation, showcase how it can be integrated in a realised teleoperation system featuring a UAV-UGV team. We then explore opportunities for future work that can be achieved with the presented framework including understanding human factors in teleoperation missions, analysing how variable environment representation effects teleoperation performance, and investigating how immersive interface design complements the operator's mental model of complex environments.

AB - The availability of consumer virtual reality peripherals has led to a growing interest in developing immersive teleoperation interfaces that feature online 3D environment reconstruction of unknown environments. However, there are many unanswered questions around how such systems are used within deployed teleoperation missions and how operators navigate and explore the 3D data presented to them. We present VRTAB-Map, a configurable immersive teleoperation framework built using the RTAB-Map SLAM library. The framework can be configured with different sensor inputs allowing for it to be used as an experimental baseline for understanding operator behaviours under different teleoperation conditions and offering a methodology for real time immersive analytics in teleoperation missions. We analyse its performance during operation and, building upon our previous work in UAV teleoperation, showcase how it can be integrated in a realised teleoperation system featuring a UAV-UGV team. We then explore opportunities for future work that can be achieved with the presented framework including understanding human factors in teleoperation missions, analysing how variable environment representation effects teleoperation performance, and investigating how immersive interface design complements the operator's mental model of complex environments.

KW - Artificial intelligence

KW - Computer systems organization

KW - Computer vision

KW - Computer vision problems

KW - Computing methodologies

KW - Embedded and cyber-physical systems

KW - External interfaces for robotics

KW - Human computer interaction (HCI)

KW - Human machine interaction (HMI)

KW - Human-centered computing

KW - Human-computer interaction (HCI)

KW - Interaction paradigms

KW - Reconstruction

KW - Robotics

KW - Simultaneous localisation and mapping (SLAM)

KW - Teleoperation

KW - Telerobotics

KW - Virtual Reality

U2 - 10.1109/ISMAR-Adjunct57072.2022.00029

DO - 10.1109/ISMAR-Adjunct57072.2022.00029

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85146054144

T3 - IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct

SP - 104

EP - 110

BT - Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 17 October 2022 through 21 October 2022

ER -

Stedman H, Kocer BB, Kovac M, Pawar VM. VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction. In Proceedings - 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2022. Institute of Electrical and Electronics Engineers (IEEE). 2022. p. 104-110. (IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct ). doi: 10.1109/ISMAR-Adjunct57072.2022.00029

VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Handle.net

Fingerprint

Cite this