A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Pai Zheng; Shufei Li; Liqiao Xia; Lihui Wang; Aydin Nassehi

doi:10.1016/j.cirp.2022.04.016

A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Pai Zheng, Shufei Li, Liqiao Xia, Lihui Wang^*, Aydin Nassehi

^*Corresponding author for this work

Department of Mechanical Engineering

Research output: Contribution to journal › Article (Academic Journal) › peer-review

49 Citations (Scopus)

72 Downloads (Pure)

Abstract

Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.

Original language	English
Pages (from-to)	377-380
Number of pages	4
Journal	CIRP Annals
Volume	71
Issue number	1
Early online date	26 Apr 2022
DOIs	https://doi.org/10.1016/j.cirp.2022.04.016
Publication status	Published - 12 Jul 2022

Bibliographical note

Publisher Copyright:
© 2022 The Author(s)

Keywords

Human robot collaboration
Manufacturing system
Visual reasoning

Access to Document

10.1016/j.cirp.2022.04.016Licence: CC BY

Full-text PDF (final published version)
This is the final published version of the article (version of record). It first appeared online via Elsevier at https://doi.org/10.1016/j.cirp.2022.04.016 . Please refer to any applicable terms of use of the publisher.
Final published version, 2.15 MBLicence: CC BY

Handle.net

Persistent link

Cite this

@article{22fc09143e3e4c88860517275877e2ca,

title = "A visual reasoning-based approach for mutual-cognitive human-robot collaboration",

abstract = "Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.",

keywords = "Human robot collaboration, Manufacturing system, Visual reasoning",

author = "Pai Zheng and Shufei Li and Liqiao Xia and Lihui Wang and Aydin Nassehi",

note = "Publisher Copyright: {\textcopyright} 2022 The Author(s)",

year = "2022",

month = jul,

day = "12",

doi = "10.1016/j.cirp.2022.04.016",

language = "English",

volume = "71",

pages = "377--380",

journal = "CIRP Annals",

issn = "0007-8506",

publisher = "Amsterdam:Elsevier",

number = "1",

}

TY - JOUR

T1 - A visual reasoning-based approach for mutual-cognitive human-robot collaboration

AU - Zheng, Pai

AU - Li, Shufei

AU - Xia, Liqiao

AU - Wang, Lihui

AU - Nassehi, Aydin

PY - 2022/7/12

Y1 - 2022/7/12

N2 - Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.

AB - Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.

KW - Human robot collaboration

KW - Manufacturing system

KW - Visual reasoning

UR - http://www.scopus.com/inward/record.url?scp=85130041144&partnerID=8YFLogxK

U2 - 10.1016/j.cirp.2022.04.016

DO - 10.1016/j.cirp.2022.04.016

M3 - Article (Academic Journal)

AN - SCOPUS:85130041144

SN - 0007-8506

VL - 71

SP - 377

EP - 380

JO - CIRP Annals

JF - CIRP Annals

IS - 1

ER -

A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Abstract

Bibliographical note

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this