EchoFlex: Hand Gesture Recognition using Ultrasound Imaging

Jess McIntosh, Asier Marzo, Mike Fraser, Carol Phillips

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

43 Citations (Scopus)


Recent improvements in ultrasound imaging enable new opportunities for hand pose detection using wearable devices. Ultrasound imaging has remained under-explored in the HCI community despite being non-invasive, harmless and capable of imaging internal body parts, with applications including smart-watch interaction, prosthesis control and instrument tuition. In this paper, we compare the performance of different forearm mounting positions for a wearable ultrasonographic device. Location plays a fundamental role in ergonomics and performance since the anatomical features differ among positions. We also investigate the performance decrease due to cross-session position shifts and develop a technique to compensate for this misalignment. Our gesture recognition algorithm combines image processing and neural networks to classify the flexion and extension of 10 discrete hand gestures with an accuracy above 98%. Furthermore, this approach can continuously track individual digit flexion with less than 5% NRMSE, and also differentiate between digit flexion at different joints.
Original languageEnglish
Title of host publicationProceedings of the 35th Annual ACM Conference on Human Factors in Computing Systems (CHI'17)
PublisherAssociation for Computing Machinery (ACM)
Number of pages1934
Publication statusPublished - 2 May 2017
Event35th Annual CHI Conference on Human Factors in Computing Systems, CHI 2017: Explore Innovate Inspire - Colorado Convention Center, Denver, United States
Duration: 6 May 201711 May 2017
Conference number: 35


Conference35th Annual CHI Conference on Human Factors in Computing Systems, CHI 2017
Abbreviated titleCHI 2017
CountryUnited States
OtherThe ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction. We are very fortunate to hold CHI 2017 in Denver, in one of the most beautiful regions in the U.S., near the foothills of the Rocky Mountains. This location served to motivate our conference theme of Explore, Innovate, Inspire. Mountains have long been a theme connected to exploration, the natural environment has provided models for numerous innovations, and the western landscape has served as a source of inspiration to many, from artists to inventors. The CHI conference brings together people from multiple disciplines and a diversity of cultures to explore new ways to practice, develop and improve methods and systems in HCI, to discuss ideas that can lead to innovations, and to inspire us all.
Internet address


Dive into the research topics of 'EchoFlex: Hand Gesture Recognition using Ultrasound Imaging'. Together they form a unique fingerprint.

Cite this