Inferring Gestural Intent as a Model to Support Accessible Virtual Reality Interactions for People with Upper-Limbs Motor Impairments

As Virtual Reality (VR) has the potential to help individuals with mobility restrictions to interact in ways that may not be possible in the physical world, there is an increasing demand for VR among this population. Meanwhile, people with an upper-limbs motor impairment who cannot use handheld controllers and hand gestures have numerous concerns regarding the accessibility of VR. Although there are alternative gesture-based hand-free interactions, like facial and eyelid gestures, the evaluation studies show these approaches can still not efficiently and smoothly complete complex interactions in noisy environments. This paper thus proposes to research a computational framework to predict user intent and infer users’ potential intention by users’ behavioral cues. Based on the estimated probabilities computed by the multivariate adaptive regression spline (MARS) model, an interaction system
may display possible interactions for users to choose from. Then the system finishes user-selected interaction semi- or total- automatically. We are currently embarking on this project to explore the extent to which such an approach can simplify the interaction process in complex tasks and achieve intelligent hand-free interaction, leading to quicker, easier, more comfortable interactions for people with upper-limbs impairments.