Look Here! : Understanding the distribution of visual attention in sign language viewing

Relatively little was known about the nature of the viewing experience in sign language. There has been a great deal of analysis of the linguistic aspects of production and only supposition and hypothesis in regard to the reception of signed content. In the set of studies described below we have analysed the viewer experience and confirm and extend a set of predictions in regard to this aspect of the process. Deaf people concentrate on the mouth region in viewing even though BSL has a great deal of manual movement. There are differences between those who learned sign language from their parents and those who learned later, with more regulated eye gaze from the first of those. Although there are some initial indications that the linguistic process drives the eye gaze, there is a great deal more fine analysis to be carried out. Deaf viewers tend to tolerate a wide range of video representations of sign language but prefer CIF images at 25fps as the best image. Sign language density of information is concentrated in the right visual field for BSL and viewers seem to rely on the central five degrees of visual angle for the most of the information. Use of video object planes in MPEG4 video has allowed the examination of the relation between the attention to the face and the hands. Viewers tolerate three or four times less acuity allocation to the hands than to the face. This has significant implications for the allocation of bit rates to encoding in H264 applications. The results imply that significant savings can be made in H264 encoding without compromising the viewer satisfaction. Complex visual presentations provoke the same viewing behaviour with attention to the mouth/face and there is as yet limited evidence on the viewers need to detect the other stimuli in anything other than peripheral vision. The conclusion is simple – Deaf people need the mouth/face area to understand BSL and the motion of the hands is perceived primarily in peripheral vision.