The role of reinforcement in "optimal" search strategies: Meeting abstract presented at VSS 2016

Highly trained observers can show near-optimal visual search strategies, similar to those of a Bayesian ideal observer. Based on two participants, Najemnik & Geisler (2005, 2008) demonstrate that optimal search can be identified with a stereotypical pattern of eye movements when searching for a target (short horizontal saccades and occasional jumps into "unknown" vertical space). We explored the optimality of visual search by conducting a study based on the previous search task, involving searching for a small gabor target in a circular background of 1/f noise. Twenty participants completed 160 trials, distributed over four blocks with four different target contrasts. We fitted a 2-state hidden Markov model to test whether the resulting temporal saccade distribution was consistent with the optimal pattern but, contrary to expectation, found idiosyncratic rather than optimal search strategies. We therefore propose a simpler model for visual search where optimality is achieved, not by solving a difficult optimisation problem, but by the application of reinforcement learning. In particular, the application of "reinforce" class of models that has proved successful in artificial models of attention. We tested for evidence of reinforcement by adding a training phase with altered reward statistics to the previously described search task. In the training phase, the target was revealed only after a leftward saccade. Fifteen participants completed 252 trials, distributed over six blocks, the middle four blocks being the training phase. The participants, although naïve to this intervention, demonstrated a clear, significant increase in leftward saccades from pre- to post-training. These findings provide strong evidence for reinforcement learning being involved in optimising visual search strategies and provide a simple explanation of "optimal" behaviour previously found in some observers.