[Download thesis in Adobe Acrobat format]

Visual Representations in a Natural Visuo-motor Task

By: Jeff Bennett Pelz
Carlson Center for Imaging Science, Rochester Institute of Technology

Advisors: Mary M. Hayhoe & Dana H. Ballard
Department of Brain and Cognitive Sciences, University of Rochester


Table of contents


This thesis investigates the role of foveating eye movements in vision. A block-copying task, in which fixations are tied closely to perceptual, cognitive, and motor events, was used to explore visual representations and visuo-motor coordination in the context of complex, ongoing behaviors. The experiments required the development of a laboratory facility capable of monitoring concurrent eye, head, and hand movements at a fine time-scale. The primary finding was that subjects made frequent eye movements to serialize the task into simpler subtasks which were executed sequentially, minimizing working memory load. This suggests that only a sparse, transient representation of task-relevant information is maintained, rather than an extensive, task-independent reconstruction of the visual world which is commonly assumed to be the goal of vision. A series of experiments in which the 'cost' of fixating a pattern to be copied was increased, or the information content of the pattern was reduced, demonstrated that the trade-off between frequent eye movements and working memory load is flexible, and that subjects are capable of dynamically adjusting that balance based on task demands.

Measurements of subjects' eye and head movements during the block-copying task led to the observation that some subjects executed independent eye and head movements, dissociating their spatial and temporal trajectories, an observation inconsistent with current models of eye and head movements that postulate a common gaze shift goal. In another series of experiments, subjects performed the block-copying task while performing one of two secondary tasks. The added cognitive load led to dramatic changes in subjects' head trajectories, and affected the strategies subjects used in constructing the duplicate pattern.

The experiments provide support for a different approach to studying visual processing, in which vision is viewed as more top-down than previously supposed. The task takes on particular importance, because behavior cannot be divorced from the immediate task(s). Fixations are shown to play crucial cognitive roles in perception, including binding task-relevant information to variables in working memory, and indexing the execution of sequential programs.

Table of Contents

Curriculum Vitae

The author was born in Palo Alto, California on March 25, 1956, and grew up in Ann Arbor, Michigan. He attended the Rochester Institute of Technology, earning a Bachelor of Fine Arts degree in Fine Arts Photography in 1980 and a Masters of Science in Imaging Science in 1986. He began his studies at the University of Rochester in the fall of 1990 at the Center for Visual Science. He pursued his research under the direction of Professor Hayhoe, and received his Master of Arts degree in 1993. He is currently on the faculty of the Center for Imaging Science at the Rochester Institute of Technology.


A great many people contributed, directly and indirectly, to the body of work that makes up this dissertation. The guidance and encouragement provided by Mary Hayhoe and Dana Ballard were invaluable. Discussions with Steven Shimozaki, Keith Karn, Per Möeller, and Michael Spivey also helped shape the experiments. Early work by Mary Hayhoe, Dana Ballard, Steven Whitehead and Feng Li provided the base on which this work was built. The work was only possible because Alan Russell, Andrew Forsberg, Deborah Bancroft, and Timothy Becker shared their ideas and skills so generously. Portions of the work represented in this thesis have been supported by NIH RO1 EY05729, NIH R24 RR06853, and AFOSR-91-0332. Additional support was received from the Rochester Institute of Technology's Center for Imaging Science Industrial Associates Program. Finally, the whole experience could never have taken place without the love, support and patience of Myra, Rachel, and Madeline Pelz.