Dynamics of Invariant Object Representations in the Monkey Inferotemporal Cortex

Abstract

Vision is computationally challenging because objects in the real world can change in size, position, viewpoint etc., and therefore cast a myriad images on the retina. Viewpoint changes are particularly challenging because new features can appear or disappear and existing features can be stretched or compressed. Even though humans are adept at recognizing objects across changes in viewpoint, the underlying neural representations are poorly understood. The goal of this thesis is to investigate viewpoint invariant object representations in the brain using recordings of single neurons in monkey visual cortex, and using behavioural experiments in humans. This thesis summarizes the results from a series of six experiments in which we recorded the responses of single neurons in the monkey inferior temporal cortex, an area critical for object recognition. In Experiment 1, we recorded neural responses to objects across two views and elucidated the dynamics of viewpoint invariance and the factors that modulate it. We observed a dramatic transition from view dependence in the early part of the neural response to view invariance in the later part. In Experiment 2, we investigated the effect of silhouetting and inversion on view invariance. In Experiment 3, we generalized our findings to multiple viewpoints and characterized view invariance for impoverished non-generic viewpoints and mirror views. In Experiment 4, we compared the magnitude and dynamics of viewpoint invariance with other known identity-preserving transformations such as size, position and rotation. In Experiment 5, we demonstrate that IT neurons potentially encode object features even after they rotate out of view. In Experiment 5, we demonstrate a generalization of view invariance, whereby neurons can decouple patterns across non-rigid surface changes. Taken together, our results reveal a dynamic picture of how view invariant representations are constructed in the brain to enable complex perceptual inferences.