Chapter 10: Ultrafast decoding from cells in the macaque monkey

Chou Hung and James DiCarlo

In a glance, a fraction of a second, our minds capture the visual scene.  We distinguish life from the inanimate, faces from background, the familiar and the unfamiliar.  The seeming ease of visual recognition belies the difficult of reading out the underlying neural code, much less the reconstruction of its mechanism in silico.  The computational difficulty of visual recognition is the combination of selectivity for specific objects and invariance across changes in viewing conditions.  Both properties have been shown to a limited extent for single neurons in the macaque anterior inferior temporal (AIT) cortex at the end of the ventral visual pathway.  An ongoing challenge is determining whether and how the activity of a population of such neurons is sufficient to encode object category and identity.  We recently showed, based on independent recordings from several hundred recording sites, that the brief (~12 msec) activity of a small population of AIT neurons is indeed sufficient to support recognition across changes in object size and position.  Remarkably, the combination of selectivity and tolerance also exists for novel objects.  This chapter will review the motivations and outcomes of that study and discuss recent work and major issues in developing effective read-out from IT cortex.

Keywords: classifier, inferotemporal cortex, single-unit recording, local field potentials, object recognition