Neural Representations of Visual Anagrams in Macaque Inferotemporal Cortex

Abstract

Visual recognition requires stable representations of object identity despite large changes in the retinal image. Yet this stability must be selective: some transformations change not only how an image looks, but also what object it appears to depict. Visual anagrams highlight this challenge: a single image can resemble two different objects at different orientations, such as an elephant and a rabbit. Here, we asked how macaque inferotemporal (IT) cortex represents such transformations. We recorded large neuronal populations while presenting visual anagrams and unambiguous exemplars of their two interpretations across rotations. Neurons preferring one exemplar tended to prefer anagram orientations supporting the corresponding interpretation. At each of the two interpretation-relevant canonical angles, population responses more closely resembled those to the matching exemplar than those to the nonmatching exemplar. This matching advantage persisted when both exemplars were presented at the alternative canonical angle. Decoders trained exclusively on exemplars across rotations also tracked the shifts in anagram representations. Visual models showed anagram-exemplar alignment as well, with stronger alignment in models with greater relative shape preference. These findings suggest that transformation tolerance in IT is sensitive to what an image appears to depict, while relative shape preference is associated with similar representational organization in visual models.