Cognitive Neuroscience

• Gagliardi, C.M., Normandin, M.E., Keinath, A.T., Julian, J.B., Lopez, M.R., Ramos-Alvarez, M.M., Epstein,R.A., & Muzzio,I.A. (2024). Distinct neural mechanisms for heading retrieval and context recognition in the hippocampus during spatial reorientation. Nature Communications, 15: 5968.
  • Distinct neural mechanisms for heading retrieval and context recognition in the hippocampus during spatial reorientation.
    Reorientation, the process of regaining one’s bearings after becoming lost, requires identification of a spatial context (context recognition) and recovery of facing direction within that context (heading retrieval). We previously showed that these processes rely on the use of features and geometry, respectively. Here, we examine reorientation behavior in a task that creates contextual ambiguity over a long timescale to demonstrate that male mice learn to combine both featural and geometric cues to recover heading. At the neural level, most CA1 neurons persistently align to geometry, and this alignment predicts heading behavior. However, a small subset of cells remaps coherently in a context-sensitive manner, which serves to predict context. Efficient heading retrieval and context recognition correlate with rate changes reflecting integration of featural and geometric information in the active ensemble. These data illustrate how context recognition and heading retrieval are coded in CA1 and how these processes change with experience.
• Peer, M., Nadar, C., & Epstein, R. A. (2023). The format of the cognitive map depends on the structure of the environment. Journal of Experimental Psychology: General, in press.
  • The format of the cognitive map depends on the structure of the environment.
    Humans and animals form cognitive maps that allow them to navigate through large-scale environments. Despite decades of research on these maps, a central question remains unclear: are these maps similar in nature across all environments, or are different kinds of maps formed in different kinds of environments? To investigate this, we examined spatial learning within three virtual environments: an open courtyard with patios connected by paths (open maze), a set of rooms connected by corridors (closed maze), and a set of isolated rooms connected only by teleporters (teleport maze). Importantly, all three environments shared the same topological graph structure. Post-learning tests showed that the environmental structure affected the accuracy, format, and variability of participants’ spatial representations. The open maze was the most accurately remembered, followed by the closed maze, and then the teleport maze. Both Euclidean and graph-like spatial codes were formed in the open and closed mazes, but participants’ navigational trajectories were more biased by graph knowledge (connectivity between rooms) in the closed maze compared to the open maze. Finally, performance in the open maze and teleport maze were relatively homogenous across participants, whereas performance in the closed maze exhibited greater individual variability. These results indicate that the structure of the environment strongly shapes the nature of the spatial representations that are formed within that environment, and that experimental findings obtained in any single environment may not generalize to others with different structure.
• Raithel, C. U., Miller, A. J., Epstein, R. A., Kahnt, T., & Gottfried, J. A. (2023). Recruitment of grid-like responses in human entorhinal and piriform cortices by odor landmark-based navigation. Current Biology, 33: 1-10.
  • Recruitment of grid-like responses in human entorhinal and piriform cortices by odor landmark-based navigation.
    Olfactory navigation is universal across the animal kingdom. Humans, however, have rarely been considered in this context. Here, we combined olfactometry techniques, virtual reality (VR) software, and neuroimaging methods to investigate whether humans can navigate an olfactory landscape by learning the spatial relationships among discrete odor cues and integrating this knowledge into a spatial map. Our data show that over time, participants improved their performance on the odor navigation task by taking more direct paths toward targets and completing more trials within a given time period. This suggests that humans can successfully navigate a complex odorous environment, reinforcing the notion of human olfactory navigation. fMRI data collected during the olfactory navigation task revealed the emergence of grid-like responses in entorhinal and piriform cortices that were attuned to the same grid orientation. This result implies the existence of a specialized olfactory grid network tasked with guiding spatial navigation based on odor landmarks.
• Brunec, I., Nantais, M., Sutton, J., Epstein, R. A., & Newcombe, N. S. (2023). Exploration patterns shape cognitive map learning. Cognition, 233: 105360.
  • Exploration patterns shape cognitive map learning.
    Spontaneous, volitional spatial exploration is crucial for building up a cognitive map of the environment. However, decades of research have primarily measured the fidelity of cognitive maps after discrete, controlled learning episodes. We know little about how cognitive maps are formed during naturalistic free exploration. Here, we investigated whether exploration trajectories predicted cognitive map accuracy, and how these patterns were shaped by environmental structure. In two experiments, participants freely explored a previously unfamiliar virtual environment. We related their exploration trajectories to a measure of how long they spent in areas with high global environmental connectivity (integration, as assessed by space syntax). In both experiments, we found that participants who spent more time on paths that offered opportunities for integration formed more accurate cognitive maps. Interestingly, we found no support for our pre-registered hypothesis that self-reported trait differences in navigation ability would mediate this relationship. Our findings suggest that exploration patterns predict cognitive map accuracy, even for people who self-report low ability, and highlight the importance of considering both environmental structure and individual variability in formal theory-and model-building.
• Harel, A., Nandor, J. D., Bonner, M. F., & Epstein, R. A. (2022). Early Electrophysiological Markers of Navigational Affordances in Scenes. Journal of Cognitive Neuroscience, 34(3): 397-410.
  • Early Electrophysiological Markers of Navigational Affordances in Scenes.
    Scene perception and spatial navigation are interdependent cognitive functions, and there is increasing evidence that cortical areas that process perceptual scene properties also carry information about the potential for navigation in the environment (navigational affordances). However, the temporal stages by which visual information is transformed into navigationally relevant information are not yet known. We hypothesized that navigational affordances are encoded during perceptual processing and therefore should modulate early visually evoked ERPs, especially the scene-selective P2 component. To test this idea, we recorded ERPs from participants while they passively viewed computer-generated room scenes matched in visual complexity. By simply changing the number of doors (0 doors, 1 door, 2 doors, 3 doors), we were able to systematically vary the number of pathways that afford movement in the local environment, while keeping the overall size and shape of the environment constant. We found that rooms with 0 doors evoked a higher P2 response than rooms with three doors, consistent with prior research reporting higher P2 amplitude to closed relative to open scenes. Moreover, we found P2 amplitude scaled linearly with the number of doors in the scenes. Navigability effects on the ERP waveform were also observed in a multivariate analysis, which showed significant decoding of the number of doors and their location at earlier time windows. Together, our results suggest that navigational affordances are represented in the early stages of scene perception. This complements research showing that the occipital place area automatically encodes the structure of navigable space and strengthens the link between scene perception and navigation.
• Sun, L., Frank, S. M., Epstein, R.A., & Tse, P. U. (2021). The parahippocampal place area and hippocampus encode the spatial significance of landmark objects. NeuroImage, 236: 118081.
  • The parahippocampal place area and hippocampus encode the spatial significance of landmark objects.
    Landmark objects are points of reference that can anchor one's internal cognitive map to the external world while navigating. They are especially useful in indoor environments where other cues such as spatial geometries are often similar across locations. We used functional magnetic resonance imaging (fMRI) and multivariate pattern analysis (MVPA) to understand how the spatial significance of landmark objects is represented in the human brain. Participants learned the spatial layout of a virtual building with arbitrary objects as unique landmarks in each room during a navigation task. They were scanned while viewing the objects before and after learning. MVPA revealed that the neural representation of landmark objects in the right parahippocampal place area (rPPA) and the hippocampus transformed systematically according to their locations. Specifically, objects in different rooms became more distinguishable than objects in the same room. These results demonstrate that rPPA and the hippocampus encode the spatial significance of landmark objects in indoor spaces.
• Peer, M. & Epstein, R. A. (2021). The human brain uses spatial schemas to represent segmented environments. Current Biology, 31: 1-12.
  • The human brain uses spatial schemas to represent segmented environments.
    Humans and animals use cognitive maps to represent the spatial structure of the environment. Although these maps are typically conceptualized as extending in an equipotential manner across known space, psychological evidence suggests that people mentally segment complex environments into subspaces. To understand the neurocognitive mechanisms behind this operation, we familiarized participants with a virtual courtyard that was divided into two halves by a river; we then used behavioral testing and fMRI to understand how spatial locations were encoded within this environment. Participants’ spatial judgments and multivoxel activation patterns were affected by the division of the courtyard, indicating that the presence of a boundary can induce mental segmentation even when all parts of the environment are co-visible. In the hippocampus and occipital place area (OPA), the segmented organization of the environment manifested in schematic spatial codes that represented geometrically equivalent locations in the two subspaces as similar. In the retrosplenial complex (RSC), responses were more consistent with an integrated spatial map. These results demonstrate that people use both local spatial schemas and integrated spatial maps to represent segmented environment. We hypothesize that schematization may serve as a general mechanism for organizing complex knowledge structures in terms of their component elements.
• Bonner, M. F. & Epstein, R. A. (2021). Object representations in the human brain reflect the co-occurrence statistics of vision and language Nature Communications, 12: 4081.
  • Object representations in the human brain reflect the co-occurrence statistics of vision and language.
    A central regularity of visual perception is the co-occurrence of objects in the natural environment. Here we use machine learning and fMRI to test the hypothesis that object cooccurrence statistics are encoded in the human visual system and elicited by the perception of individual objects. We identified low-dimensional representations that capture the latent statistical structure of object co-occurrence in real-world scenes, and we mapped these statistical representations onto voxel-wise fMRI responses during object viewing. We found that cortical responses to single objects were predicted by the statistical ensembles in which they typically occur, and that this link between objects and their visual contexts was made most strongly in parahippocampal cortex, overlapping with the anterior portion of sceneselective parahippocampal place area. In contrast, a language-based statistical model of the co-occurrence of object names in written text predicted responses in neighboring regions of object-selective visual cortex. Together, these findings show that the sensory coding of objects in the human brain reflects the latent statistics of object context in visual and linguistic experience.
• Peer, M., Brunec, I. K., Newcombe, N. S., & Epstein, R. A. (2021). Structuring knowledge with cognitive maps and cognitive graphs. Trends in Cognitive Sciences, 25(1): 37-54.
  • Structuring knowledge with cognitive maps and cognitive graphs.
    Humans and animals use mental representations of the spatial structure of the world to navigate. The classical view is that these representations take the form of Euclidean cognitive maps, but alternative theories suggest that they are cognitive graphs consisting of locations connected by paths. We review evidence suggesting that both map-like and graph-like representations exist in the mind/brain that rely on partially overlapping neural systems. Maps and graphs can operate simultaneously or separately, and they may be applied to both spatial and nonspatial knowledge. By providing structural frameworks for complex information, cognitive maps and cognitive graphs may provide fundamental organizing schemata that allow us to navigate in physical, social, and conceptual spaces.
• Keinath, A. T., Rechnitz, O., Balasubramanian, V., & Epstein, R. A. (2021). Environmental deformations dynamically shift human spatial memory. Hippocampus, 31(1): 89-101.
  • Environmental deformations dynamically shift human spatial memory.
    Place and grid cells in the hippocampal formation are commonly thought to support a unified and coherent cognitive map of space. This mapping mechanism faces a challenge when a navigator is placed in a familiar environment that has been deformed from its original shape. Under such circumstances, many transformations could plausibly serve to map a navigator's familiar cognitive map to the deformed space. Previous empirical results indicate that the firing fields of rodent place and grid cells stretch or compress in a manner that approximately matches the environmental deformation, and human spatial memory exhibits similar distortions. These effects have been interpreted as evidence that reshaping a familiar environment elicits an analogously reshaped cognitive map. However, recent work has suggested an alternative explanation, whereby deformation-induced distortions of the grid code are attributable to a mechanism that dynamically anchors grid fields to the most recently experienced boundary, thus causing history-dependent shifts in grid phase. This interpretation raises the possibility that human spatial memory will exhibit similar history-dependent dynamics. To test this prediction, we taught participants the locations of objects in a virtual environment and then probed their memory for these locations in deformed versions of this environment. Across three experiments with variable access to visual and vestibular cues, we observed the predicted pattern, whereby the remembered locations of objects were shifted from trial to trial depending on the boundary of origin of the participant's movement trajectory. These results provide evidence for a dynamic anchoring mechanism that governs both neuronal firing and spatial memory.
• Epstein, R. A. (2020). Spatial knowledge and navigation. In D. Poeppel, G. R. Mangun & M. S. Gazzaniga, (Eds), The Cognitive Neurosciences, 6th Edition. Cambridge, MA: MIT Press.
  • Spatial knowledge and navigation.
    Spatial knowledge is knowledge about where things are in the world and how they are spatially related to each other. One important use of spatial knowledge is to guide navigation from place to place. To accomplish this function, the brain must represent navigationally relevant aspects of the local environment, such as landmarks, scene geometry, and local navigational affordances. It must also form representations of the space beyond the current sensory horizon, which might take the form of a cognitive map or graph. Research to date indicates that representations of the local environmental are supported primarily by scene-responsive regions in higher visual cortex, such as the parahippocampal place area (PPA), occipital place area (OPA), and retrosplenial complex (RSC). Global spatial representations, on the other hand, are supported primarily by the hippocampal formation and the RSC. A key challenge for the field, which this chapter attempts to address, is to understand how the spatial knowledge representations revealed by cognitive behavioral studies are mediated by neural systems.
• Epstein, R.A. & Baker, C.I. (2019). Scene perception in the human brain. Annual Review of Vision Science, 5.
  • Scene perception in the human brain.
    Humans are remarkably adept at perceiving and understanding complex real-world scenes. Uncovering the neural basis of this ability is an important goal of vision science. Neuroimaging studies have identified three cortical regions that respond selectively to scenes: parahippocampal place area, retrosplenial complex/medial place area, and occipital place area. Here, we review what is known about the visual and functional properties of these brain areas. Scene-selective regions exhibit retinotopic properties and sensitivity to low-level visual features that are characteristic of scenes. They also mediate higher-level representations of layout, objects, and surface properties that allow individual scenes to be recognized and their spatial structure ascertained. Challenges for the future include developing computational models of information processing in scene regions, investigating how these regions support scene perception under ecologically realistic conditions, and understanding how they operate in the context of larger brain networks.
• Keinath, A.T., Epstein, R.A. & Balasubramanian, V. (2018). Environmental deformations dynamically shift the grid cell spatial metric. eLife, 7: e38169.
  • Environmental deformations dynamically shift the grid cell spatial metric.
    In familiar environments, the firing fields of entorhinal grid cells form regular triangular lattices. However, when the geometric shape of the environment is deformed, these time-averaged grid patterns are distorted in a grid scale-dependent and local manner. We hypothesized that this distortion in part reflects dynamic anchoring of the grid code to displaced boundaries, possibly through border cell-grid cell interactions. To test this hypothesis, we first reanalyzed two existing rodent grid rescaling datasets to identify previously unrecognized boundary-tethered shifts in grid phase that contribute to the appearance of rescaling. We then demonstrated in a computational model that boundary-tethered phase shifts, as well as scale-dependent and local distortions of the time-averaged grid pattern, could emerge from border-grid interactions without altering inherent grid scale. Together, these results demonstrate that environmental deformations induce historydependent shifts in grid phase, and implicate border-grid interactions as a potential mechanism underlying these dynamics.
• Mattar, M.G., Olkkonen, M., Epstein, R.A. & Aguirre, G.K. (2018). Adaptation decorrelates shape representations. Nature Communications, 9: 3812.
  • Adaptation decorrelates shape representations.
    Perception and neural responses are modulated by sensory history. Visual adaptation, an example of such an effect, has been hypothesized to improve stimulus discrimination by decorrelating responses across a set of neural units. While a central theoretical model, behavioral and neural evidence for this theory is limited and inconclusive. Here, we use a parametric 3D shape-space to test whether adaptation decorrelates shape representations in humans. In a behavioral experiment with 20 subjects, we find that adaptation to a shape class improves discrimination of subsequently presented stimuli with similar features. In a BOLD fMRI experiment with 10 subjects, we observe that adaptation to a shape class decorrelates the multivariate representations of subsequently presented stimuli with similar features in object-selective cortex. These results support the long-standing proposal that adaptation improves perceptual discrimination and decorrelates neural representations, offering insights into potential underlying mechanisms.
• Julian, J.B., Keinath, A.T., Marchette, S.A. & Epstein, R.A. (2018). The neurocognitive basis of spatial reorientation. Current Biology, 28: R1059–R1073.
  • The neurocognitive basis of spatial reorientation.
    The ability to recover one’s bearings when lost is a skill that is fundamental for spatial navigation. We review the cognitive and neural mechanisms that underlie this ability, with the aim of linking together previously disparate findings from animal behavior, human psychology, electrophysiology, and cognitive neuroscience. Behavioral work suggests that reorientation involves two key abilities: first, the recovery of a spatial reference frame (a cognitive map) that is appropriate to the current environment; and second, the determination of one’s heading and location relative to that reference frame. Electrophysiological recording studies, primarily in rodents, have revealed potential correlates of these operations in place, grid, border/boundary, and head-direction cells in the hippocampal formation. Cognitive neuroscience studies, primarily in humans, suggest that the perceptual inputs necessary for these operations are processed by neocortical regions such as the retrosplenial complex, occipital place area and parahippocampal place area, with the retrosplenial complex mediating spatial transformations between the local environment and the recovered spatial reference frame, the occipital place area supporting perception of local boundaries, and the parahippocampal place area processing visual information that is essential for identification of the local spatial context. By combining results across these various literatures, we converge on a unified account of reorientation that bridges the cognitive and neural domains.
• Julian, J.B., Kamps, F.S., Epstein, R.A., & Dilks, D.D. (2018). Dissociable spatial memory systems revealed by typical and atypical human development. Developmental science, e12737.
  • Dissociable spatial memory systems revealed by typical and atypical human development.
    Rodent lesion studies have revealed the existence of two causally dissociable spatial memory systems, localized to the hippocampus and striatum that are preferentially sensitive to environmental boundaries and landmark objects, respectively. Here we test whether these two memory systems are causally dissociable in humans by examining boundary and landmark-based memory in typical and atypical development. Adults with Williams syndrome (WS)—a developmental disorder with known hippocampal abnormalities—and typical children and adults, performed a navigation task that involved learning locations relative to a boundary or a landmark object. We found that boundary-based memory was severely impaired in WS compared to typically-developing mental-age matched (MA) children and chronological-age matched (CA) adults, whereas landmark-based memory was similar in all groups. Furthermore, landmark-based memory matured earlier in typical development than boundary-based memory, consistent with the idea that the WS cognitive phenotype arises from developmental arrest of late maturing cognitive systems. Together, these findings provide causal and developmental evidence for dissociable spatial memory systems in humans.
• Bonner, M.F. & Epstein, R.A. (2018). Computational mechanisms underlying cortical responses to the affordance properties of visual scenes. PLoS Comput Biol, 14(4): e1006111.
  • Computational mechanisms underlying cortical responses to the affordance properties of visual scenes.
    Biologically inspired deep convolutional neural networks (CNNs), trained for computer vision tasks, have been found to predict cortical responses with remarkable accuracy. However, the internal operations of these models remain poorly understood, and the factors that account for their success are unknown. Here we develop a set of techniques for using CNNs to gain insights into the computational mechanisms underlying cortical responses. We focused on responses in the occipital place area (OPA), a scene-selective region of dorsal occipitoparietal cortex. In a previous study, we showed that fMRI activation patterns in the OPA contain information about the navigational affordances of scenes; that is, information about where one can and cannot move within the immediate environment. We hypothesized that this affordance information could be extracted using a set of purely feedforward computations. To test this idea, we examined a deep CNN with a feedforward architecture that had been previously trained for scene classification. We found that responses in the CNN to scene images were highly predictive of fMRI responses in the OPA. Moreover the CNN accounted for the portion of OPA variance relating to the navigational affordances of scenes. The CNN could thus serve as an image-computable candidate model of affordance-related responses in the OPA. We then ran a series of in silico experiments on this model to gain insights into its internal operations. These analyses showed that the computation of affordance-related features relied heavily on visual information at high-spatial frequencies and cardinal orientations, both of which have previously been identified as low- level stimulus preferences of scene-selective visual cortex. These computations also exhibited a strong preference for information in the lower visual field, which is consistent with known retinotopic biases in the OPA. Visualizations of feature selectivity within the CNN suggested that affordance-based responses encoded features that define the layout of the spatial environment, such as boundary-defining junctions and large extended surfaces. Together, these results map the sensory functions of the OPA onto a fully quantitative model that provides insights into its visual computations. More broadly, they advance inte- grative techniques for understanding visual cortex across multiple level of analysis: from the identification of cortical sensory functions to the modeling of their underlying algorithms.
• Julian, J.B., Keinath, A.T., Frazzetta, G. & Epstein, R.A. (2018). Human entorhinal cortex represents visual space using a boundary-anchored grid. Nature Neuroscience, 21(2): 191-194.
  • Human entorhinal cortex represents visual space using a boundary-anchored grid.
    When participants performed a visual search task, functional MRI responses in entorhinal cortex exhibited a sixfold periodic modulation by gaze-movement direction. The orientation of this modulation was determined by the shape and orientation of the bounded search space. These results indicate that human entorhinal cortex represents visual space using a boundary-anchored grid, analogous to that used by rodents to represent navigable space.
• Epstein, R.A., Patai, E.Z., Julian, J.B. & Spiers, H.J. (2017). The cognitive map in humans: spatial navigation and beyond. Nature Neuroscience, 20(11): 1504-1513.
  • The cognitive map in humans: spatial navigation and beyond.
    The ‘cognitive map’ hypothesis proposes that brain builds a unified representation of the spatial environment to support memory and guide future action. Forty years of electrophysiological research in rodents suggest that cognitive maps are neurally instantiated by place, grid, border and head direction cells in the hippocampal formation and related structures. Here we review recent work that suggests a similar functional organization in the human brain and yields insights into how cognitive maps are used during spatial navigation. Specifically, these studies indicate that (i) the human hippocampus and entorhinal cortex support map-like spatial codes, (ii) posterior brain regions such as parahippocampal and retrosplenial cortices provide critical inputs that allow cognitive maps to be anchored to fixed environmental landmarks, and (iii) hippocampal and entorhinal spatial codes are used in conjunction with frontal lobe mechanisms to plan routes during navigation. We also discuss how these three basic elements of cognitive map based navigation—spatial coding, landmark anchoring and route planning—might be applied to nonspatial domains to provide the building blocks for many core elements of human thought.
• Olkkonen, M., Aguirre, G.K. & Epstein, R.A. (2017). Expectation modulates repetition priming under high stimulus variability. Journal of Vision, 17(6): 10, 1-16.
  • Expectation modulates repetition priming under high stimulus variability.
    Neural responses to stimuli are often attenuated by repeated presentation. When observed in blood oxygen level-dependent signals, this attenuation is known as fMRI adaptation (fMRIa) or fMRI repetition suppression. According to a prominent account, fMRIa reflects the fulfillment of perceptual expectations during recognition of repeated items (Summerfield, Trittschuh, Monti, Mesulam, & Egner, 2008). Supporting this idea, expectation has been shown to modulate fMRIa under some circumstances; however, it is not currently known whether expectation similarly modulates recognition performance. To address this lacuna, we measured behavioral and fMRI responses to faces while varying the extent to which each stimulus was informative about its successor. Behavioral priming was greater when repetitions were more likely, suggesting that recognition was facilitated by the expectation than an item would repeat. Notably, this effect was only observed when stimuli were drawn from a broad set of faces including many ethnicities and both genders, but not when stimuli were drawn from a narrower face set, thus making repetitions less informative. Moreover, expectation did not modulate fMRIa in face-selective cortex, contrary to previous studies, although an exploratory analysis indicated that it did so in a medial frontal region. These results support the idea that expectation modulates recognition efficiency, but insofar as behavioral effects of expectation were not accompanied by fMRI effects in visual cortex, they suggest that fMRIa cannot be entirely explained in terms of fulfilled expectations.
• Bonner, M.F. & Epstein, R.A. (2017). Coding of navigational affordances in the human visual system. Proceedings of the National Academy of Sciences, 114(18): 4793-4798.
  • Coding of navigational affordances in the human visual system.
    A central component of spatial navigation is determining where one can and cannot go in the immediate environment. We used fMRI to test the hypothesis that the human visual system solves this problem by automatically identifying the navigational affor- dances of the local scene. Multivoxel pattern analyses showed that a scene-selective region of dorsal occipitoparietal cortex, known as the occipital place area, represents pathways for movement in scenes in a manner that is tolerant to variability in other visual features. These effects were found in two experiments: One using tightly controlled artificial environments as stimuli, the other using a diverse set of complex, natural scenes. A reconstruction analysis demonstrated that the population codes of the occipital place area could be used to predict the affordances of novel scenes. Taken together, these results reveal a previously unknown mechanism for perceiving the affordance structure of navigable space.
• Hafri, A., Trueswell, J.C. & Epstein, R.A. (2017). Neural representations of observed actions generalize across static and dynamic visual input. The Journal of Neuroscience, 37(11): 3056-3071.
  • Neural representations of observed actions generalize across static and dynamic visual input.
    People interact with other people and with objects in distinct and categorizable ways (e.g., kicking is making contact with foot). We can recognize these action categories across variations in actors, objects, and settings; moreover, we can recognize them from both dynamic and static visual input. However, the neural systems that support action recognition across these perceptual differences are unclear. Here we used multivoxel pattern analysis of fMRI data to identify brain regions that support visual action categorization in a format-independent way. Human participants were scanned while viewing eight categories of interactions (e.g., pulling) depicted in two visual formats: (1) controlled videos of two interacting actors; and (2) still images selected from the internet involving different actors, objects, and scene contexts. Action category was decodable across visual formats in bilateral inferior parietal, bilateral occipitotemporal, left premotor, and left middle frontal cortex. In most of these regions, the similarity space of action categories was consistent across subjects and visual formats, a property that can contribute to a common understanding of events among individuals. These results suggest that the identified brain regions support action category codes that are crucial for action recognition and action understanding.
• Keinath, A.T., Julian, J.B., Epstein, R.A. & Muzzio, I.A. (2017). Environmental geometry aligns the hippocampal map during spatial orientation. Current Biology, 27: 1-9.
  • Environmental geometry aligns the hippocampal map during spatial orientation.
    When a navigator’s internal sense of direction is disrupted, she must rely on external cues to regain her bearings, a process termed spatial reorientation. Extensive research has demonstrated that the geometric shape of the environment exerts powerful control over reorientation behavior, but the neural and cognitive mechanisms underlying this phenomenon are not well understood. Whereas some theories claim that geometry controls behavior through an allocentric mechanism potentially tied to the hippocampus, others postulate that disoriented navigators reach their goals by using an egocentric view-matching strategy. To resolve this debate, we characterized hippocampal representations during reorientation. We first recorded from CA1 cells as disoriented mice foraged in chambers of various shapes. We found that the alignment of the recovered hippocampal map was determined by the geometry of the chamber, but not by nongeometric cues, even when these cues could be used to disambiguate geometric ambiguities. We then recorded hippocampal activity as disoriented mice performed a classical goaldirected spatial memory task in a rectangular chamber. Again, we found that the recovered hippocampal map aligned solely to the chamber geometry. Critically, we also found a strong correspondence between the hippocampal map alignment and the animal’s behavior, making it possible to predict the search location of the animal from neural responses on a trial-by-trial basis. Together, these results demonstrate that spatial reorientation involves the alignment of the hippocampal map to local geometry. We hypothesize that geometry may be an especially salient cue for reorientation because it is an inherently stable aspect of the environment.
• Marchette, S.A., Ryan, J. & Epstein, R.A. (2017). Schematic representations of local environmental space guide goal-directed navigation. Cognition, 158: 68-80.
  • Schematic representations of local environmental space guide goal-directed navigation.
    To successfully navigate to a target, it is useful to be able to define its location at multiple levels of specificity. For example, the location of a favorite coffee mug can be described in terms of which room it is in, or in terms of where it is within the room. An appealing hypothesis is that these levels of description are retrieved from memory by accessing the same representation at progressively finer levels of granularity—first remembering the general location of an object and then “zooming in.” Here we provide evidence for an alternative view, in which navigational behavior is guided by independent representations at multiple spatial scales. Subjects learned the locations of objects that were positioned within four visually distinct but geometrically similar buildings, which were in turn positioned within a broader virtual park. They were then tested on their knowledge of object location by asking them to navigate to the remembered location of each object. We examined errors during the test phase for confusions among geometrically analogous locations in different buildings—that is, navigating to the right location in the wrong building. We observed that subjects frequently made these confusions, which are analogous to remembering a passage’s location on the page of a book but not remembering the page that the passage is on. This suggests that subjects were recalling the object’s local location without recalling its global location. Further manipulations across seven experiments indicated that geometric confusions were observed even between buildings that were not metrically identical as long as geometrical equivalence could be defined. However, removing the walls so that the larger environment was no longer divided into subspaces abolished these erpatial memory contains two rors. Taken together, our results suggest that human sseparable representations of “where” an object can be found: (i) a schematic map of where an object lies with respect to local landmarks and boundaries; (ii) a representation of the identity and location of each local environment.
• Julian, J.B., Ryan, J. & Epstein, R.A. (2017). Coding of object size and object category in human visual cortex. Cerebral Cortex, 27(6): 3095-3109.
  • Coding of object size and object category in human visual cortex.
    A salient aspect of objects is their real-world size. Large objects tend to be fixed in the world and can act as navigational barriers and landmarks, whereas small objects tend to be moveable and manipulable. Previous work has identified regions of visual cortex that repond differentially to large versus small objects, but the role of size in organizing representations of object categories has not been fully explored. To address this issue, we scanned subjects while they viewed large and small objects drawn from 20 categories, with retinotopic extent equated across size classes. Univariate analyses replicated previous results showing a greater response to large than small objects in scene-responsive regions and the converse effect in the left occipitotemporal sulcus. Critically, multivariate analyses revealed organization-by-size both wihtin and across functional regions, as evidenced by activation patterns that were more similar for object categories of the same size than for object categories of different size. This effect was observed in both scene- and object-responsive regions and across high-level visual cortex as a whole, but not in early visual cortex. We hypothesize that real-world size is an important dimension for object category organization because of the many ecologically significant differenes between large and small objects.
• Vass, L.K. & Epstein, R.A. (2017). Common neural representations for visually guided reorientation and spatial imagery. Cerebral Cortex, 27(2): 1457-1471.
  • Common neural representations for visually guided reorientation and spatial imagery.
    Spatial knowledge about an environment can be cued from memory by perception of a visual scene during active navigation or by imagination of the relationships between nonvisible landmarks, such as when providing directions. It is not known whether these different ways of accessing spatial knowledge elicit the same representations in the brain. To address this issue, we scanned participants with fMRI, they performed a judgment of relative direction (JRD) task that required them to retrieve real-world spatial relationships in response to either pictorial or verbal cues. Multivoxel pattern analyses revealed several brain regions that exhibited representations that were independent of the cues to access spatial memory. Specifically, entorhinal cortex in the medial temporal lobe and the retrosplenial cortex (RSC) in the medial parietal lobe coded for the heading assumed on a particular trial, whereas the parahippocampal place area (PPA) contained information about the starting location of the JRD. These results demonstrate the existence of spatial representations in RSC, ERC, and PPA that are common to visually guided navigation and spatial imagery.
• Julian, J.B., Ryan, J., Hamilton, R.H. & Epstein, R.A. (2016). The Occipital Place Area is causally involved in representing environmental boundaries during navigation. Current Biology, 26: 1104-1109.
  • The Occipital Place Area is causally involved in representing environmental boundaries during navigation.
    Thirty years of research suggests that environmental boundaries—e.g., the walls of an experimental chamber or room—exert powerful influence on navigational behavior, often to the exclusion of other cues. Consistent with this behavioral work, neurons in brain structures that instantiate spatial memory often exhibit firing fields that are strongly controlled by environmental boundaries. Despite the clear importance of environmental boundaries for spatial coding, however, a brain region that mediates the perception of boundary information has not yet been identified. We hypothesized that the occipital place area (OPA), a scene-selective region located near the transverse occipital sulcus, might provide this perceptual source by extracting boundary information from visual scenes during navigation. To test this idea, we used transcranial magnetic stimulation (TMS) to interrupt processing in the OPA while subjects performed a virtual-reality memory task that required them to learn the spatial locations of test objects that were either fixed in place relative to the boundary of the environment or moved in tandem with a landmark object. Consistent with our prediction, we found that TMS to the right OPA impaired spatial memory for boundary-tethered, but not landmark-tethered, objects. Moreover, this effect was found when the boundary was defined by a wall, but not when it was defined by a marking on the ground. These results show that the OPA is causally involved in boundary-based spatial navigation and suggest that the OPA is the perceptual source of the boundary information that controls navigational behavior.
• Bryan, P.B., Julian, J.B. & Epstein, R.A. (2016). Rectilinear edge selectivity is insufficient to explain the category selectivity of the parahippocampal place area. Frontiers in Human Neuroscience, 10(137).
  • Rectilinear edge selectivity is insufficient to explain the category selectivity of the parahippocampal place area.
    The parahippocampal place area (PPA) is one of several brain regions that respond more strongly to scenes than to non-scene items such as objects and faces. The mechanism underlying this scene-preferential response remains unclear. One possibility is that the PPA is tuned to low-level stimulus features that are found more often in scenes than in less-preferred stimuli. Supporting this view, Nasr et al. (2014) recently observed that some of the stimuli that are known to strongly activate the PPA contain a large number of rectilinear edges. They further demonstrated that PPA response is modulated by rectilinearity for a range of non-scene images. Motivated by these results, we tested whether rectilinearity suffices to explain PPA selectivity for scenes. In the first experiment, we replicated the previous finding of modulation by rectilinearity in the PPA for arrays of 2-d shapes. However, two further experiments failed to find a rectilinearity effect for faces or scenes: high-rectilinearity faces and scenes did not activate the PPA any more strongly than low-rectilinearity faces and scenes. Moreover, the categorical advantage for scenes vs. faces was maintained in the PPA and two other scene-selective regions—the retrosplenial complex (RSC) and occipital place area (OPA)—when rectilinearity was matched between stimulus sets. We conclude that selectivity for scenes in the PPA cannot be explained by a preference for low-level rectilinear edges.
• Marchette, S.A., Vass, L.K., Ryan, J. & Epstein, R.A. (2015). Outside looking in: Landmark generalization in the human navigational system. The Journal of Neuroscience, 35(44): 14896-14908.
  • Outside looking in: Landmark generalization in the human navigational system.
    The use of landmarks is central to many navigational strategies. Here we use mutlivoxel pattern analysis of fMRI data to understand how landmarks are coded in the human brain. Subjects were scanned while viewing the interiors and exteriors of campus buildings. Despite their visual dissimilarity, interiors and exteriors corresponding to the same building elecited similar patterns in the parahippocampal place area (PPA), retrosplenial complex (RSC), and occipital place area (OPA), three regions known to respond strongly to scences and buildings. Generalizations across stimuli depended on knowing the correspondences among them in the PPA but not in the other two regions, suggesting that the PPA is the key region involved in learning the different peceptual instantiations of a landmark. In contrast, generalization depended on the ability to freely retrieve information from memory in RSC, and it did not depend on familiarity or cognitive task in OPA. Together, these results suggest a tripartite division of labor, whereby PPA codes landmark identity, RSC retrieves spatial or conceptual information associated with landmarks, and OPA processes visual features that are important for landmark recognition.
• Julian, J.B., Keinath, A.T., Muzzio, I.A. & Epstein, R.A. (2015). Place recognition and heading retrieval are mediated by dissociable cognitive systems in mice. Proceedings of the National Academy of Sciences, 112(20): 6503-6508.
  • Place recognition and heading retrieval are mediated by dissociable cognitive systems in mice.
    A lost navigator must identify its current location and recover its facing direction to restore its bearings. We tested the idea that these two tasks—place recognition and heading retrieval—might be mediated by distinct cognitive systems in mice. Previous work has shown that numerous species, including young children and rodents, use the geometric shape of local space to regain their sense of direction after disorientation, often ignoring nongeometric cues even when they are informative. Notably, these experiments have almost always been performed in single-chamber environments in which there is no ambiguity about place identity. We examined the navigational behavior of mice in a two-chamber paradigm in which animals had to both recognize the chamber in which they were located (place recognition) and recover their facing direction within that chamber (heading retrieval). In two experiments, we found that mice used nongeometric features for place recognition, but simultaneously failed to use these same features for heading retrieval, instead relying exclusively on spatial geometry. These results suggest the existence of separate systems for place recognition and heading retrieval in mice that are differentially sensitive to geometric and nongeometric cues. We speculate that a similar cognitive architecture may underlie human navigational behavior.
• Pegors, T.K., Mattar, M.G., Bryan, P.B., & Epstein, R.A. (2015). Simultaneous perceptual and response biases on sequential face attractiveness judgments. Journal of Experimental Psychology: General, 144(3): 664-73.
  • Simultaneous perceptual and response biases on sequential face attractiveness judgments.
    Face attractiveness is a social characteristic that we often use to make first-pass judgments about the people around us. However, these judgments are highly influenced by our surrounding social world, and researchers still understand little about the mechanisms underlying these influences. In a series of 3 experiments, we use a novel sequential rating paradigm that enables us to measure biases in attractiveness judgments from the previous face and the previous rating. Our results reveal 2 simultaneous and opposing influences on face attractiveness judgments that arise from past experience of faces: a response bias in which attractiveness ratings shift toward a previously given rating and a stimulus bias in which attractiveness ratings shift away from the mean attractiveness of the previous face. Further, we provide evidence that the contrastive stimulus bias (but not the assimilative response bias) is strengthened by increasing the duration of the previous stimulus, suggesting an underlying perceptual mechanism. These results demonstrate that judgments of face attractiveness are influenced by information from our evaluative and perceptual history and that these influences have measurable behavioral effects over the course of just a few seconds.
• Pegors, T.K., Kable, J., Chatterjee, A., & Epstein, R.A. (2015). Common and unique representations in frontal cortex for face and place attractiveness. Journal of Cognitive Neuroscience, 27(5): 959-973.
  • Common and unique representations in frontal cortex for face and place attractiveness.
    Although previous neuroimaging research has identified overlapping correlates of subjective value across different reward types in the ventromedial pFC (vmPFC), it is not clear whether this “common currency” evaluative signal extends to the aesthetic domain. To examine this issue, we scanned human participants with fMRI while they made attractiveness judgments of faces and places—two stimulus categories that are associated with different underlying rewards, have very different visual properties, and are rarely compared with each other. We found overlapping signals for face and place attractiveness in the vmPFC, consistent with the idea that this region codes a signal for value that applies across disparate reward types and across both economic and aesthetic judgments. However, we also identified a subregion of vmPFC within which activity patterns for face and place attractiveness were distinguishable, suggesting that some category-specific attractiveness information is retained in this region. Finally, we observed two separate functional regions in lateral OFC: one region that exhibited a category-unique response to face attractiveness and another region that responded strongly to faces but was insensitive to their value. Our results suggest that vmPFC supports a common mechanism for reward evaluation while also retaining a degree of category-specific information, whereas lateral OFC may be involved in basic reward processing that is specific to only some stimulus categories.
• Epstein, R.A. (2014). Neural systems for visual scene recognition. In M. Bar & K. Keveraga (Eds.), Scene Vision. Cambridge MA: MIT Press, pp. 105-134.
  • Neural systems for visual scene recognition.
    This chapter describes the neural systems involved in the perception and recognition of scenes; focusing particularly on the parahippocampal place area (PPA), a brain region that plays a central role in scene processing.
• Marchette, S.A., Vass, L.K., Ryan, J., & Epstein, R.A. (2014). Anchoring the neural compass: Coding of local spatial reference frames in human medial parietal lobe. Nature Neuroscience, 17(11): 1598-1606.
  • Anchoring the neural compass: Coding of local spatial reference frames in human medial parietal lobe
    The neural systems that code for location and facing direction during spatial navigation have been investigated extensively; however, the mechanisms by which these quantities are referenced to external features of the world are not well understood. To address this issue, we examined behavioral priming and functional magnetic resonance imaging activity patterns while human subjects recalled spatial views from a recently learned virtual environment. Behavioral results indicated that imagined location and facing direction were represented during this task, and multivoxel pattern analyses indicated that the retrosplenial complex (RSC) was the anatomical locus of these spatial codes. Critically, in both cases, location and direction were defined on the basis of fixed elements of the local environment and generalized across geometrically similar local environments. These results suggest that RSC anchors internal spatial representations to local topographical features, thus allowing us to stay oriented while we navigate and retrieve from memory the experience of being in a particular place.
• Epstein, R.A. & Vass, L.K. (2014). Neural systems for landmark-based wayfinding in humans. Philosophical Transactions of the Royal Society London B, 369(1635): 20120533.
  • Neural systems for landmark-based wayfinding in humans
    Humans and animals use landmarks during wayfinding to determine where they are in the world and to guide their way to their destination. To implement this strategy, known as landmark-based piloting, a navigator must be able to: (i) identify individual landmarks, (ii) use these landmarks to determine their current position and heading, (iii) access long-term knowledge about the spatial relationships between locations, and (iv) use this knowledge to plan a route to their navigational goal. Here we review neuroimaging, neuropsychological, and neurophysiological data that link the first three of these abilities to specific neural systems in the human brain. This evidence suggests that the parahippocampal place area (PPA) is critical for landmark recognition, the retrosplenial/medial parietal region is centrally involved in localization and orientation, and both medial temporal lobe (MTL) and retrosplenial/medial parietal lobe regions support long-term spatial knowledge.
• Weisberg, S.M., Schinazi, V.R., Newcombe, N.S., Shipley, T.F. & Epstein, R.A. (2013). Variations in cognitive maps: Understanding individual differences in navigation. Journal of Experimental Psychology: Learning, Memory and Cognition, 40(3): 669-682.
  • Variations in cognitive maps: Understanding individual differences in navigation
    There are marked individual differences in the formation of cognitive maps both in the real world and in virtual environments (VE; e.g., Blajenkova, Motes, & Kozhevnikov, 2005; Chai & Jacobs, 2010; Ishikawa & Montello, 2006; Wen, Ishikawa, & Sato, 2011). These differences, however, are poorly understood and can be difficult to assess except by self-report methods. VEs offer an opportunity to collect objective data in environments that can be controlled and standardized. In this study, we designed a VE consisting of buildings arrayed along 2 separated routes, allowing for differentiation of between-route and within-route representation. Performance on a pointing task and a model-building task correlated with self-reported navigation ability. However, for participants with lower levels of between-route pointing, the Santa Barbara Sense of Direction scale (Hegarty, Richardson, Montello, Lovelace, & Subbiah, 2002) did not predict individual differences in accuracy when pointing to buildings within the same route. Thus, we confirm the existence of individual differences in the ability to construct a cognitive map of an environment, identify both the strengths and the potential weaknesses of self-report measures, and isolate a dimension that may help to characterize individual differences more completely. The VE designed for this study provides an objective behavioral measure of navigation ability that can be widely used as a research tool.
• Epstein, R.A. & Julian, J.B. (2013). Scene areas in humans and macaques. Neuron, 79(4): 615-617.
  • Scene areas in humans and macaques
    Kornblith et al. (2013) identify two regions in macaque occipitotemporal cortex that encode both spatial and nonspatial aspects of visual scenes and might be the homolog of the human parahippocampal place area.
• Bastin, J., Vidal, J.R., Bouvier, S., Perrone-Bertolotti, M., Benis, D., Kahane, P., David, O., Lachaux, J.P. & Epstein, R.A. (2013). Temporal components in the parahippocampal place area revealed by human intracerebral recordings. Journal of Neuroscience, 33(24): 10123-10131.
  • Temporal components in the parahippocampal place area revealed by human intracerebral recordings
    Many high-level visual regions exhibit complex patterns of stimulus selectivity that make their responses difficult to explain in terms of a single cognitive mechanism. For example, the parahippocampal place area (PPA) responds maximally to environmental scenes during fMRI studies but also responds strongly to nonscene landmark objects, such as buildings, which have a quite different geometric structure. We hypothesized that PPA responses to scenes and buildings might be driven by different underlying mechanisms with different temporal profiles. To test this, we examined broadband λ (50 – 150 Hz) responses from human intracerebral electroencephalography recordings, a measure that is closely related to population spiking activity. We found that the PPA distinguished scene from nonscene stimuli in ~80 ms, suggesting the operation of a bottom-up process that encodes scene-specific visual or geometric features. In contrast, the differential PPA response to buildings versus nonbuildings occurred later (~170 ms) and may reflect a delayed processing of spatial or semantic features definable for both scenes and objects, perhaps incorporating signals from other cortical regions. Although the response preferences of high-level visual regions are usually interpreted in terms of the operation of a single cognitive mechanism, these results suggest that a more complex picture emerges when the dynamics of recognition are considered.
• Vass, L.K. & Epstein, R.A. (2013). Abstract representations of location and facing direction in the human brain. Journal of Neuroscience, 33(14): 6133-6142.
  • Abstract representations of location and facing direction in the human brain
    Humans, like other mobile organisms, rely on spatial representations to guide navigation from place to place. Although previous work has identified neural systems involved in wayfinding, the specific spatial codes supported by these systems are not well understood. We use functional magnetic resonance imaging to identify regions within the human medial temporal and medial parietal lobes that encode two fundamental spatial quantities−location and facing direction−in a manner that abstracts away from sensory inputs. University students were scanned while viewing photographs taken at several familiar campus locations. Multivoxel pattern analyses indicated that the left presubiculum, retrosplenial complex, and parietal-occipital sulcus coded location identity even across non-overlapping views, whereas the right presubiculum coded facing direction even across noncontiguous locations. The location and direction codes supported by these regions may be critical to our ability to navigate within the extended environment and to understand its large-scale spatial structure.
• Schinazi, V.R., Nardi, D., Newcombe, N.S., Shipley, T.F.& Epstein, R.A. (2013). Hippocampal size predicts rapid learning of a cognitive map in humans. Hippocampus, 23(6): 515-528.
  • Hippocampal size predicts rapid learning of a cognitive map in humans
    The idea that humans use flexible map-like representations of their environment to guide spatial navigation has a long and controversial history. One reason for this enduring controversy might be that individuals vary considerably in their ability to form and utilize cognitive maps. Here we investigate the behavioral and neuroanatomical signatures of these individual differences. Participants learned an unfamiliar campus environment over a period of three weeks. In their first visit, they learned the position of different buildings along two routes in separate areas of the campus. During the following weeks, they learned these routes for a second and third time, along with two paths that connected both areas of the campus. Behavioral assessments after each learning session indicated that subjects formed a coherent representation of the spatial structure of the entire campus after learning a single connecting path. Volumetric analyses of structural MRI data and voxel-based morphometry (VBM) indicated that the size of the right posterior hippocampus predicted the ability to use this spatial knowledge to make inferences about the relative positions of different buildings on the campus. An inverse relationship between gray matter volume and performance was observed in the caudate. These results suggest that (i) humans can rapidly acquire cognitive maps of large-scale environments and (ii) individual differences in hippocampal anatomy may provide the neuroanatomical substrate for individual differences in the ability to learn and flexibly use these cognitive maps.
• Troiani, V., Stigliani, A., Smith, M.E. & Epstein, R.A. (2012). Multiple object properties drive scene-selective regions. Cerebral Cortex, 24(4): 883-897.
  • Multiple object properties drive scene-selective regions
    Neuroimaging studies have identified brain regions that respond preferentially to specific stimulus categories, including 3 areas that activate maximally during viewing of real-world scenes: The parahippocampal place area (PPA), retrosplenial complex (RSC), and transverse occipital sulcus (TOS). Although these findings suggest the existence of regions specialized for scene processing, this interpretation is challenged by recent reports that activity in scenepreferring regions is modulated by properties of isolated single objects. To understand the mechanisms underlying these objectrelated responses, we collected functional magnetic resonance imaging data while subjects viewed objects rated along 7 dimensions, shown both in isolation and on a scenic background. Consistent with previous reports, we find that scene-preferring regions are sensitive to multiple object properties; however, results of an item analysis suggested just 2 independent factors-visual size and the landmark suitability of the objects-sufficed to explain most of the response. This object-based modulation was found in PPA and RSC irrespective of the presence or absence of a scenic background, but was only observed in TOS for isolated objects. We hypothesize that scene-preferring regions might process both visual qualities unique to scenes and spatial qualities that can appertain to either scenes or objects.
• Epstein, R.A. & Morgan, L.K. (2012). Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis. Neuropsychologia, 50(4): 530-543.
  • Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis
    Human observers can recognize real-world visual scenes with great efficiency. Cortical regions such as the parahippocampal place area (PPA) and retrosplenial complex (RSC) have been implicated in scene recognition, but the specific representations supported by these regions are largely unknown. We used functional magnetic resonance imaging adaptation (fMRIa) and multi-voxel pattern analysis (MVPA) to explore this issue, focusing on whether the PPA and RSC represent scenes in terms of general categories, or as specific scenic exemplars. Subjects were scanned while viewing images drawn from 10 outdoor scene categories in two scan runs and images of 10 familiar landmarks from their home college campus in two scan runs. Analyses of multi-voxel patterns revealed that the PPA and RSC encoded both category and landmark information, with a slight advantage for landmark coding in RSC. fMRIa, on the other hand, revealed a very different picture: both PPA and RSC adapted when landmark information was repeated, but category adaptation was only observed in a small subregion of the left PPA. These inconsistencies between the MVPA and fMRIa data suggests that these two techniques interrogate different aspects of the neuronal code. We propose three hypotheses about the mechanisms that might underlie adaptation and multi-voxel signals.
• MacEvoy, S.P. & Epstein, R.A. (2011). Constructing scenes from objects in human occipitotemporal cortex. Nature Neuroscience, 14: 1323-1329.
  • Constructing scenes from objects in human occipitotemporal cortex
    We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.
• Epstein, R.A. & MacEvoy, S.P. (2011). Making a scene in the brain. In L. Harris & M. Jenkin (Eds.), Vision in 3d Environments. Cambridge: Cambridge University Press.
• Epstein, R.A. (2011). Cognitive Neuroscience: Scene layout from vision and touch. Current Biology, 21(11): R437-R438.
  • Cognitive Neuroscience: Scene layout from vision and touch
    Parahippocampal and retrosplenial cortices respond strongly to visual scenes. A new study shows that these regions also activate when scenes are perceived haptically — even in the blind.
• Morgan, L.K., MacEvoy, S.P., Aguirre, G.K., & Epstein, R.A. (2011). Distances between real-world locations are represented in the human hippocampus. Journal of Neuroscience, 31(4): 1238-1245.
  • Distances between real-world locations are represented in the human hippocampus
    Spatial navigation is believed to be guided in part by reference to an internal map of the environment. We used functional magnetic resonance imaging (fMRI) to test for a key aspect of a cognitive map: preservation of real-world distance relationships. University students were scanned while viewing photographs of familiar campus landmarks. fMRI response levels in the left hippocampus corresponded to real-world distances between landmarks shown on successive trials, indicating that this region considered closer landmarks to be more representationally similar and more distant landmarks to be more representationally distinct. In contrast, posterior visually responsive regions such as retrosplenial complex and the parahippocampal place area were sensitive to landmark repetition and encoded landmark identity in their multivoxel activity patterns but did not show a distance-related response. These data suggest the existence of a map-like representation in the human medial temporal lobe thatencodes the coordinates offamiliar locations in large-scale, real-world environments.
• Ward, E.J., MacEvoy, S.P. & Epstein, R.A. (2010). Eye-centered encoding of visual space in scene-selective regions. Journal of Vision, 10(14):6: 1-12.
  • Eye-centered encoding of visual space in scene-selective regions
    We used functional magnetic resonance imaging (fMRI) to investigate the reference frames used to encode visual information in scene-responsive cortical regions. At early levels of the cortical visual hierarchy, neurons possess spatially selective receptive fields (RFs) that are yoked to specific locations on the retina. In lieu of this eye-centered organization, we speculated that visual areas implicated in scene processing, such as the parahippocampal place area (PPA), the retrosplenial complex (RSC), and transverse occipital sulcus (TOS) might instead possess RFs defined in head-, body-, or world-centered reference frames. To test this, we scanned subjects while they viewed objects and scenes presented at four screen locations while they maintained fixation at one of three possible gaze positions. We then examined response profiles as a function of either fixation-referenced or screen-referenced position. Contrary to our prediction, the PPA and TOS exhibited position–response curves that moved with the fixation point rather than being anchored to the screen, a pattern indicative of eye-centered encoding. RSC, on the other hand, did not exhibit a position–response curve in either reference frame. By showing an important commonality between the PPA/TOS and other visually responsive regions, the results emphasize the critical involvement of these regions in the visual analysis of scenes.
• Schinazi, V.R. & Epstein, R.A. (2010). Neural correlates of real-world route learning. NeuroImage, 53(2): 725-735.
  • Neural correlates of real-world route learning
    Classical theories of spatial microgenesis (Siegel and White, 1975) posit that information about landmarks and the paths between them is acquired prior to the establishment of more holistic survey-level representations. To test this idea, we examined the neural and behavioral correlates of landmark and path encoding during a real-world route learning episode. Subjects were taught a novel 3 km route around the University of Pennsylvania campus and then brought to the laboratory where they performed a recognition task that required them to discriminate between on-route and off-route buildings. Each building was preceded by a masked prime, which could either be the building that immediately preceded the target building along the route or immediately succeeded it. Consistent with previous reports using a similar paradigm in a virtual environment (Janzen and Weststeijn, 2007), buildings at navigational decision points (DPs) were more easily recognized than non-DP buildings and recognition was facilitated by in-route vs. against-route primes. Functional magnetic resonance imaging (fMRI) data collected during the recognition task revealed two effects of interest: first, greater response to DP vs. non-DP buildings in a wide network of brain regions previously implicated in spatial processing; second, a significant interaction between building location (DP vs. non-DP) and route direction (in-route vs. against-route) in a retrosplenial/parietal-occipital sulcus region previously labeled the retrosplenial complex (RSC). These results indicate that newly learned real-world routes are coded in terms of paths between decision points and suggest that the RSC may be a critical locus for integrating landmark and path information.
• Epstein, R.A. & Ward, E. J. (2010). How reliable are visual context effects in the parahippocampal place area? Cerebral Cortex, 20(2): 294-303.
  • How reliable are visual context effects in the parahippocampal place area?
    The parahippocampal place area (PPA) is a region of human cortex that responds more strongly to visual scenes (e.g., landscapes or cityscapes) than to other visual stimuli. It has been proposed that the primary function of the PPA is encoding of contextual information about object co-occurrence. Supporting this context hypothesis are reports that the PPA responds more strongly to strong-context than to weak-context objects and more strongly to famous faces (for which contextual associations are available) than to nonfamous faces. We reexamined the reliability of these 2 effects by scanning subjects with functional magnetic resonance imaging while they viewed strong- and weak-context objects, scrambled versions of these objects, and famous and nonfamous faces. ''Contextual'' effects for objects were observed to be reliable in the PPA at slow presentation rates but not at faster presentation rates intended to discourage scene imagery. We were unable to replicate the earlier finding of preferential PPA response to famous versus nonfamous faces. These results are difficult to reconcile with the hypothesis that the PPA encodes contextual associations but are consistent with a competing hypothesis that the PPA encodes scenic layout.
• MacEvoy, S.P. & Epstein, R.A. (2009). Decoding the representation of multiple simultaneous objects in human occipitotemporal cortex. Current Biology, 19: 943-947.
  • Decoding the representation of multiple simultaneous objects in human occipitotemporal cortex
    Previous work using functional magnetic resonance imaging has shown that the identities of isolated objects viewed by human subjects can be extracted from distributed patterns of brain activity. Outside the laboratory, however, objects almost never appear in isolation; thus it is important to understand how multiple simultaneously occurring objects are encoded by the visual system. We used multivoxel pattern analysis to examine this issue, testing whether activity patterns in the lateral occipital complex (LOC) evoked by object pairs showed an ordered relationship to patterns evoked by their constituent objects. Applying a searchlight analysis to identify voxels with the highest signal-to-noise ratios, we found that responses to object pairs within these informative voxels were well predicted by the averages of responses to their constituent objects. Consistent with this result, we were able to classify object pairs by using synthetic patterns created by averaging single-object patterns. These results indicate that the representation of multiple objects in LOC is governed by a response normalization mechanism similar to that reported in visual areas of several nonhuman species. They also suggest a population coding scheme that preserves information about multiple objects under conditions of distributed attention, facilitating fast object and scene recognition during natural vision.
• Epstein, R.A. (2008). Parahippocampal and retrosplenial contributions to human spatial navigation. Trends in Cognitive Sciences, 12: 388-396.
  • Parahippocampal and retrosplenial contributions to human spatial navigation
    Spatial navigation is a core cognitive ability in humans and animals. Neuroimaging studies have identified two functionally defined brain regions that activate during navigational tasks and also during passive viewing of navigationally relevant stimuli such as environmental scenes: the parahippocampal place area (PPA) and the retrosplenial complex (RSC). Recent findings indicate that the PPA and RSC have distinct and complementary roles in spatial navigation, with the PPA more concerned with representation of the local visual scene and RSC more concerned with situating the scene within the broader spatial environment. These findings are a first step towards understanding the separate components of the cortical network that mediates spatial navigation in humans.
• Epstein, R.A., Parker, W. E. & Feiler, A.M. (2008). Two kinds of fMRI repetition suppression? Evidence for dissociable neural mechanisms. Journal of Neurophysiology, 99: 2877-2886.
  • Two kinds of fMRI repetition suppression? Evidence for dissociable neural mechanisms
    Repetition suppression (RS) is a reduction of neural response that is often observed when stimuli are presented more than once. Many functional magnetic resonance imaging (fMRI) studies have exploited RS to probe the sensitivity of cortical regions to variations in different stimulus dimensions; however, the neural mechanisms underlying fMRI-RS are not fully understood. Here we test the hypothesis that long-interval (betweentrial) and short-interval (within-trial) RS effects are caused by distinct and independent neural mechanisms. Subjects were scanned while viewing visual scenes that were repeated over both long and short intervals. Within the parahippocampal place area (PPA) and other brain regions, suppression effects relating to both long- and shortinterval repetition were observed. Critically, two sources of evidence indicated that these effects were engendered by different underlying mechanisms. First, long- and short-interval RS effects were entirely noninteractive even although they were measured within the same set of trials during which subjects performed a constant behavioral task, thus fulfilling the formal requirements for a process dissociation. Second, long- and short-interval RS were differentially sensitive to viewpoint: short-interval RS was only significant when scenes were repeated from the same viewpoint while long-interval RS less viewpoint-dependent. Taken together, these results indicate that long- and short-interval fMRI-RS are mediated by different neural mechanisms that independently modulate the overall fMRI signal. These findings have important implications for understanding the results of studies that use fMRI-RS to explore representational spaces.
• MacEvoy, S.P., & Epstein, R.A. (2007). Position selectivity in scene- and object-responsive occipitotemporal regions. Journal of Neurophysiology, 98: 2089-2098.
  • Position selectivity in scene- and object-responsive occipitotemporal regions
    Complex visual scenes preferentially activate several areas of the human brain, including the parahippocampal place area (PPA), the retrosplenial complex (RSC), and the transverse occipital sulcus (TOS). The sensitivity of neurons in these regions to the retinal position of stimuli is unknown, but could provide insight into their roles in scene perception and navigation. To address this issue, we used functional magnetic resonance imaging (fMRI) to measure neural responses evoked by sequences of scenes and objects confined to either the left or right visual hemifields. We also measured the level of adaptation produced when stimuli were either presented first in one hemifield and then repeated in the opposite hemifield or repeated in the same hemifield. Although overall responses in the PPA, RSC, and TOS tended to be higher for contralateral stimuli than for ipsilateral stimuli, all three regions exhibited position-invariant adaptation, insofar as the magnitude of adaptation did not depend on whether stimuli were repeated in the same or opposite hemifields. In contrast, object-selective regions showed significantly greater adaptation when objects were repeated in the same hemifield. These results suggest that neuronal receptive fields (RFs) in scene-selective regions span the vertical meridian, whereas RFs in object-selective regions do not. The PPA, RSC, and TOS may support scene perception and navigation by maintaining stable representations of large-scale features of the visual environment that are insensitive to the shifts in retinal stimulation that occur frequently during natural vision.
• Epstein, R.A., Parker, W.E., & Feiler, A.M. (2007). Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition. Journal of Neuroscience, 27: 6141-6149.
  • Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition
    A key component of spatial navigation is the ability to use visual information to ascertain where one is located and how one is oriented in the world. We used functional magnetic resonance imaging to examine the neural correlates of this phenomenon in humans. Subjects were scanned while retrieving different kinds of topographical and nontopographical information in response to visual scenes. In the three critical conditions, they viewed images of a familiar college campus, and reported either the location of the place depicted in the image (location task), the compass direction that the camera was facing when the image was taken (orientation task), or whether the location was on campus or not (familiarity task). Our analyses focused on the retrosplenial cortex (RSC)/parietal-occipital sulcus region and the parahippocampal place area (PPA), which previous studies indicate play a critical role in place recognition. RSC activity depended on the type of information retrieved, with the strongest response in the location task. In contrast, PPA activity did not depend on the retrieval task. Additional analyses revealed a strong effect of familiarity in RSC but not in the PPA, with the former region responding much more strongly to images of the familiar campus than to images of an unfamiliar campus. These results suggest that the PPA and RSC play distinct but complementary roles in place recognition. In particular, the PPA may primarily support perception of the immediate scene, whereas RSC may support memory retrieval mechanisms that allow the scene to be localized within the broader spatial environment.
• Epstein, R.A., Higgins, J.S., Jablonski, K., & Feiler, A.M. (2007). Visual scene processing in familiar and unfamiliar environments. Journal of Neurophysiology, 97: 3670-3683.
  • Visual scene processing in familiar and unfamiliar environments
    Humans and animals use information obtained from the local visual scene to orient themselves in the wider world. Although neural systems involved in scene perception have been identified, the extent to which processing in these systems is affected by previous experience is unclear. We addressed this issue by scanning subjects with functional magnetic resonance imaging (fMRI) while they viewed photographs of familiar and unfamiliar locations. Scene-selective regions in parahippocampal cortex (the parahippocampal place area, or PPA), retrosplenial cortex (RSC), and the transverse occipital sulcus (TOS) responded more strongly to images of familiar locations than to images of unfamiliar locations with the strongest effects (
    50% increase) in RSC. Examination of fMRI repetition suppression (RS) effects indicated that images of familiar and unfamiliar locations were processed with the same degree of viewpoint specificity; however, increased viewpoint invariance was observed as individual scenes became more familiar over the course of a scan session. Surprisingly, these within-scan-session viewpointinvariant RS effects were only observed when scenes were repeated across different trials but not when scenes were repeated within a trial, suggesting that within- and between-trial RS effects may index different aspects of visual scene processing. The sensitivity to environmental familiarity observed in the PPA, RSC, and TOS supports earlier claims that these regions mediate the extraction of navigationally relevant spatial information from visual scenes. As locations become familiar, the neural representations of these locations become enriched, but the viewpoint invariance of these representations does not change.
• Dove, A., Manly, T., Epstein, R., & Owen, A.M. (2007). The engagement of mid-ventrolateral prefrontal cortex and posterior brain regions in intentional cognitive activity. Human Brain Mapping 29: 107–119.
  • The engagement of mid-ventrolateral prefrontal cortex and posterior brain regions in intentional cognitive activity
    It is now widely recognized that cognitive processes are carried out by a distributed network of brain areas, some of which are involved in perceptual processing of a stimulus, whilst others are involved in cognitive control processes required to carry out certain tasks. In this study, differential contributions of higher visual areas and of an area involved in cognitive control processes were investigated in a task requiring participants to simply look at a stimulus or to look with the intention of remembering. Varying the extent to which intentional cognitive processes were required and the stimulus material in this task allowed the analysis of ''top-down'' and ''bottom-up'' influences on these areas, respectively. Significant increases in the mid-ventrolateral prefrontal cortex (mid-VLPFC) were only observed when the stimuli were viewed with an intention in mind, irrespective of the stimulus type. In contrast, activity in the parahippocampal place area and the fusiform face area, was only modulated in conditions requiring intentional control when stimuli were presented that also elicited activity in these regions during passive viewing. These findings help to clarify the complimentary role that the midVLPFC and posterior higher visual areas play in controlled and relatively automatic memory processing.
• Epstein, R.A. & Higgins, J.S. (2007). Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. Cerebral Cortex, 17: 1680-1693.
  • Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition
    Human observers can quickly and accurately interpret the meaning of complex visual scenes. The neural mechanisms underlying this ability are largely unexplored. We used functional magnetic resonance imaging to measure cortical activity while subjects identified briefly presented scenes as specific familiar locations ("Houston Hall"), general place categories ("kitchen"), or general situational categories ("party"). Scene-responsive voxels in the parahippocampal place area (PPA) and retrosplenial cortex (RSC) were highly sensitive to recognition level when identifying scenes, responding more strongly during location identification than during place category or situation identification. In contrast, the superior temporal sulcus, cingulate sulcus, and supermarginal gyrus displayed the opposite pattern, responding more strongly during place category and situation identification. Consideration of results from 4 experiments suggests that the PPA represents the visuospatial structure of individual scenes, whereas RSC supports processes that allow scenes to be localized within a larger extended environment. These results suggest that different scene identification tasks tap distinct cortical networks. In particular, we hypothesize that the PPA and RSC are critically involved in the identification of specific locations but play a less central role in other scene recognition tasks.
• Hon, N., Epstein, R.A., Owen, A.M. & Duncan, J. (2006). Frontoparietal activity with minimal decision and control. Journal of Neuroscience, 26: 9805-9809.
  • Frontoparietal activity with minimal decision and control
    In the human brain, a well known frontoparietal circuit, including lateral prefrontal cortex (LPFC), presupplementary motor area/anterior cingulate cortex (pre-SMA/ACC), and both the superior and inferior parietal cortex, is involved in cognitive control. One proposal is thatthe frontoparietal cortex holds a flexible description ofattended or task-relevantinformation, biasing processing in favor of this information in many different parts of the brain. Here, we separate frontoparietal coding of attended information from its active use in behavior. In two experiments, subjects watch a stream of visual stimuli in a fixed location. In the first experiment, there is no task to perform; in the second, decisions are orthogonal to the occurrence of new stimulus events. Even in these simple circumstances, we find that attended stimulus changes give extensive activation ofLPFC, pre-SMA/ACC and parietal cortex, whereas unattended changes do not. Even without behavior to control, these classical "control" regions are active in simple update of attended information.
• Epstein, R.A., Higgins, J.S., Parker, W., Aguirre, G.K., & Cooperman, S. (2006). Cortical correlates of face and scene inversion: A comparison. Neuropsychologia, 44: 1145-1158.
  • Cortical correlates of face and scene inversion: A comparison
    Face recognition is more strongly impaired by stimulus inversion than nonface object recognition. This phenomenon, known as the face inversion effect (FIE), suggests that the visual system contains specialized processing mechanisms that are more engaged by upright faces than by inverted faces or nonface objects. Neuroimaging and neuropsychological studies indicate that environmental scenes may also recruit specialized-purpose processing machinery but a comparable inversion effect for scenes has not been established. Here we demonstrate that both face and scene inversion lead to behavioral penalties during performance of a continuous visual matching task; however, the scene inversion effect was less robust and declined in magnitude over the course of the experiment. Scene inversion led to greater neural response in the functionally defined lateral occipital (LO) object area for inverted versus upright scenes and reduced response in the parahippocampal place area (PPA), while face inversion lead to greater response in LO and the right middle fusiform (MF) object area for inverted versus upright faces but no change in the fusiform face area (FFA). A whole-brain analysis revealed several regions that responded more strongly to either upright versus inverted faces or upright versus inverted scenes, some of which may be involved in post-recognition processing. These results demonstrate that both face and scene inversion cause a shift from specialized processing streams towards generic object-processing mechanisms; however, this shift only leads to a reliable behavioral penalty in the case of face inversion.
• Epstein, R.A. (2005). The cortical basis of visual scene processing. Visual Cognition, 12: 954-978.
  • The cortical basis of visual scene processing
    Several lines of evidence suggest that the human brain contains special-purpose machinery for processing information about visual scenes. In particular, a region in medial occipitotemporal cortex - the "parahippocampal place area", or PPA - represents the geometric structure of scenes as defined primarily by their background elements. Neuroimaging studies have demonstrated that the PPA responds preferentially to scenes but not to the objects within them, while neuropsychological studies have shown that damage to this region leads to an impaired ability to learn new scenes. More recent evidence suggests that the PPA encodes novel scenes in a viewpoint-specific manner and that these codes are more reliable in good navigators than bad navigators. The PPA may be part of a larger network of regions involved in processing navigationally relevant spatial information. The role of this region in place recognition and gist comprehension is also discussed.
• Epstein, R.A., Higgins, J.S. & Thompson-Schill, S.L. (2005). Learning places from views: Variation in scene processing as a function of experience and navigational ability. Journal of Cognitive Neuroscience, 17: 73-83.
  • Learning places from views: Variation in scene processing as a function of experience and navigational ability
    Humans and animals use information obtained from different viewpoints to form representations of the spatial structure of the world. We used functional magnetic resonance imaging (fMRI ) adaptation to investigate the neural basis of this learning process and to show how the concomitant representations vary across individuals as a function of navigational ability. In particular, we examined the effect of repeating viewpoint and/or place information over both short (within-trial) and long (across-scan) intervals on the neural response in scene processing regions. Short-term fMRI adaptation effects in the parahippocampal cortex were initially highly viewpoint-specific butbecame less so over time. Long-term fMRI repetition effects included a significant viewpoint-invariant component. When individual differences in navigational ability were considered, a significant correlation between the strength of these effects and self-reported navigational competence was observed. In particular, good navigators encoded representations that differed between new and old views and new and old places, whereas bad navigators did not. These results suggest that cortical scene representations evolve over time to become more viewpointinvariant and that the quality of these representations directly impacts navigational ability.
• Gauthier, I., Curby, K.M., Skludlarski, P. & Epstein, R.A. (2005). Individual differences in FFA activity suggest independent processing at different spatial scales. Cognitive, Affective & Behavioral Neuroscience, 5: 222-234.
  • Individual differences in FFA activity suggest independent processing at different spatial scales
    The brain processes images at different spatial scales, but it is unclear how far into the visual stream different scales remain segregated. Using functional magnetic resonance imaging, we found evidence that BOLD activity in the fusiform face area (FFA) reflects computations based on separate spatial frequency inputs. When subjects perform different tasks (attend location vs. identity; attend whole vs. parts) or the same task with different stimuli (upright or inverted) with high- and low-pass images of cars and faces, individual differences in the FFA in one condition are correlated with those in the other condition. However, FFA activity in response to low-pass stimuli is independent of its response to highpass stimuli. These results suggest that spatial scales are not integrated before the FFA and that processing in this area could support the flexible use of different sources of information present in broadpass images.
• Lee, A.C.H., Bussey, T.J., Murray, E.A., Saksida, L.M., Epstein, R.A., Kapur, N., Hodges, J.R. & Graham, K.S. (2005). Perceptual deficits in amnesia: challenging the medial temporal lobe 'mnemonic' view. Neuropsychologia, 43: 1-11.
  • Perceptual deficits in amnesia: challenging the medial temporal lobe 'mnemonic' view
    Recent animal studies suggest that the medial temporal lobe (MTL), which is thought to subserve memory exclusively, may support nonmnemonic perceptual processes, with the hippocampus and perirhinal cortex contributing to spatial and object perception, respectively. There is, however, no support for this view in humans, with human MTL lesions causing prominent memory deficits in the context of apparently normal perception. We assessed visual discrimination in amnesic cases to reveal that while selective hippocampal damaged patients could discriminate faces, objects, abstract art and colour, they were significantly poorer in discriminating spatial scenes. By contrast, patients with MTL damage, including perirhinal cortex, were significantly impaired in discriminating scenes, faces, and to a lesser extent objects, with relatively intact discrimination of art and colour. These novel observations imply that the human MTL subserves both perceptual and mnemonic functions, with the hippocampus and perirhinal cortex playing distinct roles in spatial and object discrimination, respectively.
• Epstein, R., Graham, K.S., & Downing, P.E. (2003). Viewpoint-specific scene representations in human parahippocampal cortex. Neuron, 37: 865-876.
  • Viewpoint-specific scene representations in human parahippocampal cortex
    The "parahippocampal place area" (PPA) responds more strongly in functional magnetic resonance imaging (fMRI) to scenes than to faces, objects, or other visual stimuli. We used an event-related fMRI adaptation paradigm to test whether the PPA represents scenes in a viewpoint-specific or viewpoint-invariant manner. The PPA responded just as strongly to viewpoint changes that preserved intrinsic scene geometry as it did to complete scene changes, but less strongly to object changes within the scene. In contrast, lateral occipital cortex responded more strongly to object changes than to spatial changes. These results demonstrate that scene processing in the PPA is viewpoint specific and suggest that the PPA represents the relationship between the observer and the surfaces that define local space.
• Epstein, R., DeYoe, E.A., Press, D.Z., Rosen, A.C. & Kanwisher, N. (2001). Neuropsychological evidence for a topographical learning mechanism in parahippocampal cortex. Cognitive Neuropsychology, 18: 481-508.
  • Neuropsychological evidence for a topographical learning mechanism in parahippocampal cortex
    The Parahippocampal Place Area (PPA; Epstein & Kanwisher, 1998) is a region within posterior parahippocampal cortex that responds selectively to visual stimuli that convey information about the layout of local space. Here we describe two patients who suffered damage to the PPA after vascular incidents. Both subsequently exhibited memory problems for topographical materials and were unable to navigate unassisted in unfamiliar environments. Performance on a continuous n-back visual memory test was significantly lower for novel scene-like stimuli than for novel object-like stimuli. In contrast, performance was normal on a famous landmark recognition task and on two perceptual tasks that required on-line analysis of scene geometry. Both patients were able to produce accurate maps of premorbidly learned places but were unable to produce accurate maps of new places. These results converge with previous neuroimaging work to demonstrate that the PPA (1) is selectively involved in processing information about the geometry of surrounding space, and (2) may play a more critical role in the encoding of this information into memory than in the initial perceptual processing, recognition, or recall of this information.
• Epstein, R. & Kanwisher, N. (1999). Repetition blindness for locations: Evidence for automatic spatial coding in an RSVP task. Journal of Experimental Psychology: Human Perception and Performance, 25: 1855-1866.
  • Repetition blindness for locations: Evidence for automatic spatial coding in an RSVP task
    The authors report a new phenomenon called repetition blindness (RB) for locations: When 3 or 4 letters are presented rapidly and sequentially at random locations within a spatial array, experimental participants have difficulty reporting pairs of letters appearing in the same location within 250 ms of each other. This deficit occurs both during report of letter identities and during report of the locations in which the letters appear; it can also be found using a partial report task. During letter report, the deficit is found for 4-location arrays but not for 8-location arrays. In contrast, letter RB is not found during location report even when the letters are always chosen from a set of 4. These results indicate that a small number of locations--but not letters--can be encoded automatically even when they are not explicitly reported. The authors argue that RB for locations results from a difficulty individuating 2 tokens at the same spatial location.
• Epstein, R., Harris, A., Stanley, D. & Kanwisher, N. (1999). The parahippocampal place area: Recognition, navigation, or encoding? Neuron, 23:115-125.
  • The parahippocampal place area: Recognition, navigation, or encoding?
    The parahippocampal place area (PPA) has been demonstrated to respond more strongly in fMRI to scenes depicting places than to other kinds of visual stimuli. Here, we test several hypotheses about the function of the PPA. We find that PPA activity (1) is not affected by the subjects' familiarity with the place depicted, (2) does not increase when subjects experience a sense of motion through the scene, and (3) is greater when viewing novel versus repeated scenes but not novel versus repeated faces. Thus, we find no evidence that the PPA is involved in matching perceptual information to stored representations in memory, in planning routes, or in monitoring locomotion through the local or distal environment but some evidence that it is involved in encoding new perceptual information about the appearance and layout of scenes.
• Epstein, R. & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature, 392: 598-601.
  • A cortical representation of the local visual environment
    Medial temporal brain regions such as the hippocampal formation and parahippocampal cortex have been generally implicated in navigation and visual memory. However, the specific function of each of these regions is not yet clear. Here we present evidence that a particular area within human parahippocampal cortex is involved in a critical component of navigation: perceiving the local visual environment. This region, which we name the "parahippocampal place area" (PPA), responds selectively and automatically in functional magnetic resonance imaging (fMRI) to passively viewed scenes, but only weakly to single objects and not at all to faces. The critical factor for this activation appears to be the presence in the stimulus of information about the layout of local space. The response in the PPA to scenes with spatial layout but no discrete objects (empty rooms) is as strong as the response to complex meaningful scenes containing multiple objects (the same rooms furnished) and over twice as strong as the response to arrays of multiple objects without three-dimensional spatial context (the furniture from these rooms on a blank background). This response is reduced if the surfaces in the scene are rearranged so that they no longer define a coherent space. We propose that the PPA represents places by encoding the geometry of the local environment.

Conciousness & Phenomenology

• Epstein, R. (2004). Consciousness, art and the brain: Lessons from Marcel Proust. Consciousness & Cognition, 13: 213-240.
  • Consciousness, art and the brain: Lessons from Marcel Proust
    In his novel Remembrance of Things Past, Marcel Proust argues that conventional descriptions of the phenomenology of consciousness are incomplete because they focus too much on the highly-salient sensory information that dominates each moment of awareness and ignore the network of associations that lies in the background. In this paper, I explicate Proust's theory of conscious experience and show how it leads him directly to a theory of aesthetic perception. Proust's division of awareness into two components roughly corresponds to William James' division of the stream of thought into a ''nucleus'' and ''fringe.'' Proust argues that the function of art is to evoke the underlying associative network indirectly in the mind of the observer by using carefully chosen sensory surfaces to control the stream of thought. I propose a possible neural basis for this Proustian/Jamesian phenomenology, and argue that the general principles of Proustian aesthetics can be applied to all forms of art. I conclude that a scientific theory of art should follow in a straightforward manner from a scientific theory of consciousness.
• Commentary: Galin, D. (2004). Aesthetic experience: Marcel Proust and the neo-Jamesian structure of awareness. Consciousness & Cognition, 13: 241-253.
• Commentary: Dryden, D. (2004). Memory, imagination, and the cognitive value of the arts. Consciousness & Cognition, 13: 254-267.
• Epstein, R. (2000). The neural-cognitive basis of the Jamesian stream of thought. Consciousness & Cognition, 9: 550-575.
• Commentary: Galin, D. (2000). Comments on Epstein's neurocognitive interpretation of William James' model of consciousness. Consciousness & Cognition, 9: 576-583.
• Epstein, R. (2000). Substantive thoughts about substantive thoughts: A reply to Galin. Consciousness & Cognition, 9: 584-590.

Computer Vision

• Yuille, A.L., Snow, D., Epstein, R. & Belhumeur, P.N. (1999). Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability. International Journal of Computer Vision, 35: 203-222.
• Epstein, R. , Yuille, A.L. & Belhumeur, P.N. (1996). Learning object representations from lighting variations. In Object Representation in Computer Vision II, J. Ponce, A. Zisserman & M. Hebert, eds. Springer Lecture Notes in Computer Science, 108-116.
• Epstein, R., Hallinan, P.W. & Yuille, A.L. (1995). 5±2 eigenimages suffice: An empirical investigation of low-dimensional lighting models. In Proceedings of the IEEE Workshop on Physics-Based Modeling in Computer Vision.
• Epstein, R. & Yuille, A.L. (1994). Training a general purpose deformable template. In Proceedings of the First IEEE Conference on Image Processing, Vol. 1. Austin, TX: IEEE Society Press, 203-207.