We provide measurements of brain responses using the technique of functional magnetic resonance imaging (fMRI) for millimeter spatial resolution. This technique measures brain activity indirectly by detecting changes in blood flow associated with brain activity. fMRI is a core technique in cognitive neuroscience to observe the human brain in action noninvasively.
The 2023 Challenge data comes from the Natural Scenes Dataset (NSD) (Allen et al., 2022), a massive 8-subjects dataset of 7T fMRI responses to images of natural scenes coming from the COCO database (Lin et al., 2014).
During the NSD experiment each subject viewed 10,000 distinct images, and a special set of 1000 images was shared across subjects (8 subjects x 9000 unique images + 1000 shared images = 73,000 images). Each of the 10,000 images was presented three times, for a total of 30,000 image trials per subject. Subjects were instructed to focus on a fixation cross at the center of the screen and performed a continuous recognition task in which they reported whether the current image had been presented at any previous point in the experiment. All images were presented with a visual angle of 8.4° x 8.4°.
For every subject the NSD experiment was split across 40 scan sessions: not all subjects completed all of them, resulting in different amounts of recorded data between subjects. The fMRI data of the last three sessions of every subject is withheld and constitutes the basis for the test split of the Algonauts Project 2023 Challenge, whereas the data of the remaining sessions is released and constitutes the basis for the train split of the Challenge.
The Challenge uses preprocessed fMRI responses (BOLD response amplitudes) from each subject that have been projected onto a common cortical surface group template (FreeSurfer's fsaverage surface) (more information on preprocessing in the NSD paper). The whole brain surfaces are mapped out in vertices, and the Challenge data consists of a subset of cortical surface vertices in the visual cortex (a region of the brain specialized in processing visual input) that were maximally responsive to visual stimulation. The fMRI data is divided into the left (LH) and right (RH) hemispheres, each consisting of 19,004 and 20,544 vertices, with the exception of subjects 6 (18,978 LH and 20,220 RH vertices) and 8 (18,981 LH and 20,530 RH vertices) due to missing data. We z-scored the activity of each voxel independently for each NSD scan session, and then averaged the fMRI responses across repeats of the same stimuli images.
Challenge vertices. Brain surface plots of the vertices used for the Challenge (in purple).
