Understanding how the human brain works is one of the greatest challenges that science faces today. The Algonauts challenge proposes an open and quantified test of computational models on human brain data, including both spatial (i.e. fMRI) and temporal (i.e. MEG/EEG) measurements. This will allow us to assess the real progress of the field of cognitive computational neuroscience (see also brain score and here).
The primary target of the 2019 challenge is the visual brain: the part of the brain that is responsible for seeing. Currently, particular deep neural networks trained on object recognition (Yamins et al. 2014, Khaligh-Razavi & Kriegeskorte 2014, Cichy et al. 2016 ...) do best in explaining brain activity. Can your model do better?
The main goal of the 2019 Algonauts challenge is to predict brain activity from two sources—fMRI data from two brain regions (Track 1), or MEG data from two temporal windows (Track 2)—using computational models. The brain activity is in response to viewing sets of images; for each image set, fMRI and MEG data are collected from the same 15 human subjects. Participants can choose to play in Track 1 (fMRI), Track 2 (MEG), or both.
Underlying object recognition is a hierarchical
processing cascade (called the “ventral visual stream”) in which neural activity
unfolds in space and time. At the beginning of the hierarchy, there is a region
called the early visual cortex (EVC), shown in red in the figure.
Neurons in this area respond to lines or edges with specific orientations. As EVC
is early in the processing cascade, it responds early in time.
Later in this hierarchy there is a region called the inferior temporal cortex (IT), shown in yellow. Neurons in this region have been found to respond to images of objects.
In the 2019 edition of the Algonauts challenge, the target is to explain brain activity in two segments of the visual processing cascade. The brain regions to explain are EVC and IT for Track 1, and two intervals in time (early interval around the peak of response in EVC and later interval around the peak of response in IT, with respect to when an image was shown to human subjects) for Track 2.
Given a set of images consisting of everyday objects and corresponding brain activity recorded while human subjects viewed those images, participants will devise computational models which predict brain activity, which will be used to predict the brain activity for a brand new set of images.
The goal of Track 1 (fMRI) is to construct models that best predict activity in 2 regions of the brain, early (EVC) and late (IT) in the human visual hierarchy. Participants submit their model responses to a test image set which we will compare against held-out fMRI data. The representational similarity analysis approach (a technique that maps models and fMRI data to a common similarity space to enable comparison) will be used to score your submissions.
For the Track 1 competition we provide the following:
The goal of Track 2 (MEG) is to construct models that best predict brain data from 2 time intervals, early and late stages of visual processing with respect to onset of an image. Participants submit their model responses to a test image set which we will compare against held-out MEG data. The representational similarity analysis approach (a technique that maps models and MEG data to a common similarity space to enable comparison) will be used to score your submissions. Time intervals of 20ms are defined based on the peak latency of MEG/fMRI fusion time series (0–200ms) in EVC and IT, and are shown with the corresponding image sets below (see Cichy et al. 2014, 2016; Mohsenzadeh et al. 2019 for MEG/fMRI fusion method).
For the Track 2 competition we provide the following:
Current non-invasive techniques to measure brain
activity resolve the brain either well in space or in time, but not both. We provide
measurements of brain activity while humans viewed a set of images from two techniques:
(1) functional magnetic resonance imaging (fMRI) for millimeter spatial resolution and
(2) magnetoencephalography (MEG) for millisecond temporal resolution. There is a
challenge track for explaining brain data in space (fMRI) and in time (MEG)
Click here to learn more about fMRI and MEG
To compare computational models to human brains we use representational similarity
analysis (RSA). The idea is that if models and brains are similar, then they treat the
same images as similar or dissimilar. Using RSA allows us to compare human brains and
models at the relevant level of representations in spite of the numerous differences
between them (e.g. in-silico vs. biological).
Click here to learn about RSA and the comparison metric
You can submit your model out of the box without any further training. However, we
provide training brain data that can help optimize your model for predicting brain data.
Download the data and development kit here
The training data consist of brain activity from both fMRI and MEG in response to viewing images from two sets—the 92 and 118 sets. For each image set, the fMRI and MEG data are collected from the same 15 human subjects. For fMRI, the data provided is based on correlation distances for two brain regions, EVC and IT and for MEG it is based on correlation distances in two time intervals (one corresponding to early brain responses associated with EVC and the other for later brain responses associated with IT).
Consists of 78 images only.
Sample code for generating activations from AlexNet/VGG/ResNet, computing RDMs from these networks, and evaluating model RDMs against human brain data RDMs.
1) Participants can use any external data for model building. Participants that use the test image set for training will be disqualified (including brain data generated using the test image set).
2) Each participant (single researchers or team) can make 10 submissions per day per track for a maximum of 250 submissions.
3) Participants should be ready to upload a short report (up to 4 pages) describing their model building process for their best model to a preprint server (e.g biorxiv or arxiv). Details will be specified closer to submission deadline (June 22).
4) Teams with best submission results will receive an invitation to participate in the Algonauts Workshop, held at MIT on July 19-20, to give a talk about their method during the workshop.
Note: Teams with best submission results will receive an invitation by July 1 to participate in the Algonauts Workshop, held at MIT on 19-20 July, to give a talk about their method during the workshop.