A new brain imaging dataset for decoding mental images from sound

The AVID dataset uses 5.0 Tesla functional magnetic resonance imaging to study visual mental imagery.

AVID is a 5.0 Tesla functional magnetic resonance imaging (fMRI) resource. It is designed for auditory-induced visual mental imagery (VMI) decoding.

The dataset contains approximately 67 hours of dense recordings from nine participants. It includes a scene-level complex split using naturalistic Mandarin descriptions. It also includes a controlled simple split with explicitly specified visual attributes.

Visual mental imagery (VMI) decoding uses brain activity to reconstruct internally generated visual scenes. Auditory-induced VMI decoding involves imagery triggered by sound.

Current VMI decoding progress is limited by a lack of training-scale VMI datasets. Existing datasets often have visual and mnemonic confounds. These make it difficult to separate neural activity from signals driven by prior or concurrent visual input.

A standardized evaluation protocol was formalized to decouple source-image consistency from semantic alignment. Baseline results show current reconstruction pipelines remain limited on AVID. The most successful reconstructions preserve recognizable scene level semantics from spoken descriptions.

Key facts

  • AVID includes approximately 67 hours of recordings from nine participants.
  • The dataset uses 5.0 Tesla functional magnetic resonance imaging (fMRI).
  • AVID provides a resource for auditory-induced visual mental imagery (VMI) decoding.

All sections

Edition