Aligning EEG with text for decoding imagined speech

A new framework maps EEG trials to language representations to decode 39 intent categories.

Researchers developed a contrastive learning framework to decode imagined speech. This method aligns EEG trials with language representations in the embedding space of a frozen pretrained text encoder.

The approach uses an EEG encoder trained with a composite objective. This objective combines semantically reweighted EEG-sentence alignment and text-prototype-based class discrimination.

The framework was evaluated on the Chisco imagined speech dataset across five subjects. It measured Top-k accuracy, macro-F1, and balanced pairwise matching accuracy (2v2).

Maximum EEG-text cosine similarity, or MaxCosine, was used as an indicator of decoding reliability. This metric identified subsets of held-out trials with higher Top-1 and Top-5 accuracy.

The findings suggest that EEG-language alignment can retain fixed-intent decoding. It also supports similarity-based retrieval and the evaluation of broader semantic structure.

Key facts

  • The framework aligns EEG trials with language representations.
  • The method was tested on the Chisco imagined speech dataset using five subjects.
  • The approach preserves separability among 39 intent categories.
  • MaxCosine can identify trials with higher Top-1 and Top-5 accuracy.

All sections

Edition