Decoding sensor space data with generalization across time and conditions

This example runs the analysis described in [1]. It illustrates how one can fit a linear classifier to identify a discriminatory topography at a given time instant and subsequently assess whether this linear model can accurately predict all of the time samples of a second set of conditions.

# Authors: Jean-Rémi King <jeanremi.king@gmail.com>
#          Alexandre Gramfort <alexandre.gramfort@inria.fr>
#          Denis Engemann <denis.engemann@gmail.com>
#
# License: BSD-3-Clause
# Copyright the MNE-Python contributors.

import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

import mne
from mne.datasets import sample
from mne.decoding import GeneralizingEstimator

print(__doc__)

# Preprocess data
data_path = sample.data_path()
# Load and filter data, set up epochs
meg_path = data_path / "MEG" / "sample"
raw_fname = meg_path / "sample_audvis_filt-0-40_raw.fif"
events_fname = meg_path / "sample_audvis_filt-0-40_raw-eve.fif"
raw = mne.io.read_raw_fif(raw_fname, preload=True)
picks = mne.pick_types(raw.info, meg=True, exclude="bads")  # Pick MEG channels
raw.filter(1.0, 30.0, fir_design="firwin")  # Band pass filtering signals
events = mne.read_events(events_fname)
event_id = {
    "Auditory/Left": 1,
    "Auditory/Right": 2,
    "Visual/Left": 3,
    "Visual/Right": 4,
}
tmin = -0.050
tmax = 0.400
# decimate to make the example faster to run, but then use verbose='error' in
# the Epochs constructor to suppress warning about decimation causing aliasing
decim = 2
epochs = mne.Epochs(
    raw,
    events,
    event_id=event_id,
    tmin=tmin,
    tmax=tmax,
    proj=True,
    picks=picks,
    baseline=None,
    preload=True,
    reject=dict(mag=5e-12),
    decim=decim,
    verbose="error",
)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_filt-0-40_raw.fif...
    Read a total of 4 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
        Average EEG reference (1 x 60)  idle
    Range : 6450 ... 48149 =     42.956 ...   320.665 secs
Ready.
Reading 0 ... 41699  =      0.000 ...   277.709 secs...
Filtering raw data in 1 contiguous segment
Setting up band-pass filter from 1 - 30 Hz

FIR filter parameters
---------------------
Designing a one-pass, zero-phase, non-causal bandpass filter:
- Windowed time-domain design (firwin) method
- Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation
- Lower passband edge: 1.00
- Lower transition bandwidth: 1.00 Hz (-6 dB cutoff frequency: 0.50 Hz)
- Upper passband edge: 30.00 Hz
- Upper transition bandwidth: 7.50 Hz (-6 dB cutoff frequency: 33.75 Hz)
- Filter length: 497 samples (3.310 s)

We will train the classifier on all left visual vs auditory trials and test on all right visual vs auditory trials.

clf = make_pipeline(
    StandardScaler(),
    LogisticRegression(solver="liblinear"),  # liblinear is faster than lbfgs
)
time_gen = GeneralizingEstimator(clf, scoring="roc_auc", n_jobs=None, verbose=True)

# Fit classifiers on the epochs where the stimulus was presented to the left.
# Note that the experimental condition y indicates auditory or visual
time_gen.fit(X=epochs["Left"].get_data(copy=False), y=epochs["Left"].events[:, 2] > 2)

  0%|          | Fitting GeneralizingEstimator : 0/35 [00:00<?,       ?it/s]
  6%|▌         | Fitting GeneralizingEstimator : 2/35 [00:00<00:00,   57.60it/s]
 11%|█▏        | Fitting GeneralizingEstimator : 4/35 [00:00<00:00,   58.54it/s]
 20%|██        | Fitting GeneralizingEstimator : 7/35 [00:00<00:00,   69.19it/s]
 26%|██▌       | Fitting GeneralizingEstimator : 9/35 [00:00<00:00,   66.58it/s]
 37%|███▋      | Fitting GeneralizingEstimator : 13/35 [00:00<00:00,   77.81it/s]
 46%|████▌     | Fitting GeneralizingEstimator : 16/35 [00:00<00:00,   79.95it/s]
 54%|█████▍    | Fitting GeneralizingEstimator : 19/35 [00:00<00:00,   80.67it/s]
 63%|██████▎   | Fitting GeneralizingEstimator : 22/35 [00:00<00:00,   81.92it/s]
 71%|███████▏  | Fitting GeneralizingEstimator : 25/35 [00:00<00:00,   82.77it/s]
 77%|███████▋  | Fitting GeneralizingEstimator : 27/35 [00:00<00:00,   78.73it/s]
 83%|████████▎ | Fitting GeneralizingEstimator : 29/35 [00:00<00:00,   76.55it/s]
 91%|█████████▏| Fitting GeneralizingEstimator : 32/35 [00:00<00:00,   77.89it/s]
100%|██████████| Fitting GeneralizingEstimator : 35/35 [00:00<00:00,   79.45it/s]
100%|██████████| Fitting GeneralizingEstimator : 35/35 [00:00<00:00,   78.85it/s]

<GeneralizingEstimator(base_estimator=Pipeline(steps=[('standardscaler', StandardScaler()), ('logisticregression', LogisticRegression(solver='liblinear'))]), scoring='roc_auc', verbose=True, fitted with 35 estimators, fitted with 35 estimators>

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Score on the epochs where the stimulus was presented to the right.

scores = time_gen.score(
    X=epochs["Right"].get_data(copy=False), y=epochs["Right"].events[:, 2] > 2
)

  0%|          | Scoring GeneralizingEstimator : 0/1225 [00:00<?,       ?it/s]
  1%|          | Scoring GeneralizingEstimator : 12/1225 [00:00<00:03,  331.73it/s]
  2%|▏         | Scoring GeneralizingEstimator : 24/1225 [00:00<00:03,  343.99it/s]
  3%|▎         | Scoring GeneralizingEstimator : 37/1225 [00:00<00:03,  358.50it/s]
  4%|▍         | Scoring GeneralizingEstimator : 49/1225 [00:00<00:03,  357.94it/s]
  5%|▍         | Scoring GeneralizingEstimator : 61/1225 [00:00<00:03,  357.52it/s]
  6%|▌         | Scoring GeneralizingEstimator : 74/1225 [00:00<00:03,  362.41it/s]
  7%|▋         | Scoring GeneralizingEstimator : 86/1225 [00:00<00:03,  361.50it/s]
  8%|▊         | Scoring GeneralizingEstimator : 98/1225 [00:00<00:03,  360.59it/s]
  9%|▉         | Scoring GeneralizingEstimator : 110/1225 [00:00<00:03,  360.03it/s]
 10%|▉         | Scoring GeneralizingEstimator : 122/1225 [00:00<00:03,  355.98it/s]
 10%|█         | Scoring GeneralizingEstimator : 127/1225 [00:00<00:03,  327.47it/s]
 12%|█▏        | Scoring GeneralizingEstimator : 141/1225 [00:00<00:03,  335.75it/s]
 13%|█▎        | Scoring GeneralizingEstimator : 154/1225 [00:00<00:03,  340.80it/s]
 13%|█▎        | Scoring GeneralizingEstimator : 163/1225 [00:00<00:03,  332.99it/s]
 14%|█▍        | Scoring GeneralizingEstimator : 170/1225 [00:00<00:03,  321.54it/s]
 15%|█▍        | Scoring GeneralizingEstimator : 180/1225 [00:00<00:03,  319.33it/s]
 16%|█▌        | Scoring GeneralizingEstimator : 192/1225 [00:00<00:03,  322.40it/s]
 17%|█▋        | Scoring GeneralizingEstimator : 205/1225 [00:00<00:03,  327.62it/s]
 18%|█▊        | Scoring GeneralizingEstimator : 217/1225 [00:00<00:03,  329.81it/s]
 19%|█▉        | Scoring GeneralizingEstimator : 230/1225 [00:00<00:02,  332.84it/s]
 20%|█▉        | Scoring GeneralizingEstimator : 241/1225 [00:00<00:02,  332.33it/s]
 21%|██        | Scoring GeneralizingEstimator : 253/1225 [00:00<00:02,  334.06it/s]
 22%|██▏       | Scoring GeneralizingEstimator : 265/1225 [00:00<00:02,  335.58it/s]
 23%|██▎       | Scoring GeneralizingEstimator : 277/1225 [00:00<00:02,  337.05it/s]
 24%|██▎       | Scoring GeneralizingEstimator : 289/1225 [00:00<00:02,  338.21it/s]
 25%|██▍       | Scoring GeneralizingEstimator : 302/1225 [00:00<00:02,  341.31it/s]
 26%|██▌       | Scoring GeneralizingEstimator : 316/1225 [00:00<00:02,  344.39it/s]
 27%|██▋       | Scoring GeneralizingEstimator : 329/1225 [00:00<00:02,  347.10it/s]
 28%|██▊       | Scoring GeneralizingEstimator : 341/1225 [00:00<00:02,  347.50it/s]
 29%|██▊       | Scoring GeneralizingEstimator : 351/1225 [00:01<00:02,  344.33it/s]
 30%|██▉       | Scoring GeneralizingEstimator : 364/1225 [00:01<00:02,  346.57it/s]
 31%|███       | Scoring GeneralizingEstimator : 376/1225 [00:01<00:02,  347.19it/s]
 32%|███▏      | Scoring GeneralizingEstimator : 389/1225 [00:01<00:02,  349.50it/s]
 33%|███▎      | Scoring GeneralizingEstimator : 401/1225 [00:01<00:02,  349.88it/s]
 34%|███▎      | Scoring GeneralizingEstimator : 413/1225 [00:01<00:02,  350.27it/s]
 35%|███▍      | Scoring GeneralizingEstimator : 428/1225 [00:01<00:02,  355.15it/s]
 36%|███▌      | Scoring GeneralizingEstimator : 442/1225 [00:01<00:02,  358.69it/s]
 37%|███▋      | Scoring GeneralizingEstimator : 455/1225 [00:01<00:02,  360.14it/s]
 38%|███▊      | Scoring GeneralizingEstimator : 468/1225 [00:01<00:02,  361.64it/s]
 39%|███▉      | Scoring GeneralizingEstimator : 481/1225 [00:01<00:02,  363.03it/s]
 40%|████      | Scoring GeneralizingEstimator : 494/1225 [00:01<00:02,  363.62it/s]
 41%|████      | Scoring GeneralizingEstimator : 505/1225 [00:01<00:01,  361.32it/s]
 42%|████▏     | Scoring GeneralizingEstimator : 519/1225 [00:01<00:01,  363.02it/s]
 44%|████▎     | Scoring GeneralizingEstimator : 533/1225 [00:01<00:01,  364.21it/s]
 45%|████▍     | Scoring GeneralizingEstimator : 546/1225 [00:01<00:01,  365.41it/s]
 46%|████▌     | Scoring GeneralizingEstimator : 558/1225 [00:01<00:01,  364.84it/s]
 47%|████▋     | Scoring GeneralizingEstimator : 571/1225 [00:01<00:01,  365.96it/s]
 48%|████▊     | Scoring GeneralizingEstimator : 584/1225 [00:01<00:01,  367.00it/s]
 49%|████▊     | Scoring GeneralizingEstimator : 597/1225 [00:01<00:01,  368.04it/s]
 50%|████▉     | Scoring GeneralizingEstimator : 611/1225 [00:01<00:01,  370.31it/s]
 51%|█████     | Scoring GeneralizingEstimator : 623/1225 [00:01<00:01,  369.52it/s]
 52%|█████▏    | Scoring GeneralizingEstimator : 635/1225 [00:01<00:01,  368.80it/s]
 53%|█████▎    | Scoring GeneralizingEstimator : 649/1225 [00:01<00:01,  371.26it/s]
 54%|█████▍    | Scoring GeneralizingEstimator : 662/1225 [00:01<00:01,  371.38it/s]
 55%|█████▍    | Scoring GeneralizingEstimator : 673/1225 [00:01<00:01,  369.01it/s]
 56%|█████▌    | Scoring GeneralizingEstimator : 684/1225 [00:01<00:01,  366.79it/s]
 57%|█████▋    | Scoring GeneralizingEstimator : 697/1225 [00:01<00:01,  367.78it/s]
 58%|█████▊    | Scoring GeneralizingEstimator : 710/1225 [00:01<00:01,  368.77it/s]
 59%|█████▉    | Scoring GeneralizingEstimator : 723/1225 [00:02<00:01,  368.10it/s]
 60%|██████    | Scoring GeneralizingEstimator : 735/1225 [00:02<00:01,  367.41it/s]
 61%|██████    | Scoring GeneralizingEstimator : 747/1225 [00:02<00:01,  366.84it/s]
 62%|██████▏   | Scoring GeneralizingEstimator : 761/1225 [00:02<00:01,  369.28it/s]
 63%|██████▎   | Scoring GeneralizingEstimator : 775/1225 [00:02<00:01,  371.65it/s]
 64%|██████▍   | Scoring GeneralizingEstimator : 788/1225 [00:02<00:01,  372.37it/s]
 65%|██████▌   | Scoring GeneralizingEstimator : 801/1225 [00:02<00:01,  372.92it/s]
 66%|██████▋   | Scoring GeneralizingEstimator : 813/1225 [00:02<00:01,  371.99it/s]
 67%|██████▋   | Scoring GeneralizingEstimator : 826/1225 [00:02<00:01,  372.71it/s]
 68%|██████▊   | Scoring GeneralizingEstimator : 838/1225 [00:02<00:01,  371.85it/s]
 69%|██████▉   | Scoring GeneralizingEstimator : 850/1225 [00:02<00:01,  369.70it/s]
 71%|███████   | Scoring GeneralizingEstimator : 864/1225 [00:02<00:00,  372.07it/s]
 72%|███████▏  | Scoring GeneralizingEstimator : 877/1225 [00:02<00:00,  372.74it/s]
 73%|███████▎  | Scoring GeneralizingEstimator : 890/1225 [00:02<00:00,  373.36it/s]
 74%|███████▎  | Scoring GeneralizingEstimator : 901/1225 [00:02<00:00,  370.97it/s]
 75%|███████▍  | Scoring GeneralizingEstimator : 915/1225 [00:02<00:00,  373.21it/s]
 76%|███████▌  | Scoring GeneralizingEstimator : 929/1225 [00:02<00:00,  375.23it/s]
 77%|███████▋  | Scoring GeneralizingEstimator : 942/1225 [00:02<00:00,  375.75it/s]
 78%|███████▊  | Scoring GeneralizingEstimator : 955/1225 [00:02<00:00,  376.25it/s]
 79%|███████▉  | Scoring GeneralizingEstimator : 970/1225 [00:02<00:00,  378.31it/s]
 80%|████████  | Scoring GeneralizingEstimator : 983/1225 [00:02<00:00,  378.71it/s]
 81%|████████▏ | Scoring GeneralizingEstimator : 996/1225 [00:02<00:00,  379.06it/s]
 82%|████████▏ | Scoring GeneralizingEstimator : 1009/1225 [00:02<00:00,  379.39it/s]
 83%|████████▎ | Scoring GeneralizingEstimator : 1022/1225 [00:02<00:00,  379.63it/s]
 84%|████████▍ | Scoring GeneralizingEstimator : 1035/1225 [00:02<00:00,  379.94it/s]
 86%|████████▌ | Scoring GeneralizingEstimator : 1048/1225 [00:02<00:00,  380.24it/s]
 87%|████████▋ | Scoring GeneralizingEstimator : 1060/1225 [00:02<00:00,  379.06it/s]
 87%|████████▋ | Scoring GeneralizingEstimator : 1071/1225 [00:02<00:00,  376.22it/s]
 88%|████████▊ | Scoring GeneralizingEstimator : 1084/1225 [00:02<00:00,  376.07it/s]
 90%|████████▉ | Scoring GeneralizingEstimator : 1097/1225 [00:03<00:00,  376.19it/s]
 91%|█████████ | Scoring GeneralizingEstimator : 1110/1225 [00:03<00:00,  376.42it/s]
 92%|█████████▏| Scoring GeneralizingEstimator : 1122/1225 [00:03<00:00,  375.37it/s]
 93%|█████████▎| Scoring GeneralizingEstimator : 1135/1225 [00:03<00:00,  374.77it/s]
 94%|█████████▎| Scoring GeneralizingEstimator : 1148/1225 [00:03<00:00,  375.34it/s]
 95%|█████████▍| Scoring GeneralizingEstimator : 1161/1225 [00:03<00:00,  375.88it/s]
 96%|█████████▌| Scoring GeneralizingEstimator : 1174/1225 [00:03<00:00,  376.27it/s]
 97%|█████████▋| Scoring GeneralizingEstimator : 1187/1225 [00:03<00:00,  376.64it/s]
 98%|█████████▊| Scoring GeneralizingEstimator : 1200/1225 [00:03<00:00,  377.08it/s]
 99%|█████████▉| Scoring GeneralizingEstimator : 1213/1225 [00:03<00:00,  377.50it/s]
100%|██████████| Scoring GeneralizingEstimator : 1225/1225 [00:03<00:00,  378.90it/s]
100%|██████████| Scoring GeneralizingEstimator : 1225/1225 [00:03<00:00,  366.96it/s]

Plot

fig, ax = plt.subplots(layout="constrained")
im = ax.matshow(
    scores,
    vmin=0,
    vmax=1.0,
    cmap="RdBu_r",
    origin="lower",
    extent=epochs.times[[0, -1, 0, -1]],
)
ax.axhline(0.0, color="k")
ax.axvline(0.0, color="k")
ax.xaxis.set_ticks_position("bottom")
ax.set_xlabel(
    'Condition: "Right"\nTesting Time (s)',
)
ax.set_ylabel('Condition: "Left"\nTraining Time (s)')
ax.set_title("Generalization across time and condition", fontweight="bold")
fig.colorbar(im, ax=ax, label="Performance (ROC AUC)")
plt.show()

References

[1]

Jean-Rémi King and Stanislas Dehaene. Characterizing the dynamics of mental representations: the temporal generalization method. Trends in Cognitive Sciences, 18(4):203–210, 2014. doi:10.1016/j.tics.2014.01.002.