Decoding (MVPA)

Design philosophy

Decoding (a.k.a. MVPA) in MNE largely follows the machine learning API of the scikit-learn package. Each estimator implements fit, transform, fit_transform, and (optionally) inverse_transform methods. For more details on this design, visit scikit-learn. For additional theoretical insights into the decoding framework in MNE [1].

For ease of comprehension, we will denote instantiations of the class using the same name as the class but in small caps instead of camel cases.

Let’s start by loading data for a simple two-class problem:

import matplotlib.pyplot as plt
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

import mne
from mne.datasets import sample
from mne.decoding import (
    CSP,
    GeneralizingEstimator,
    LinearModel,
    Scaler,
    SlidingEstimator,
    Vectorizer,
    cross_val_multiscore,
    get_coef,
)

data_path = sample.data_path()

subjects_dir = data_path / "subjects"
meg_path = data_path / "MEG" / "sample"
raw_fname = meg_path / "sample_audvis_filt-0-40_raw.fif"
tmin, tmax = -0.200, 0.500
event_id = {"Auditory/Left": 1, "Visual/Left": 3}  # just use two
raw = mne.io.read_raw_fif(raw_fname)
raw.pick(picks=["grad", "stim", "eog"])

# The subsequent decoding analyses only capture evoked responses, so we can
# low-pass the MEG data. Usually a value more like 40 Hz would be used,
# but here low-pass at 20 so we can more heavily decimate, and allow
# the example to run faster. The 2 Hz high-pass helps improve CSP.
raw.load_data().filter(2, 20)
events = mne.find_events(raw, "STI 014")

# Set up bad channels (modify to your needs)
raw.info["bads"] += ["MEG 2443"]  # bads + 2 more

# Read epochs
epochs = mne.Epochs(
    raw,
    events,
    event_id,
    tmin,
    tmax,
    proj=True,
    picks=("grad", "eog"),
    baseline=(None, 0.0),
    preload=True,
    reject=dict(grad=4000e-13, eog=150e-6),
    decim=3,
    verbose="error",
)
epochs.pick(picks="meg", exclude="bads")  # remove stim and EOG
del raw

X = epochs.get_data(copy=False)  # MEG signals: n_epochs, n_meg_channels, n_times
y = epochs.events[:, 2]  # target: auditory left vs visual left

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_filt-0-40_raw.fif...
    Read a total of 4 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
        Average EEG reference (1 x 60)  idle
    Range : 6450 ... 48149 =     42.956 ...   320.665 secs
Ready.
Reading 0 ... 41699  =      0.000 ...   277.709 secs...
Filtering raw data in 1 contiguous segment
Setting up band-pass filter from 2 - 20 Hz

FIR filter parameters
---------------------
Designing a one-pass, zero-phase, non-causal bandpass filter:
- Windowed time-domain design (firwin) method
- Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation
- Lower passband edge: 2.00
- Lower transition bandwidth: 2.00 Hz (-6 dB cutoff frequency: 1.00 Hz)
- Upper passband edge: 20.00 Hz
- Upper transition bandwidth: 5.00 Hz (-6 dB cutoff frequency: 22.50 Hz)
- Filter length: 249 samples (1.658 s)

[Parallel(n_jobs=1)]: Done  17 tasks      | elapsed:    0.0s
[Parallel(n_jobs=1)]: Done  71 tasks      | elapsed:    0.1s
[Parallel(n_jobs=1)]: Done 161 tasks      | elapsed:    0.1s
319 events found on stim channel STI 014
Event IDs: [ 1  2  3  4  5 32]

Transformation classes

Scaler

The mne.decoding.Scaler will standardize the data based on channel scales. In the simplest modes scalings=None or scalings=dict(...), each data channel type (e.g., mag, grad, eeg) is treated separately and scaled by a constant. This is the approach used by e.g., mne.compute_covariance() to standardize channel scales.

If scalings='mean' or scalings='median', each channel is scaled using empirical measures. Each channel is scaled independently by the mean and standand deviation, or median and interquartile range, respectively, across all epochs and time points during fit (during training). The transform() method is called to transform data (training or test set) by scaling all time points and epochs on a channel-by-channel basis. To perform both the fit and transform operations in a single call, the fit_transform() method may be used. To invert the transform, inverse_transform() can be used. For scalings='median', scikit-learn version 0.17+ is required.

Note

Using this class is different from directly applying sklearn.preprocessing.StandardScaler or sklearn.preprocessing.RobustScaler offered by scikit-learn. These scale each classification feature, e.g. each time point for each channel, with mean and standard deviation computed across epochs, whereas mne.decoding.Scaler scales each channel using mean and standard deviation computed across all of its time points and epochs.

Vectorizer

Scikit-learn API provides functionality to chain transformers and estimators by using sklearn.pipeline.Pipeline. We can construct decoding pipelines and perform cross-validation and grid-search. However scikit-learn transformers and estimators generally expect 2D data (n_samples * n_features), whereas MNE transformers typically output data with a higher dimensionality (e.g. n_samples * n_channels * n_frequencies * n_times). A Vectorizer therefore needs to be applied between the MNE and the scikit-learn steps like:

# Uses all MEG sensors and time points as separate classification
# features, so the resulting filters used are spatio-temporal
clf = make_pipeline(
    Scaler(epochs.info),
    Vectorizer(),
    LogisticRegression(solver="liblinear"),  # liblinear is faster than lbfgs
)

scores = cross_val_multiscore(clf, X, y, cv=5, n_jobs=None)

# Mean scores across cross-validation splits
score = np.mean(scores, axis=0)
print(f"Spatio-temporal: {100 * score:0.1f}%")

Spatio-temporal: 99.2%

PSDEstimator

The mne.decoding.PSDEstimator computes the power spectral density (PSD) using the multitaper method. It takes a 3D array as input, converts it into 2D and computes the PSD.

FilterEstimator

The mne.decoding.FilterEstimator filters the 3D epochs data.

Spatial filters

Just like temporal filters, spatial filters provide weights to modify the data along the sensor dimension. They are popular in the BCI community because of their simplicity and ability to distinguish spatially-separated neural activity.

Common spatial pattern

mne.decoding.CSP is a technique to analyze multichannel data based on recordings from two classes [2] (see also https://en.wikipedia.org/wiki/Common_spatial_pattern).

Let \(X \in R^{C\times T}\) be a segment of data with \(C\) channels and \(T\) time points. The data at a single time point is denoted by \(x(t)\) such that \(X=[x(t), x(t+1), ..., x(t+T-1)]\). Common spatial pattern (CSP) finds a decomposition that projects the signal in the original sensor space to CSP space using the following transformation:

(1)\[x_{CSP}(t) = W^{T}x(t)\]

where each column of \(W \in R^{C\times C}\) is a spatial filter and each row of \(x_{CSP}\) is a CSP component. The matrix \(W\) is also called the de-mixing matrix in other contexts. Let \(\Sigma^{+} \in R^{C\times C}\) and \(\Sigma^{-} \in R^{C\times C}\) be the estimates of the covariance matrices of the two conditions. CSP analysis is given by the simultaneous diagonalization of the two covariance matrices

(2)\[W^{T}\Sigma^{+}W = \lambda^{+}\]

(3)\[W^{T}\Sigma^{-}W = \lambda^{-}\]

where \(\lambda^{C}\) is a diagonal matrix whose entries are the eigenvalues of the following generalized eigenvalue problem

(4)\[\Sigma^{+}w = \lambda \Sigma^{-}w\]

Large entries in the diagonal matrix corresponds to a spatial filter which gives high variance in one class but low variance in the other. Thus, the filter facilitates discrimination between the two classes.

Examples

• Motor imagery decoding from EEG data using the Common Spatial Pattern (CSP)

• Decoding in time-frequency space using Common Spatial Patterns (CSP)

Note

The winning entry of the Grasp-and-lift EEG competition in Kaggle used the CSP implementation in MNE and was featured as a script of the week.

We can use CSP with these data with:

csp = CSP(n_components=3, norm_trace=False)
clf_csp = make_pipeline(csp, LinearModel(LogisticRegression(solver="liblinear")))
scores = cross_val_multiscore(clf_csp, X, y, cv=5, n_jobs=None)
print(f"CSP: {100 * scores.mean():0.1f}%")

Computing rank from data with rank=None
    Using tolerance 6e-11 (2.2e-16 eps * 203 dim * 1.3e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.
Computing rank from data with rank=None
    Using tolerance 6e-11 (2.2e-16 eps * 203 dim * 1.3e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.
Computing rank from data with rank=None
    Using tolerance 6e-11 (2.2e-16 eps * 203 dim * 1.3e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.
Computing rank from data with rank=None
    Using tolerance 5.8e-11 (2.2e-16 eps * 203 dim * 1.3e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.
Computing rank from data with rank=None
    Using tolerance 5.9e-11 (2.2e-16 eps * 203 dim * 1.3e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.
CSP: 89.4%

Source power comodulation (SPoC)

Source Power Comodulation (mne.decoding.SPoC) [3] identifies the composition of orthogonal spatial filters that maximally correlate with a continuous target.

SPoC can be seen as an extension of the CSP where the target is driven by a continuous variable rather than a discrete variable. Typical applications include extraction of motor patterns using EMG power or audio patterns using sound envelope.

Examples

• Continuous Target Decoding with SPoC

xDAWN

mne.preprocessing.Xdawn is a spatial filtering method designed to improve the signal to signal + noise ratio (SSNR) of the ERP responses [4]. Xdawn was originally designed for P300 evoked potential by enhancing the target response with respect to the non-target response. The implementation in MNE-Python is a generalization to any type of ERP.

Examples

• XDAWN Denoising

• XDAWN Decoding From EEG data

Effect-matched spatial filtering

The result of mne.decoding.EMS is a spatial filter at each time point and a corresponding time course [5]. Intuitively, the result gives the similarity between the filter at each time point and the data vector (sensors) at that time point.

Examples

• Compute effect-matched-spatial filtering (EMS)

Patterns vs. filters

When interpreting the components of the CSP (or spatial filters in general), it is often more intuitive to think about how \(x(t)\) is composed of the different CSP components \(x_{CSP}(t)\). In other words, we can rewrite Equation (1) as follows:

(5)\[x(t) = (W^{-1})^{T}x_{CSP}(t)\]

The columns of the matrix \((W^{-1})^T\) are called spatial patterns. This is also called the mixing matrix. The example Linear classifier on sensor data with plot patterns and filters discusses the difference between patterns and filters.

These can be plotted with:

# Fit CSP on full data and plot
csp.fit(X, y)
csp.plot_patterns(epochs.info)
csp.plot_filters(epochs.info, scalings=1e-9)

• 
• 

Computing rank from data with rank=None
    Using tolerance 6.6e-11 (2.2e-16 eps * 203 dim * 1.5e+03  max singular value)
    Estimated rank (data): 203
    data: rank 203 computed from 203 data channels with 0 projectors
Reducing data rank from 203 -> 203
Estimating class=1 covariance using EMPIRICAL
Done.
Estimating class=3 covariance using EMPIRICAL
Done.

Decoding over time

This strategy consists in fitting a multivariate predictive model on each time instant and evaluating its performance at the same instant on new epochs. The mne.decoding.SlidingEstimator will take as input a pair of features \(X\) and targets \(y\), where \(X\) has more than 2 dimensions. For decoding over time the data \(X\) is the epochs data of shape n_epochs × n_channels × n_times. As the last dimension of \(X\) is the time, an estimator will be fit on every time instant.

This approach is analogous to SlidingEstimator-based approaches in fMRI, where here we are interested in when one can discriminate experimental conditions and therefore figure out when the effect of interest happens.

When working with linear models as estimators, this approach boils down to estimating a discriminative spatial filter for each time instant.

Temporal decoding

We’ll use a Logistic Regression for a binary classification as machine learning model.

# We will train the classifier on all left visual vs auditory trials on MEG

clf = make_pipeline(StandardScaler(), LogisticRegression(solver="liblinear"))

time_decod = SlidingEstimator(clf, n_jobs=None, scoring="roc_auc", verbose=True)
# here we use cv=3 just for speed
scores = cross_val_multiscore(time_decod, X, y, cv=3, n_jobs=None)

# Mean scores across cross-validation splits
scores = np.mean(scores, axis=0)

# Plot
fig, ax = plt.subplots()
ax.plot(epochs.times, scores, label="score")
ax.axhline(0.5, color="k", linestyle="--", label="chance")
ax.set_xlabel("Times")
ax.set_ylabel("AUC")  # Area Under the Curve
ax.legend()
ax.axvline(0.0, color="k", linestyle="-")
ax.set_title("Sensor space decoding")

  0%|          | Fitting SlidingEstimator : 0/36 [00:00<?,       ?it/s]
 19%|█▉        | Fitting SlidingEstimator : 7/36 [00:00<00:00,  206.93it/s]
 44%|████▍     | Fitting SlidingEstimator : 16/36 [00:00<00:00,  237.53it/s]
 69%|██████▉   | Fitting SlidingEstimator : 25/36 [00:00<00:00,  247.89it/s]
 92%|█████████▏| Fitting SlidingEstimator : 33/36 [00:00<00:00,  245.27it/s]
100%|██████████| Fitting SlidingEstimator : 36/36 [00:00<00:00,  249.80it/s]

  0%|          | Fitting SlidingEstimator : 0/36 [00:00<?,       ?it/s]
 22%|██▏       | Fitting SlidingEstimator : 8/36 [00:00<00:00,  235.83it/s]
 44%|████▍     | Fitting SlidingEstimator : 16/36 [00:00<00:00,  236.82it/s]
 72%|███████▏  | Fitting SlidingEstimator : 26/36 [00:00<00:00,  257.77it/s]
 94%|█████████▍| Fitting SlidingEstimator : 34/36 [00:00<00:00,  252.48it/s]
100%|██████████| Fitting SlidingEstimator : 36/36 [00:00<00:00,  255.22it/s]

  0%|          | Fitting SlidingEstimator : 0/36 [00:00<?,       ?it/s]
 19%|█▉        | Fitting SlidingEstimator : 7/36 [00:00<00:00,  207.36it/s]
 47%|████▋     | Fitting SlidingEstimator : 17/36 [00:00<00:00,  252.42it/s]
 75%|███████▌  | Fitting SlidingEstimator : 27/36 [00:00<00:00,  266.81it/s]
 97%|█████████▋| Fitting SlidingEstimator : 35/36 [00:00<00:00,  259.04it/s]
100%|██████████| Fitting SlidingEstimator : 36/36 [00:00<00:00,  261.25it/s]

You can retrieve the spatial filters and spatial patterns if you explicitly use a LinearModel

clf = make_pipeline(
    StandardScaler(), LinearModel(LogisticRegression(solver="liblinear"))
)
time_decod = SlidingEstimator(clf, n_jobs=None, scoring="roc_auc", verbose=True)
time_decod.fit(X, y)

coef = get_coef(time_decod, "patterns_", inverse_transform=True)
evoked_time_gen = mne.EvokedArray(coef, epochs.info, tmin=epochs.times[0])
joint_kwargs = dict(ts_args=dict(time_unit="s"), topomap_args=dict(time_unit="s"))
evoked_time_gen.plot_joint(
    times=np.arange(0.0, 0.500, 0.100), title="patterns", **joint_kwargs
)

  0%|          | Fitting SlidingEstimator : 0/36 [00:00<?,       ?it/s]
 11%|█         | Fitting SlidingEstimator : 4/36 [00:00<00:00,  118.60it/s]
 25%|██▌       | Fitting SlidingEstimator : 9/36 [00:00<00:00,  132.49it/s]
 36%|███▌      | Fitting SlidingEstimator : 13/36 [00:00<00:00,  127.70it/s]
 53%|█████▎    | Fitting SlidingEstimator : 19/36 [00:00<00:00,  141.26it/s]
 67%|██████▋   | Fitting SlidingEstimator : 24/36 [00:00<00:00,  142.79it/s]
 81%|████████  | Fitting SlidingEstimator : 29/36 [00:00<00:00,  143.89it/s]
 94%|█████████▍| Fitting SlidingEstimator : 34/36 [00:00<00:00,  144.63it/s]
100%|██████████| Fitting SlidingEstimator : 36/36 [00:00<00:00,  144.45it/s]
No projector specified for this dataset. Please consider the method self.add_proj.

Temporal generalization

Temporal generalization is an extension of the decoding over time approach. It consists in evaluating whether the model estimated at a particular time instant accurately predicts any other time instant. It is analogous to transferring a trained model to a distinct learning problem, where the problems correspond to decoding the patterns of brain activity recorded at distinct time instants.

The object to for Temporal generalization is mne.decoding.GeneralizingEstimator. It expects as input \(X\) and \(y\) (similarly to SlidingEstimator) but generates predictions from each model for all time instants. The class GeneralizingEstimator is generic and will treat the last dimension as the one to be used for generalization testing. For convenience, here, we refer to it as different tasks. If \(X\) corresponds to epochs data then the last dimension is time.

This runs the analysis used in [6] and further detailed in [7]:

# define the Temporal generalization object
time_gen = GeneralizingEstimator(clf, n_jobs=None, scoring="roc_auc", verbose=True)

# again, cv=3 just for speed
scores = cross_val_multiscore(time_gen, X, y, cv=3, n_jobs=None)

# Mean scores across cross-validation splits
scores = np.mean(scores, axis=0)

# Plot the diagonal (it's exactly the same as the time-by-time decoding above)
fig, ax = plt.subplots()
ax.plot(epochs.times, np.diag(scores), label="score")
ax.axhline(0.5, color="k", linestyle="--", label="chance")
ax.set_xlabel("Times")
ax.set_ylabel("AUC")
ax.legend()
ax.axvline(0.0, color="k", linestyle="-")
ax.set_title("Decoding MEG sensors over time")

  0%|          | Fitting GeneralizingEstimator : 0/36 [00:00<?,       ?it/s]
 17%|█▋        | Fitting GeneralizingEstimator : 6/36 [00:00<00:00,  177.82it/s]
 36%|███▌      | Fitting GeneralizingEstimator : 13/36 [00:00<00:00,  192.96it/s]
 58%|█████▊    | Fitting GeneralizingEstimator : 21/36 [00:00<00:00,  208.63it/s]
 81%|████████  | Fitting GeneralizingEstimator : 29/36 [00:00<00:00,  216.17it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  214.95it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  213.75it/s]

  0%|          | Scoring GeneralizingEstimator : 0/1296 [00:00<?,       ?it/s]
  2%|▏         | Scoring GeneralizingEstimator : 23/1296 [00:00<00:01,  676.25it/s]
  4%|▎         | Scoring GeneralizingEstimator : 47/1296 [00:00<00:01,  683.72it/s]
  5%|▌         | Scoring GeneralizingEstimator : 71/1296 [00:00<00:01,  686.56it/s]
  7%|▋         | Scoring GeneralizingEstimator : 95/1296 [00:00<00:01,  686.82it/s]
  9%|▉         | Scoring GeneralizingEstimator : 118/1296 [00:00<00:01,  686.26it/s]
 11%|█         | Scoring GeneralizingEstimator : 142/1296 [00:00<00:01,  685.63it/s]
 13%|█▎        | Scoring GeneralizingEstimator : 165/1296 [00:00<00:01,  684.53it/s]
 15%|█▍        | Scoring GeneralizingEstimator : 189/1296 [00:00<00:01,  685.08it/s]
 16%|█▋        | Scoring GeneralizingEstimator : 213/1296 [00:00<00:01,  685.51it/s]
 18%|█▊        | Scoring GeneralizingEstimator : 236/1296 [00:00<00:01,  681.82it/s]
 20%|█▉        | Scoring GeneralizingEstimator : 259/1296 [00:00<00:01,  681.97it/s]
 22%|██▏       | Scoring GeneralizingEstimator : 282/1296 [00:00<00:01,  682.08it/s]
 24%|██▎       | Scoring GeneralizingEstimator : 305/1296 [00:00<00:01,  682.13it/s]
 25%|██▌       | Scoring GeneralizingEstimator : 328/1296 [00:00<00:01,  682.26it/s]
 27%|██▋       | Scoring GeneralizingEstimator : 352/1296 [00:00<00:01,  682.68it/s]
 29%|██▉       | Scoring GeneralizingEstimator : 376/1296 [00:00<00:01,  682.71it/s]
 31%|███       | Scoring GeneralizingEstimator : 399/1296 [00:00<00:01,  682.73it/s]
 33%|███▎      | Scoring GeneralizingEstimator : 422/1296 [00:00<00:01,  682.74it/s]
 34%|███▍      | Scoring GeneralizingEstimator : 445/1296 [00:00<00:01,  682.46it/s]
 36%|███▌      | Scoring GeneralizingEstimator : 468/1296 [00:00<00:01,  682.30it/s]
 38%|███▊      | Scoring GeneralizingEstimator : 492/1296 [00:00<00:01,  682.47it/s]
 40%|███▉      | Scoring GeneralizingEstimator : 516/1296 [00:00<00:01,  682.68it/s]
 42%|████▏     | Scoring GeneralizingEstimator : 539/1296 [00:00<00:01,  682.74it/s]
 43%|████▎     | Scoring GeneralizingEstimator : 562/1296 [00:00<00:01,  681.82it/s]
 45%|████▌     | Scoring GeneralizingEstimator : 586/1296 [00:00<00:01,  682.67it/s]
 47%|████▋     | Scoring GeneralizingEstimator : 609/1296 [00:00<00:01,  682.77it/s]
 49%|████▉     | Scoring GeneralizingEstimator : 632/1296 [00:00<00:00,  682.64it/s]
 51%|█████     | Scoring GeneralizingEstimator : 655/1296 [00:00<00:00,  682.52it/s]
 52%|█████▏    | Scoring GeneralizingEstimator : 678/1296 [00:00<00:00,  682.14it/s]
 54%|█████▍    | Scoring GeneralizingEstimator : 701/1296 [00:01<00:00,  682.03it/s]
 56%|█████▌    | Scoring GeneralizingEstimator : 724/1296 [00:01<00:00,  682.02it/s]
 58%|█████▊    | Scoring GeneralizingEstimator : 747/1296 [00:01<00:00,  682.05it/s]
 59%|█████▉    | Scoring GeneralizingEstimator : 770/1296 [00:01<00:00,  682.06it/s]
 61%|██████    | Scoring GeneralizingEstimator : 793/1296 [00:01<00:00,  682.20it/s]
 63%|██████▎   | Scoring GeneralizingEstimator : 816/1296 [00:01<00:00,  682.07it/s]
 65%|██████▍   | Scoring GeneralizingEstimator : 839/1296 [00:01<00:00,  681.78it/s]
 67%|██████▋   | Scoring GeneralizingEstimator : 863/1296 [00:01<00:00,  681.91it/s]
 68%|██████▊   | Scoring GeneralizingEstimator : 887/1296 [00:01<00:00,  681.81it/s]
 70%|███████   | Scoring GeneralizingEstimator : 910/1296 [00:01<00:00,  681.82it/s]
 72%|███████▏  | Scoring GeneralizingEstimator : 933/1296 [00:01<00:00,  681.86it/s]
 74%|███████▍  | Scoring GeneralizingEstimator : 956/1296 [00:01<00:00,  681.72it/s]
 76%|███████▌  | Scoring GeneralizingEstimator : 979/1296 [00:01<00:00,  681.71it/s]
 77%|███████▋  | Scoring GeneralizingEstimator : 1003/1296 [00:01<00:00,  681.81it/s]
 79%|███████▉  | Scoring GeneralizingEstimator : 1026/1296 [00:01<00:00,  681.50it/s]
 81%|████████  | Scoring GeneralizingEstimator : 1049/1296 [00:01<00:00,  681.42it/s]
 83%|████████▎ | Scoring GeneralizingEstimator : 1072/1296 [00:01<00:00,  681.18it/s]
 84%|████████▍ | Scoring GeneralizingEstimator : 1095/1296 [00:01<00:00,  680.94it/s]
 86%|████████▋ | Scoring GeneralizingEstimator : 1118/1296 [00:01<00:00,  681.05it/s]
 88%|████████▊ | Scoring GeneralizingEstimator : 1141/1296 [00:01<00:00,  681.16it/s]
 90%|████████▉ | Scoring GeneralizingEstimator : 1164/1296 [00:01<00:00,  681.21it/s]
 92%|█████████▏| Scoring GeneralizingEstimator : 1187/1296 [00:01<00:00,  681.13it/s]
 93%|█████████▎| Scoring GeneralizingEstimator : 1210/1296 [00:01<00:00,  680.84it/s]
 95%|█████████▌| Scoring GeneralizingEstimator : 1232/1296 [00:01<00:00,  679.39it/s]
 97%|█████████▋| Scoring GeneralizingEstimator : 1254/1296 [00:01<00:00,  677.92it/s]
 99%|█████████▊| Scoring GeneralizingEstimator : 1278/1296 [00:01<00:00,  678.36it/s]
100%|██████████| Scoring GeneralizingEstimator : 1296/1296 [00:01<00:00,  678.25it/s]
100%|██████████| Scoring GeneralizingEstimator : 1296/1296 [00:01<00:00,  680.57it/s]

  0%|          | Fitting GeneralizingEstimator : 0/36 [00:00<?,       ?it/s]
 17%|█▋        | Fitting GeneralizingEstimator : 6/36 [00:00<00:00,  177.70it/s]
 39%|███▉      | Fitting GeneralizingEstimator : 14/36 [00:00<00:00,  206.18it/s]
 61%|██████    | Fitting GeneralizingEstimator : 22/36 [00:00<00:00,  217.25it/s]
 81%|████████  | Fitting GeneralizingEstimator : 29/36 [00:00<00:00,  214.83it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  218.39it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  217.33it/s]

  0%|          | Scoring GeneralizingEstimator : 0/1296 [00:00<?,       ?it/s]
  2%|▏         | Scoring GeneralizingEstimator : 23/1296 [00:00<00:01,  662.23it/s]
  4%|▎         | Scoring GeneralizingEstimator : 46/1296 [00:00<00:01,  670.07it/s]
  5%|▌         | Scoring GeneralizingEstimator : 69/1296 [00:00<00:01,  674.63it/s]
  7%|▋         | Scoring GeneralizingEstimator : 92/1296 [00:00<00:01,  677.18it/s]
  9%|▉         | Scoring GeneralizingEstimator : 115/1296 [00:00<00:01,  677.48it/s]
 11%|█         | Scoring GeneralizingEstimator : 138/1296 [00:00<00:01,  678.42it/s]
 12%|█▏        | Scoring GeneralizingEstimator : 161/1296 [00:00<00:01,  678.52it/s]
 14%|█▍        | Scoring GeneralizingEstimator : 184/1296 [00:00<00:01,  679.31it/s]
 16%|█▌        | Scoring GeneralizingEstimator : 207/1296 [00:00<00:01,  678.69it/s]
 18%|█▊        | Scoring GeneralizingEstimator : 230/1296 [00:00<00:01,  678.97it/s]
 20%|█▉        | Scoring GeneralizingEstimator : 253/1296 [00:00<00:01,  679.00it/s]
 21%|██▏       | Scoring GeneralizingEstimator : 276/1296 [00:00<00:01,  679.47it/s]
 23%|██▎       | Scoring GeneralizingEstimator : 299/1296 [00:00<00:01,  679.57it/s]
 25%|██▍       | Scoring GeneralizingEstimator : 323/1296 [00:00<00:01,  679.79it/s]
 27%|██▋       | Scoring GeneralizingEstimator : 346/1296 [00:00<00:01,  680.21it/s]
 28%|██▊       | Scoring GeneralizingEstimator : 369/1296 [00:00<00:01,  680.53it/s]
 30%|███       | Scoring GeneralizingEstimator : 392/1296 [00:00<00:01,  680.03it/s]
 32%|███▏      | Scoring GeneralizingEstimator : 415/1296 [00:00<00:01,  680.37it/s]
 34%|███▍      | Scoring GeneralizingEstimator : 438/1296 [00:00<00:01,  679.97it/s]
 36%|███▌      | Scoring GeneralizingEstimator : 462/1296 [00:00<00:01,  680.36it/s]
 38%|███▊      | Scoring GeneralizingEstimator : 486/1296 [00:00<00:01,  681.07it/s]
 39%|███▉      | Scoring GeneralizingEstimator : 509/1296 [00:00<00:01,  680.94it/s]
 41%|████      | Scoring GeneralizingEstimator : 532/1296 [00:00<00:01,  680.37it/s]
 43%|████▎     | Scoring GeneralizingEstimator : 556/1296 [00:00<00:01,  681.60it/s]
 45%|████▍     | Scoring GeneralizingEstimator : 580/1296 [00:00<00:01,  681.81it/s]
 47%|████▋     | Scoring GeneralizingEstimator : 604/1296 [00:00<00:01,  682.09it/s]
 48%|████▊     | Scoring GeneralizingEstimator : 628/1296 [00:00<00:00,  682.31it/s]
 50%|█████     | Scoring GeneralizingEstimator : 651/1296 [00:00<00:00,  682.44it/s]
 52%|█████▏    | Scoring GeneralizingEstimator : 675/1296 [00:00<00:00,  682.89it/s]
 54%|█████▍    | Scoring GeneralizingEstimator : 698/1296 [00:01<00:00,  682.74it/s]
 56%|█████▌    | Scoring GeneralizingEstimator : 721/1296 [00:01<00:00,  682.52it/s]
 57%|█████▋    | Scoring GeneralizingEstimator : 745/1296 [00:01<00:00,  682.82it/s]
 59%|█████▉    | Scoring GeneralizingEstimator : 769/1296 [00:01<00:00,  682.82it/s]
 61%|██████    | Scoring GeneralizingEstimator : 792/1296 [00:01<00:00,  682.88it/s]
 63%|██████▎   | Scoring GeneralizingEstimator : 816/1296 [00:01<00:00,  683.17it/s]
 65%|██████▍   | Scoring GeneralizingEstimator : 840/1296 [00:01<00:00,  683.26it/s]
 67%|██████▋   | Scoring GeneralizingEstimator : 864/1296 [00:01<00:00,  683.37it/s]
 69%|██████▊   | Scoring GeneralizingEstimator : 888/1296 [00:01<00:00,  683.84it/s]
 70%|███████   | Scoring GeneralizingEstimator : 910/1296 [00:01<00:00,  680.51it/s]
 72%|███████▏  | Scoring GeneralizingEstimator : 930/1296 [00:01<00:00,  674.10it/s]
 74%|███████▎  | Scoring GeneralizingEstimator : 953/1296 [00:01<00:00,  673.11it/s]
 75%|███████▌  | Scoring GeneralizingEstimator : 975/1296 [00:01<00:00,  670.97it/s]
 77%|███████▋  | Scoring GeneralizingEstimator : 999/1296 [00:01<00:00,  671.81it/s]
 79%|███████▉  | Scoring GeneralizingEstimator : 1022/1296 [00:01<00:00,  672.38it/s]
 81%|████████  | Scoring GeneralizingEstimator : 1046/1296 [00:01<00:00,  673.06it/s]
 83%|████████▎ | Scoring GeneralizingEstimator : 1070/1296 [00:01<00:00,  673.69it/s]
 84%|████████▍ | Scoring GeneralizingEstimator : 1093/1296 [00:01<00:00,  674.17it/s]
 86%|████████▌ | Scoring GeneralizingEstimator : 1116/1296 [00:01<00:00,  674.48it/s]
 88%|████████▊ | Scoring GeneralizingEstimator : 1139/1296 [00:01<00:00,  674.60it/s]
 90%|████████▉ | Scoring GeneralizingEstimator : 1162/1296 [00:01<00:00,  674.62it/s]
 91%|█████████▏| Scoring GeneralizingEstimator : 1185/1296 [00:01<00:00,  674.86it/s]
 93%|█████████▎| Scoring GeneralizingEstimator : 1209/1296 [00:01<00:00,  675.61it/s]
 95%|█████████▌| Scoring GeneralizingEstimator : 1232/1296 [00:01<00:00,  675.88it/s]
 97%|█████████▋| Scoring GeneralizingEstimator : 1255/1296 [00:01<00:00,  676.12it/s]
 99%|█████████▊| Scoring GeneralizingEstimator : 1278/1296 [00:01<00:00,  676.50it/s]
100%|██████████| Scoring GeneralizingEstimator : 1296/1296 [00:01<00:00,  676.92it/s]
100%|██████████| Scoring GeneralizingEstimator : 1296/1296 [00:01<00:00,  677.87it/s]

  0%|          | Fitting GeneralizingEstimator : 0/36 [00:00<?,       ?it/s]
 17%|█▋        | Fitting GeneralizingEstimator : 6/36 [00:00<00:00,  177.87it/s]
 39%|███▉      | Fitting GeneralizingEstimator : 14/36 [00:00<00:00,  207.55it/s]
 61%|██████    | Fitting GeneralizingEstimator : 22/36 [00:00<00:00,  218.18it/s]
 83%|████████▎ | Fitting GeneralizingEstimator : 30/36 [00:00<00:00,  223.39it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  224.41it/s]
100%|██████████| Fitting GeneralizingEstimator : 36/36 [00:00<00:00,  223.05it/s]

  0%|          | Scoring GeneralizingEstimator : 0/1296 [00:00<?,       ?it/s]
  2%|▏         | Scoring GeneralizingEstimator : 23/1296 [00:00<00:01,  661.36it/s]
  4%|▎         | Scoring GeneralizingEstimator : 46/1296 [00:00<00:01,  671.99it/s]
  5%|▌         | Scoring GeneralizingEstimator : 69/1296 [00:00<00:01,  675.73it/s]
  7%|▋         | Scoring GeneralizingEstimator : 93/1296 [00:00<00:01,  678.50it/s]
  9%|▉         | Scoring GeneralizingEstimator : 116/1296 [00:00<00:01,  679.67it/s]
 11%|█         | Scoring GeneralizingEstimator : 139/1296 [00:00<00:01,  679.88it/s]
 12%|█▎        | Scoring GeneralizingEstimator : 162/1296 [00:00<00:01,  679.86it/s]
 14%|█▍        | Scoring GeneralizingEstimator : 185/1296 [00:00<00:01,  679.54it/s]
 16%|█▌        | Scoring GeneralizingEstimator : 208/1296 [00:00<00:01,  676.59it/s]
 18%|█▊        | Scoring GeneralizingEstimator : 231/1296 [00:00<00:01,  676.24it/s]
 20%|█▉        | Scoring GeneralizingEstimator : 254/1296 [00:00<00:01,  676.42it/s]
 21%|██▏       | Scoring GeneralizingEstimator : 277/1296 [00:00<00:01,  676.07it/s]
 23%|██▎       | Scoring GeneralizingEstimator : 300/1296 [00:00<00:01,  674.85it/s]
 25%|██▍       | Scoring GeneralizingEstimator : 323/1296 [00:00<00:01,  675.08it/s]
 27%|██▋       | Scoring GeneralizingEstimator : 346/1296 [00:00<00:01,  675.25it/s]
 28%|██▊       | Scoring GeneralizingEstimator : 369/1296 [00:00<00:01,  675.45it/s]
 30%|███       | Scoring GeneralizingEstimator : 392/1296 [00:00<00:01,  675.08it/s]
 32%|███▏      | Scoring GeneralizingEstimator : 415/1296 [00:00<00:01,  675.71it/s]
 34%|███▍      | Scoring GeneralizingEstimator : 438/1296 [00:00<00:01,  675.31it/s]
 36%|███▌      | Scoring GeneralizingEstimator : 461/1296 [00:00<00:01,  675.40it/s]
 37%|███▋      | Scoring GeneralizingEstimator : 484/1296 [00:00<00:01,  676.05it/s]
 39%|███▉      | Scoring GeneralizingEstimator : 507/1296 [00:00<00:01,  675.89it/s]
 41%|████      | Scoring GeneralizingEstimator : 530/1296 [00:00<00:01,  675.65it/s]
 43%|████▎     | Scoring GeneralizingEstimator : 553/1296 [00:00<00:01,  675.78it/s]
 44%|████▍     | Scoring GeneralizingEstimator : 576/1296 [00:00<00:01,  675.12it/s]
 46%|████▌     | Scoring GeneralizingEstimator : 599/1296 [00:00<00:01,  675.11it/s]
 48%|████▊     | Scoring GeneralizingEstimator : 622/1296 [00:00<00:00,  675.08it/s]
 50%|████▉     | Scoring GeneralizingEstimator : 645/1296 [00:00<00:00,  675.36it/s]
 52%|█████▏    | Scoring GeneralizingEstimator : 668/1296 [00:00<00:00,  675.14it/s]
 53%|█████▎    | Scoring GeneralizingEstimator : 691/1296 [00:01<00:00,  674.77it/s]
 55%|█████▌    | Scoring GeneralizingEstimator : 714/1296 [00:01<00:00,  674.52it/s]
 57%|█████▋    | Scoring GeneralizingEstimator : 737/1296 [00:01<00:00,  674.55it/s]
 59%|█████▊    | Scoring GeneralizingEstimator : 760/1296 [00:01<00:00,  673.88it/s]
 60%|██████    | Scoring GeneralizingEstimator : 783/1296 [00:01<00:00,  674.43it/s]
 62%|██████▏   | Scoring GeneralizingEstimator : 806/1296 [00:01<00:00,  673.93it/s]
 64%|██████▍   | Scoring GeneralizingEstimator : 828/1296 [00:01<00:00,  672.80it/s]
 66%|██████▌   | Scoring GeneralizingEstimator : 851/1296 [00:01<00:00,  673.36it/s]
 67%|██████▋   | Scoring GeneralizingEstimator : 874/1296 [00:01<00:00,  673.24it/s]
 69%|██████▉   | Scoring GeneralizingEstimator : 897/1296 [00:01<00:00,  673.27it/s]
 71%|███████   | Scoring GeneralizingEstimator : 920/1296 [00:01<00:00,  673.58it/s]
 73%|███████▎  | Scoring GeneralizingEstimator : 943/1296 [00:01<00:00,  673.82it/s]
 75%|███████▍  | Scoring GeneralizingEstimator : 966/1296 [00:01<00:00,  673.85it/s]
 76%|███████▋  | Scoring GeneralizingEstimator : 989/1296 [00:01<00:00,  674.02it/s]
 78%|███████▊  | Scoring GeneralizingEstimator : 1012/1296 [00:01<00:00,  674.57it/s]
 80%|███████▉  | Scoring GeneralizingEstimator : 1035/1296 [00:01<00:00,  674.76it/s]
 82%|████████▏ | Scoring GeneralizingEstimator : 1058/1296 [00:01<00:00,  675.25it/s]
 83%|████████▎ | Scoring GeneralizingEstimator : 1081/1296 [00:01<00:00,  675.69it/s]
 85%|████████▌ | Scoring GeneralizingEstimator : 1104/1296 [00:01<00:00,  675.77it/s]
 87%|████████▋ | Scoring GeneralizingEstimator : 1127/1296 [00:01<00:00,  676.08it/s]
 89%|████████▉ | Scoring GeneralizingEstimator : 1151/1296 [00:01<00:00,  676.48it/s]
 91%|█████████ | Scoring GeneralizingEstimator : 1174/1296 [00:01<00:00,  676.24it/s]
 92%|█████████▏| Scoring GeneralizingEstimator : 1197/1296 [00:01<00:00,  676.47it/s]
 94%|█████████▍| Scoring GeneralizingEstimator : 1220/1296 [00:01<00:00,  676.85it/s]
 96%|█████████▌| Scoring GeneralizingEstimator : 1243/1296 [00:01<00:00,  676.76it/s]
 98%|█████████▊| Scoring GeneralizingEstimator : 1266/1296 [00:01<00:00,  677.09it/s]
100%|█████████▉| Scoring GeneralizingEstimator : 1290/1296 [00:01<00:00,  677.20it/s]
100%|██████████| Scoring GeneralizingEstimator : 1296/1296 [00:01<00:00,  675.95it/s]

Plot the full (generalization) matrix:

fig, ax = plt.subplots(1, 1)
im = ax.imshow(
    scores,
    interpolation="lanczos",
    origin="lower",
    cmap="RdBu_r",
    extent=epochs.times[[0, -1, 0, -1]],
    vmin=0.0,
    vmax=1.0,
)
ax.set_xlabel("Testing Time (s)")
ax.set_ylabel("Training Time (s)")
ax.set_title("Temporal generalization")
ax.axvline(0, color="k")
ax.axhline(0, color="k")
cbar = plt.colorbar(im, ax=ax)
cbar.set_label("AUC")

Projecting sensor-space patterns to source space

If you use a linear classifier (or regressor) for your data, you can also project these to source space. For example, using our evoked_time_gen from before:

cov = mne.compute_covariance(epochs, tmax=0.0)
del epochs
fwd = mne.read_forward_solution(meg_path / "sample_audvis-meg-eeg-oct-6-fwd.fif")
inv = mne.minimum_norm.make_inverse_operator(evoked_time_gen.info, fwd, cov, loose=0.0)
stc = mne.minimum_norm.apply_inverse(evoked_time_gen, inv, 1.0 / 9.0, "dSPM")
del fwd, inv

Reducing data rank from 203 -> 203
Estimating covariance using EMPIRICAL
Done.
Number of samples used : 1353
[done]
Reading forward solution from /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis-meg-eeg-oct-6-fwd.fif...
    Reading a source space...
    Computing patch statistics...
    Patch information added...
    Distance information added...
    [done]
    Reading a source space...
    Computing patch statistics...
    Patch information added...
    Distance information added...
    [done]
    2 source spaces read
    Desired named matrix (kind = 3523 (FIFF_MNE_FORWARD_SOLUTION_GRAD)) not available
    Read MEG forward solution (7498 sources, 306 channels, free orientations)
    Desired named matrix (kind = 3523 (FIFF_MNE_FORWARD_SOLUTION_GRAD)) not available
    Read EEG forward solution (7498 sources, 60 channels, free orientations)
    Forward solutions combined: MEG, EEG
    Source spaces transformed to the forward solution coordinate frame
Computing inverse operator with 203 channels.
    203 out of 366 channels remain after picking
Selected 203 channels
Creating the depth weighting matrix...
    203 planar channels
    limit = 7262/7498 = 10.020866
    scale = 2.58122e-08 exp = 0.8
    Picked elements from a free-orientation depth-weighting prior into the fixed-orientation one
    Average patch normals will be employed in the rotation to the local surface coordinates....
    Converting to surface-based source orientations...
    [done]
Whitening the forward solution.
Computing rank from covariance with rank=None
    Using tolerance 1.6e-13 (2.2e-16 eps * 203 dim * 3.6  max singular value)
    Estimated rank (grad): 203
    GRAD: rank 203 computed from 203 data channels with 0 projectors
    Setting small GRAD eigenvalues to zero (without PCA)
Creating the source covariance matrix
Adjusting source covariance matrix.
Computing SVD of whitened and weighted lead field matrix.
    largest singular value = 3.91709
    scaling factor to adjust the trace = 6.26373e+18 (nchan = 203 nzero = 0)
Preparing the inverse operator for use...
    Scaled noise and source covariance from nave = 1 to nave = 1
    Created the regularized inverter
    The projection vectors do not apply to these channels.
    Created the whitener using a noise covariance matrix with rank 203 (0 small eigenvalues omitted)
    Computing noise-normalization factors (dSPM)...
[done]
Applying inverse operator to ""...
    Picked 203 channels from the data
    Computing inverse...
    Eigenleads need to be weighted ...
    Computing residual...
    Explained  76.4% variance
    dSPM...
[done]

And this can be visualized using stc.plot:

brain = stc.plot(
    hemi="split", views=("lat", "med"), initial_time=0.1, subjects_dir=subjects_dir
)

Using control points [1.98776221 2.41838256 8.06628583]
True

Source-space decoding

Source space decoding is also possible, but because the number of features can be much larger than in the sensor space, univariate feature selection using ANOVA f-test (or some other metric) can be done to reduce the feature dimension. Interpreting decoding results might be easier in source space as compared to sensor space.

Examples

• Decoding source space data

Exercise

• Explore other datasets from MNE (e.g. Face dataset from SPM to predict Face vs. Scrambled)

References

[1]

Jean-Rémi King, Laura Gwilliams, Chris Holdgraf, Jona Sassenhagen, Alexandre Barachant, Denis Engemann, Eric Larson, and Alexandre Gramfort. Encoding and decoding neuronal dynamics: methodological framework to uncover the algorithms of cognition. hal-01848442, 2018. URL: https://hal.archives-ouvertes.fr/hal-01848442.

[2]

Zoltan J. Koles. The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalography and Clinical Neurophysiology, 79(6):440–447, 1991. doi:10.1016/0013-4694(91)90163-X.

[3]

Sven Dähne, Frank C. Meinecke, Stefan Haufe, Johannes Höhne, Michael Tangermann, Klaus-Robert Müller, and Vadim V. Nikulin. SPoC: a novel framework for relating the amplitude of neuronal oscillations to behaviorally relevant parameters. NeuroImage, 86:111–122, 2014. doi:10.1016/j.neuroimage.2013.07.079.

[4]

Bertrand Rivet, Antoine Souloumiac, Virginie Attina, and Guillaume Gibert. xDAWN algorithm to enhance evoked potentials: application to brain–computer interface. IEEE Transactions on Biomedical Engineering, 56(8):2035–2043, 2009. doi:10.1109/TBME.2009.2012869.

[5]

Aaron Schurger, Sebastien Marti, and Stanislas Dehaene. Reducing multi-sensor data to a single time course that reveals experimental effects. BMC Neuroscience, 2013. doi:10.1186/1471-2202-14-122.

[6]

Jean-Rémi King, Alexandre Gramfort, Aaron Schurger, Lionel Naccache, and Stanislas Dehaene. Two distinct dynamic modes subtend the detection of unexpected sounds. PLoS ONE, 9(1):e85791, 2014. doi:10.1371/journal.pone.0085791.

[7]

Jean-Rémi King and Stanislas Dehaene. Characterizing the dynamics of mental representations: the temporal generalization method. Trends in Cognitive Sciences, 18(4):203–210, 2014. doi:10.1016/j.tics.2014.01.002.