Note
Go to the end to download the full example code
Spatiotemporal permutation F-test on full sensor data#
Tests for differential evoked responses in at least one condition using a permutation clustering test. The FieldTrip neighbor templates will be used to determine the adjacency between sensors. This serves as a spatial prior to the clustering. Spatiotemporal clusters will then be visualized using custom matplotlib code.
Here, the unit of observation is epochs from a specific study subject. However, the same logic applies when the unit observation is a number of study subject each of whom contribute their own averaged data (i.e., an average of their epochs). This would then be considered an analysis at the “2nd level”.
See the FieldTrip tutorial for a caveat regarding the possible interpretation of “significant” clusters.
For more information on cluster-based permutation testing in MNE-Python, see also: Non-parametric 1 sample cluster statistic on single trial power
# Authors: Denis Engemann <denis.engemann@gmail.com>
# Jona Sassenhagen <jona.sassenhagen@gmail.com>
# Alex Rockhill <aprockhill@mailbox.org>
# Stefan Appelhoff <stefan.appelhoff@mailbox.org>
#
# License: BSD-3-Clause
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import make_axes_locatable
import scipy.stats
import mne
from mne.stats import spatio_temporal_cluster_test, combine_adjacency
from mne.datasets import sample
from mne.channels import find_ch_adjacency
from mne.viz import plot_compare_evokeds
from mne.time_frequency import tfr_morlet
Set parameters#
data_path = sample.data_path()
meg_path = data_path / 'MEG' / 'sample'
raw_fname = meg_path / 'sample_audvis_filt-0-40_raw.fif'
event_fname = meg_path / 'sample_audvis_filt-0-40_raw-eve.fif'
event_id = {'Aud/L': 1, 'Aud/R': 2, 'Vis/L': 3, 'Vis/R': 4}
tmin = -0.2
tmax = 0.5
# Setup for reading the raw data
raw = mne.io.read_raw_fif(raw_fname, preload=True)
raw.filter(1, 30)
events = mne.read_events(event_fname)
Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_filt-0-40_raw.fif...
Read a total of 4 projection items:
PCA-v1 (1 x 102) idle
PCA-v2 (1 x 102) idle
PCA-v3 (1 x 102) idle
Average EEG reference (1 x 60) idle
Range : 6450 ... 48149 = 42.956 ... 320.665 secs
Ready.
Reading 0 ... 41699 = 0.000 ... 277.709 secs...
Filtering raw data in 1 contiguous segment
Setting up band-pass filter from 1 - 30 Hz
FIR filter parameters
---------------------
Designing a one-pass, zero-phase, non-causal bandpass filter:
- Windowed time-domain design (firwin) method
- Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation
- Lower passband edge: 1.00
- Lower transition bandwidth: 1.00 Hz (-6 dB cutoff frequency: 0.50 Hz)
- Upper passband edge: 30.00 Hz
- Upper transition bandwidth: 7.50 Hz (-6 dB cutoff frequency: 33.75 Hz)
- Filter length: 497 samples (3.310 sec)
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 1 out of 1 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 2 out of 2 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 3 out of 3 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 4 out of 4 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 366 out of 366 | elapsed: 0.7s finished
Read epochs for the channel of interest#
picks = mne.pick_types(raw.info, meg='mag', eog=True)
reject = dict(mag=4e-12, eog=150e-6)
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, picks=picks,
baseline=None, reject=reject, preload=True)
epochs.drop_channels(['EOG 061'])
epochs.equalize_event_counts(event_id)
# Obtain the data as a 3D matrix and transpose it such that
# the dimensions are as expected for the cluster permutation test:
# n_epochs × n_times × n_channels
X = [epochs[event_name].get_data() for event_name in event_id]
X = [np.transpose(x, (0, 2, 1)) for x in X]
Not setting metadata
288 matching events found
No baseline correction applied
Created an SSP operator (subspace dimension = 3)
4 projection items activated
Using data from preloaded Raw for 288 events and 106 original time points ...
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on MAG : ['MEG 1711']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on MAG : ['MEG 1711']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
Rejecting epoch based on EOG : ['EOG 061']
49 bad epochs dropped
Removing projector <Projection | Average EEG reference, active : True, n_channels : 60>
Dropped 19 epochs: 50, 51, 84, 93, 95, 96, 129, 146, 149, 150, 154, 156, 157, 189, 194, 200, 202, 210, 211
Find the FieldTrip neighbor definition to setup sensor adjacency#
adjacency, ch_names = find_ch_adjacency(epochs.info, ch_type='mag')
print(type(adjacency)) # it's a sparse matrix!
mne.viz.plot_ch_adjacency(epochs.info, adjacency, ch_names)
Reading adjacency matrix for neuromag306mag.
<class 'scipy.sparse._csr.csr_matrix'>
Compute permutation statistic#
How does it work? We use clustering to “bind” together features which are similar. Our features are the magnetic fields measured over our sensor array at different times. This reduces the multiple comparison problem. To compute the actual test-statistic, we first sum all F-values in all clusters. We end up with one statistic for each cluster. Then we generate a distribution from the data by shuffling our conditions between our samples and recomputing our clusters and the test statistics. We test for the significance of a given cluster by computing the probability of observing a cluster of that size [1][2].
# We are running an F test, so we look at the upper tail
# see also: https://stats.stackexchange.com/a/73993
tail = 1
# We want to set a critical test statistic (here: F), to determine when
# clusters are being formed. Using Scipy's percent point function of the F
# distribution, we can conveniently select a threshold that corresponds to
# some alpha level that we arbitrarily pick.
alpha_cluster_forming = 0.001
# For an F test we need the degrees of freedom for the numerator
# (number of conditions - 1) and the denominator (number of observations
# - number of conditions):
n_conditions = len(event_id)
n_observations = len(X[0])
dfn = n_conditions - 1
dfd = n_observations - n_conditions
# Note: we calculate 1 - alpha_cluster_forming to get the critical value
# on the right tail
f_thresh = scipy.stats.f.ppf(1 - alpha_cluster_forming, dfn=dfn, dfd=dfd)
# run the cluster based permutation analysis
cluster_stats = spatio_temporal_cluster_test(X, n_permutations=1000,
threshold=f_thresh, tail=tail,
n_jobs=None, buffer_size=None,
adjacency=adjacency)
F_obs, clusters, p_values, _ = cluster_stats
stat_fun(H1): min=0.004107 max=196.094418
Running initial clustering …
Found 22 clusters
0%| | Permuting : 0/999 [00:00<?, ?it/s]
0%| | Permuting : 1/999 [00:00<00:34, 29.10it/s]
0%| | Permuting : 3/999 [00:00<00:22, 44.28it/s]
1%| | Permuting : 6/999 [00:00<00:16, 59.77it/s]
1%| | Permuting : 9/999 [00:00<00:14, 67.54it/s]
1%|1 | Permuting : 11/999 [00:00<00:15, 65.68it/s]
1%|1 | Permuting : 14/999 [00:00<00:14, 69.99it/s]
2%|1 | Permuting : 17/999 [00:00<00:13, 73.08it/s]
2%|2 | Permuting : 20/999 [00:00<00:12, 75.40it/s]
2%|2 | Permuting : 23/999 [00:00<00:12, 77.21it/s]
3%|2 | Permuting : 26/999 [00:00<00:12, 78.65it/s]
3%|2 | Permuting : 29/999 [00:00<00:12, 79.83it/s]
3%|3 | Permuting : 31/999 [00:00<00:12, 77.60it/s]
3%|3 | Permuting : 34/999 [00:00<00:12, 78.74it/s]
4%|3 | Permuting : 36/999 [00:00<00:12, 76.84it/s]
4%|3 | Permuting : 39/999 [00:00<00:12, 77.96it/s]
4%|4 | Permuting : 41/999 [00:00<00:12, 76.28it/s]
4%|4 | Permuting : 44/999 [00:00<00:12, 77.35it/s]
5%|4 | Permuting : 47/999 [00:00<00:12, 78.30it/s]
5%|4 | Permuting : 49/999 [00:00<00:12, 76.75it/s]
5%|5 | Permuting : 52/999 [00:00<00:12, 77.69it/s]
6%|5 | Permuting : 55/999 [00:00<00:12, 78.53it/s]
6%|5 | Permuting : 57/999 [00:00<00:12, 77.10it/s]
6%|6 | Permuting : 60/999 [00:00<00:12, 77.94it/s]
6%|6 | Permuting : 63/999 [00:00<00:11, 78.70it/s]
7%|6 | Permuting : 65/999 [00:00<00:12, 77.30it/s]
7%|6 | Permuting : 68/999 [00:00<00:11, 78.08it/s]
7%|7 | Permuting : 71/999 [00:00<00:11, 78.79it/s]
7%|7 | Permuting : 74/999 [00:00<00:11, 79.43it/s]
8%|7 | Permuting : 76/999 [00:00<00:11, 78.05it/s]
8%|7 | Permuting : 79/999 [00:01<00:11, 78.73it/s]
8%|8 | Permuting : 82/999 [00:01<00:11, 79.36it/s]
8%|8 | Permuting : 84/999 [00:01<00:11, 78.09it/s]
9%|8 | Permuting : 87/999 [00:01<00:11, 78.74it/s]
9%|9 | Permuting : 90/999 [00:01<00:11, 79.35it/s]
9%|9 | Permuting : 93/999 [00:01<00:11, 79.92it/s]
10%|9 | Permuting : 96/999 [00:01<00:11, 80.44it/s]
10%|9 | Permuting : 99/999 [00:01<00:11, 80.93it/s]
10%|# | Permuting : 101/999 [00:01<00:11, 79.67it/s]
10%|# | Permuting : 104/999 [00:01<00:11, 80.20it/s]
11%|# | Permuting : 107/999 [00:01<00:11, 80.68it/s]
11%|#1 | Permuting : 110/999 [00:01<00:10, 81.14it/s]
11%|#1 | Permuting : 113/999 [00:01<00:10, 81.56it/s]
12%|#1 | Permuting : 116/999 [00:01<00:10, 81.96it/s]
12%|#1 | Permuting : 118/999 [00:01<00:10, 80.69it/s]
12%|#2 | Permuting : 121/999 [00:01<00:10, 81.14it/s]
12%|#2 | Permuting : 124/999 [00:01<00:10, 81.55it/s]
13%|#2 | Permuting : 127/999 [00:01<00:10, 81.90it/s]
13%|#2 | Permuting : 129/999 [00:01<00:10, 80.66it/s]
13%|#3 | Permuting : 132/999 [00:01<00:10, 81.10it/s]
13%|#3 | Permuting : 134/999 [00:01<00:10, 79.86it/s]
14%|#3 | Permuting : 137/999 [00:01<00:10, 80.34it/s]
14%|#4 | Permuting : 140/999 [00:01<00:10, 80.79it/s]
14%|#4 | Permuting : 142/999 [00:01<00:10, 79.63it/s]
15%|#4 | Permuting : 145/999 [00:01<00:10, 80.12it/s]
15%|#4 | Permuting : 148/999 [00:01<00:10, 80.57it/s]
15%|#5 | Permuting : 150/999 [00:01<00:10, 79.42it/s]
15%|#5 | Permuting : 153/999 [00:01<00:10, 79.89it/s]
16%|#5 | Permuting : 156/999 [00:01<00:10, 80.35it/s]
16%|#5 | Permuting : 158/999 [00:01<00:10, 79.23it/s]
16%|#6 | Permuting : 161/999 [00:02<00:10, 79.71it/s]
16%|#6 | Permuting : 164/999 [00:02<00:10, 80.17it/s]
17%|#6 | Permuting : 167/999 [00:02<00:10, 80.62it/s]
17%|#6 | Permuting : 169/999 [00:02<00:10, 79.50it/s]
17%|#7 | Permuting : 172/999 [00:02<00:10, 79.98it/s]
18%|#7 | Permuting : 175/999 [00:02<00:10, 80.44it/s]
18%|#7 | Permuting : 177/999 [00:02<00:10, 79.34it/s]
18%|#8 | Permuting : 180/999 [00:02<00:10, 79.82it/s]
18%|#8 | Permuting : 182/999 [00:02<00:10, 78.76it/s]
18%|#8 | Permuting : 184/999 [00:02<00:10, 77.72it/s]
19%|#8 | Permuting : 187/999 [00:02<00:10, 78.28it/s]
19%|#8 | Permuting : 189/999 [00:02<00:10, 77.30it/s]
19%|#9 | Permuting : 192/999 [00:02<00:10, 77.88it/s]
20%|#9 | Permuting : 195/999 [00:02<00:10, 78.43it/s]
20%|#9 | Permuting : 198/999 [00:02<00:10, 78.95it/s]
20%|## | Permuting : 200/999 [00:02<00:10, 77.94it/s]
20%|## | Permuting : 203/999 [00:02<00:10, 78.49it/s]
21%|## | Permuting : 206/999 [00:02<00:10, 79.02it/s]
21%|## | Permuting : 209/999 [00:02<00:09, 79.52it/s]
21%|##1 | Permuting : 211/999 [00:02<00:10, 78.48it/s]
21%|##1 | Permuting : 214/999 [00:02<00:09, 79.01it/s]
22%|##1 | Permuting : 217/999 [00:02<00:09, 79.49it/s]
22%|##2 | Permuting : 220/999 [00:02<00:09, 79.96it/s]
22%|##2 | Permuting : 222/999 [00:02<00:09, 78.91it/s]
23%|##2 | Permuting : 225/999 [00:02<00:09, 79.41it/s]
23%|##2 | Permuting : 228/999 [00:02<00:09, 79.88it/s]
23%|##3 | Permuting : 231/999 [00:02<00:09, 80.32it/s]
23%|##3 | Permuting : 234/999 [00:02<00:09, 80.74it/s]
24%|##3 | Permuting : 236/999 [00:02<00:09, 79.64it/s]
24%|##3 | Permuting : 239/999 [00:03<00:09, 80.09it/s]
24%|##4 | Permuting : 242/999 [00:03<00:09, 80.54it/s]
24%|##4 | Permuting : 244/999 [00:03<00:09, 79.46it/s]
25%|##4 | Permuting : 247/999 [00:03<00:09, 79.93it/s]
25%|##5 | Permuting : 250/999 [00:03<00:09, 80.37it/s]
25%|##5 | Permuting : 253/999 [00:03<00:09, 80.80it/s]
26%|##5 | Permuting : 256/999 [00:03<00:09, 81.19it/s]
26%|##5 | Permuting : 258/999 [00:03<00:09, 80.07it/s]
26%|##6 | Permuting : 261/999 [00:03<00:09, 80.49it/s]
26%|##6 | Permuting : 264/999 [00:03<00:09, 80.90it/s]
27%|##6 | Permuting : 267/999 [00:03<00:09, 81.30it/s]
27%|##7 | Permuting : 270/999 [00:03<00:08, 81.67it/s]
27%|##7 | Permuting : 272/999 [00:03<00:09, 80.54it/s]
28%|##7 | Permuting : 275/999 [00:03<00:08, 80.95it/s]
28%|##7 | Permuting : 278/999 [00:03<00:08, 81.35it/s]
28%|##8 | Permuting : 281/999 [00:03<00:08, 81.72it/s]
28%|##8 | Permuting : 284/999 [00:03<00:08, 82.08it/s]
29%|##8 | Permuting : 286/999 [00:03<00:08, 80.93it/s]
29%|##8 | Permuting : 289/999 [00:03<00:08, 81.31it/s]
29%|##9 | Permuting : 292/999 [00:03<00:08, 81.68it/s]
29%|##9 | Permuting : 294/999 [00:03<00:08, 80.56it/s]
30%|##9 | Permuting : 297/999 [00:03<00:08, 80.97it/s]
30%|### | Permuting : 300/999 [00:03<00:08, 81.36it/s]
30%|### | Permuting : 302/999 [00:03<00:08, 80.23it/s]
31%|### | Permuting : 305/999 [00:03<00:08, 80.64it/s]
31%|### | Permuting : 308/999 [00:03<00:08, 81.03it/s]
31%|###1 | Permuting : 310/999 [00:03<00:08, 79.92it/s]
31%|###1 | Permuting : 313/999 [00:03<00:08, 80.36it/s]
32%|###1 | Permuting : 316/999 [00:03<00:08, 80.77it/s]
32%|###1 | Permuting : 319/999 [00:03<00:08, 81.17it/s]
32%|###2 | Permuting : 322/999 [00:04<00:08, 81.55it/s]
32%|###2 | Permuting : 324/999 [00:04<00:08, 80.42it/s]
33%|###2 | Permuting : 327/999 [00:04<00:08, 80.83it/s]
33%|###3 | Permuting : 330/999 [00:04<00:08, 81.22it/s]
33%|###3 | Permuting : 333/999 [00:04<00:08, 81.60it/s]
34%|###3 | Permuting : 336/999 [00:04<00:08, 81.96it/s]
34%|###3 | Permuting : 338/999 [00:04<00:08, 80.82it/s]
34%|###4 | Permuting : 341/999 [00:04<00:08, 81.21it/s]
34%|###4 | Permuting : 343/999 [00:04<00:08, 80.10it/s]
35%|###4 | Permuting : 346/999 [00:04<00:08, 80.53it/s]
35%|###4 | Permuting : 349/999 [00:04<00:08, 80.94it/s]
35%|###5 | Permuting : 351/999 [00:04<00:08, 79.85it/s]
35%|###5 | Permuting : 354/999 [00:04<00:08, 80.29it/s]
36%|###5 | Permuting : 357/999 [00:04<00:07, 80.71it/s]
36%|###5 | Permuting : 359/999 [00:04<00:08, 79.62it/s]
36%|###6 | Permuting : 362/999 [00:04<00:07, 80.08it/s]
36%|###6 | Permuting : 364/999 [00:04<00:08, 79.02it/s]
37%|###6 | Permuting : 367/999 [00:04<00:07, 79.51it/s]
37%|###6 | Permuting : 369/999 [00:04<00:08, 78.48it/s]
37%|###7 | Permuting : 372/999 [00:04<00:07, 78.99it/s]
37%|###7 | Permuting : 374/999 [00:04<00:08, 78.00it/s]
38%|###7 | Permuting : 377/999 [00:04<00:07, 78.54it/s]
38%|###8 | Permuting : 380/999 [00:04<00:07, 79.05it/s]
38%|###8 | Permuting : 382/999 [00:04<00:07, 78.05it/s]
39%|###8 | Permuting : 385/999 [00:04<00:07, 78.58it/s]
39%|###8 | Permuting : 387/999 [00:04<00:07, 77.59it/s]
39%|###9 | Permuting : 390/999 [00:04<00:07, 78.14it/s]
39%|###9 | Permuting : 392/999 [00:04<00:07, 77.16it/s]
40%|###9 | Permuting : 395/999 [00:04<00:07, 77.73it/s]
40%|###9 | Permuting : 397/999 [00:05<00:07, 76.79it/s]
40%|###9 | Permuting : 399/999 [00:05<00:07, 75.89it/s]
40%|#### | Permuting : 402/999 [00:05<00:07, 76.52it/s]
41%|#### | Permuting : 405/999 [00:05<00:07, 77.12it/s]
41%|#### | Permuting : 407/999 [00:05<00:07, 76.21it/s]
41%|####1 | Permuting : 410/999 [00:05<00:07, 76.83it/s]
41%|####1 | Permuting : 412/999 [00:05<00:07, 75.94it/s]
42%|####1 | Permuting : 415/999 [00:05<00:07, 76.56it/s]
42%|####1 | Permuting : 418/999 [00:05<00:07, 77.16it/s]
42%|####2 | Permuting : 420/999 [00:05<00:07, 76.26it/s]
42%|####2 | Permuting : 423/999 [00:05<00:07, 76.89it/s]
43%|####2 | Permuting : 425/999 [00:05<00:07, 75.99it/s]
43%|####2 | Permuting : 428/999 [00:05<00:07, 76.63it/s]
43%|####3 | Permuting : 431/999 [00:05<00:07, 77.23it/s]
43%|####3 | Permuting : 434/999 [00:05<00:07, 77.80it/s]
44%|####3 | Permuting : 436/999 [00:05<00:07, 76.87it/s]
44%|####3 | Permuting : 439/999 [00:05<00:07, 77.45it/s]
44%|####4 | Permuting : 442/999 [00:05<00:07, 78.01it/s]
44%|####4 | Permuting : 444/999 [00:05<00:07, 77.06it/s]
45%|####4 | Permuting : 447/999 [00:05<00:07, 77.64it/s]
45%|####5 | Permuting : 450/999 [00:05<00:07, 78.19it/s]
45%|####5 | Permuting : 452/999 [00:05<00:07, 77.23it/s]
46%|####5 | Permuting : 455/999 [00:05<00:06, 77.80it/s]
46%|####5 | Permuting : 458/999 [00:05<00:06, 78.35it/s]
46%|####6 | Permuting : 461/999 [00:05<00:06, 78.80it/s]
46%|####6 | Permuting : 463/999 [00:05<00:06, 77.81it/s]
47%|####6 | Permuting : 466/999 [00:05<00:06, 78.35it/s]
47%|####6 | Permuting : 469/999 [00:05<00:06, 78.87it/s]
47%|####7 | Permuting : 471/999 [00:05<00:06, 77.89it/s]
47%|####7 | Permuting : 474/999 [00:05<00:06, 78.43it/s]
48%|####7 | Permuting : 477/999 [00:06<00:06, 78.93it/s]
48%|####7 | Permuting : 479/999 [00:06<00:06, 77.94it/s]
48%|####8 | Permuting : 482/999 [00:06<00:06, 78.47it/s]
48%|####8 | Permuting : 484/999 [00:06<00:06, 77.50it/s]
49%|####8 | Permuting : 487/999 [00:06<00:06, 78.05it/s]
49%|####9 | Permuting : 490/999 [00:06<00:06, 78.58it/s]
49%|####9 | Permuting : 493/999 [00:06<00:06, 79.09it/s]
50%|####9 | Permuting : 495/999 [00:06<00:06, 78.08it/s]
50%|####9 | Permuting : 498/999 [00:06<00:06, 78.62it/s]
50%|##### | Permuting : 501/999 [00:06<00:06, 79.13it/s]
50%|##### | Permuting : 503/999 [00:06<00:06, 78.13it/s]
51%|##### | Permuting : 506/999 [00:06<00:06, 78.66it/s]
51%|##### | Permuting : 509/999 [00:06<00:06, 79.17it/s]
51%|#####1 | Permuting : 512/999 [00:06<00:06, 79.65it/s]
51%|#####1 | Permuting : 514/999 [00:06<00:06, 78.63it/s]
52%|#####1 | Permuting : 517/999 [00:06<00:06, 79.13it/s]
52%|#####2 | Permuting : 520/999 [00:06<00:06, 79.62it/s]
52%|#####2 | Permuting : 522/999 [00:06<00:06, 78.59it/s]
53%|#####2 | Permuting : 525/999 [00:06<00:05, 79.10it/s]
53%|#####2 | Permuting : 528/999 [00:06<00:05, 79.58it/s]
53%|#####3 | Permuting : 530/999 [00:06<00:05, 78.56it/s]
53%|#####3 | Permuting : 533/999 [00:06<00:05, 79.06it/s]
54%|#####3 | Permuting : 536/999 [00:06<00:05, 79.54it/s]
54%|#####3 | Permuting : 538/999 [00:06<00:05, 78.52it/s]
54%|#####4 | Permuting : 541/999 [00:06<00:05, 79.03it/s]
54%|#####4 | Permuting : 544/999 [00:06<00:05, 79.51it/s]
55%|#####4 | Permuting : 546/999 [00:06<00:05, 78.49it/s]
55%|#####4 | Permuting : 549/999 [00:06<00:05, 79.00it/s]
55%|#####5 | Permuting : 552/999 [00:06<00:05, 79.49it/s]
55%|#####5 | Permuting : 554/999 [00:07<00:05, 78.46it/s]
56%|#####5 | Permuting : 557/999 [00:07<00:05, 78.96it/s]
56%|#####6 | Permuting : 560/999 [00:07<00:05, 79.45it/s]
56%|#####6 | Permuting : 563/999 [00:07<00:05, 79.92it/s]
57%|#####6 | Permuting : 566/999 [00:07<00:05, 80.37it/s]
57%|#####6 | Permuting : 568/999 [00:07<00:05, 79.31it/s]
57%|#####7 | Permuting : 571/999 [00:07<00:05, 79.79it/s]
57%|#####7 | Permuting : 574/999 [00:07<00:05, 80.23it/s]
58%|#####7 | Permuting : 576/999 [00:07<00:05, 79.17it/s]
58%|#####7 | Permuting : 579/999 [00:07<00:05, 79.66it/s]
58%|#####8 | Permuting : 582/999 [00:07<00:05, 80.11it/s]
58%|#####8 | Permuting : 584/999 [00:07<00:05, 79.06it/s]
59%|#####8 | Permuting : 587/999 [00:07<00:05, 79.54it/s]
59%|#####9 | Permuting : 590/999 [00:07<00:05, 80.00it/s]
59%|#####9 | Permuting : 593/999 [00:07<00:05, 80.44it/s]
60%|#####9 | Permuting : 595/999 [00:07<00:05, 79.36it/s]
60%|#####9 | Permuting : 598/999 [00:07<00:05, 79.84it/s]
60%|###### | Permuting : 601/999 [00:07<00:04, 80.27it/s]
60%|###### | Permuting : 604/999 [00:07<00:04, 80.70it/s]
61%|###### | Permuting : 607/999 [00:07<00:04, 81.09it/s]
61%|###### | Permuting : 609/999 [00:07<00:04, 79.99it/s]
61%|######1 | Permuting : 612/999 [00:07<00:04, 80.43it/s]
62%|######1 | Permuting : 615/999 [00:07<00:04, 80.84it/s]
62%|######1 | Permuting : 618/999 [00:07<00:04, 81.23it/s]
62%|######2 | Permuting : 620/999 [00:07<00:04, 80.11it/s]
62%|######2 | Permuting : 623/999 [00:07<00:04, 80.55it/s]
63%|######2 | Permuting : 625/999 [00:07<00:04, 79.48it/s]
63%|######2 | Permuting : 628/999 [00:07<00:04, 79.94it/s]
63%|######3 | Permuting : 631/999 [00:07<00:04, 80.39it/s]
63%|######3 | Permuting : 634/999 [00:07<00:04, 80.81it/s]
64%|######3 | Permuting : 636/999 [00:08<00:04, 79.72it/s]
64%|######3 | Permuting : 639/999 [00:08<00:04, 80.16it/s]
64%|######4 | Permuting : 641/999 [00:08<00:04, 79.11it/s]
64%|######4 | Permuting : 643/999 [00:08<00:04, 78.11it/s]
65%|######4 | Permuting : 646/999 [00:08<00:04, 78.65it/s]
65%|######4 | Permuting : 649/999 [00:08<00:04, 79.16it/s]
65%|######5 | Permuting : 651/999 [00:08<00:04, 78.16it/s]
65%|######5 | Permuting : 654/999 [00:08<00:04, 78.69it/s]
66%|######5 | Permuting : 656/999 [00:08<00:04, 77.72it/s]
66%|######5 | Permuting : 659/999 [00:08<00:04, 78.26it/s]
66%|######6 | Permuting : 661/999 [00:08<00:04, 77.29it/s]
66%|######6 | Permuting : 664/999 [00:08<00:04, 77.84it/s]
67%|######6 | Permuting : 666/999 [00:08<00:04, 76.91it/s]
67%|######6 | Permuting : 669/999 [00:08<00:04, 77.50it/s]
67%|######7 | Permuting : 671/999 [00:08<00:04, 76.58it/s]
67%|######7 | Permuting : 673/999 [00:08<00:04, 75.71it/s]
68%|######7 | Permuting : 676/999 [00:08<00:04, 76.36it/s]
68%|######7 | Permuting : 678/999 [00:08<00:04, 75.50it/s]
68%|######8 | Permuting : 680/999 [00:08<00:04, 74.68it/s]
68%|######8 | Permuting : 683/999 [00:08<00:04, 75.39it/s]
69%|######8 | Permuting : 685/999 [00:08<00:04, 74.58it/s]
69%|######8 | Permuting : 688/999 [00:08<00:04, 75.29it/s]
69%|######9 | Permuting : 690/999 [00:08<00:04, 74.48it/s]
69%|######9 | Permuting : 692/999 [00:08<00:04, 73.66it/s]
70%|######9 | Permuting : 695/999 [00:08<00:04, 74.40it/s]
70%|######9 | Permuting : 697/999 [00:08<00:04, 73.64it/s]
70%|######9 | Permuting : 699/999 [00:08<00:04, 72.92it/s]
70%|####### | Permuting : 702/999 [00:08<00:04, 73.70it/s]
70%|####### | Permuting : 704/999 [00:08<00:04, 72.96it/s]
71%|####### | Permuting : 707/999 [00:09<00:03, 73.75it/s]
71%|####### | Permuting : 709/999 [00:09<00:03, 73.03it/s]
71%|#######1 | Permuting : 712/999 [00:09<00:03, 73.81it/s]
71%|#######1 | Permuting : 714/999 [00:09<00:03, 73.07it/s]
72%|#######1 | Permuting : 717/999 [00:09<00:03, 73.86it/s]
72%|#######1 | Permuting : 719/999 [00:09<00:03, 73.11it/s]
72%|#######2 | Permuting : 722/999 [00:09<00:03, 73.88it/s]
72%|#######2 | Permuting : 724/999 [00:09<00:03, 73.15it/s]
73%|#######2 | Permuting : 726/999 [00:09<00:03, 72.44it/s]
73%|#######2 | Permuting : 729/999 [00:09<00:03, 73.25it/s]
73%|#######3 | Permuting : 731/999 [00:09<00:03, 72.55it/s]
73%|#######3 | Permuting : 734/999 [00:09<00:03, 73.35it/s]
74%|#######3 | Permuting : 736/999 [00:09<00:03, 72.64it/s]
74%|#######3 | Permuting : 739/999 [00:09<00:03, 73.45it/s]
74%|#######4 | Permuting : 741/999 [00:09<00:03, 72.73it/s]
74%|#######4 | Permuting : 744/999 [00:09<00:03, 73.53it/s]
75%|#######4 | Permuting : 746/999 [00:09<00:03, 72.81it/s]
75%|#######4 | Permuting : 749/999 [00:09<00:03, 73.60it/s]
75%|#######5 | Permuting : 752/999 [00:09<00:03, 74.36it/s]
75%|#######5 | Permuting : 754/999 [00:09<00:03, 73.59it/s]
76%|#######5 | Permuting : 757/999 [00:09<00:03, 74.32it/s]
76%|#######6 | Permuting : 760/999 [00:09<00:03, 75.05it/s]
76%|#######6 | Permuting : 762/999 [00:09<00:03, 74.26it/s]
76%|#######6 | Permuting : 764/999 [00:09<00:03, 73.50it/s]
77%|#######6 | Permuting : 767/999 [00:09<00:03, 74.27it/s]
77%|#######6 | Permuting : 769/999 [00:09<00:03, 73.51it/s]
77%|#######7 | Permuting : 771/999 [00:09<00:03, 72.79it/s]
77%|#######7 | Permuting : 774/999 [00:09<00:03, 73.58it/s]
78%|#######7 | Permuting : 776/999 [00:09<00:03, 72.86it/s]
78%|#######7 | Permuting : 779/999 [00:09<00:02, 73.65it/s]
78%|#######8 | Permuting : 782/999 [00:10<00:02, 74.40it/s]
79%|#######8 | Permuting : 785/999 [00:10<00:02, 75.11it/s]
79%|#######8 | Permuting : 787/999 [00:10<00:02, 74.30it/s]
79%|#######9 | Permuting : 790/999 [00:10<00:02, 75.00it/s]
79%|#######9 | Permuting : 793/999 [00:10<00:02, 75.67it/s]
80%|#######9 | Permuting : 795/999 [00:10<00:02, 74.84it/s]
80%|#######9 | Permuting : 798/999 [00:10<00:02, 75.53it/s]
80%|######## | Permuting : 800/999 [00:10<00:02, 74.68it/s]
80%|######## | Permuting : 803/999 [00:10<00:02, 75.37it/s]
81%|######## | Permuting : 806/999 [00:10<00:02, 76.03it/s]
81%|######## | Permuting : 808/999 [00:10<00:02, 75.18it/s]
81%|########1 | Permuting : 811/999 [00:10<00:02, 75.84it/s]
81%|########1 | Permuting : 814/999 [00:10<00:02, 76.48it/s]
82%|########1 | Permuting : 816/999 [00:10<00:02, 75.62it/s]
82%|########1 | Permuting : 819/999 [00:10<00:02, 76.28it/s]
82%|########2 | Permuting : 821/999 [00:10<00:02, 75.43it/s]
82%|########2 | Permuting : 824/999 [00:10<00:02, 76.10it/s]
83%|########2 | Permuting : 827/999 [00:10<00:02, 76.73it/s]
83%|########2 | Permuting : 829/999 [00:10<00:02, 75.85it/s]
83%|########3 | Permuting : 832/999 [00:10<00:02, 76.49it/s]
84%|########3 | Permuting : 835/999 [00:10<00:02, 77.11it/s]
84%|########3 | Permuting : 837/999 [00:10<00:02, 76.20it/s]
84%|########4 | Permuting : 840/999 [00:10<00:02, 76.81it/s]
84%|########4 | Permuting : 842/999 [00:10<00:02, 75.93it/s]
85%|########4 | Permuting : 845/999 [00:10<00:02, 76.56it/s]
85%|########4 | Permuting : 848/999 [00:10<00:01, 77.17it/s]
85%|########5 | Permuting : 851/999 [00:10<00:01, 77.75it/s]
85%|########5 | Permuting : 853/999 [00:10<00:01, 76.83it/s]
86%|########5 | Permuting : 856/999 [00:10<00:01, 77.43it/s]
86%|########5 | Permuting : 858/999 [00:11<00:01, 76.52it/s]
86%|########6 | Permuting : 861/999 [00:11<00:01, 77.13it/s]
86%|########6 | Permuting : 864/999 [00:11<00:01, 77.72it/s]
87%|########6 | Permuting : 866/999 [00:11<00:01, 76.79it/s]
87%|########6 | Permuting : 869/999 [00:11<00:01, 77.39it/s]
87%|########7 | Permuting : 871/999 [00:11<00:01, 76.47it/s]
87%|########7 | Permuting : 874/999 [00:11<00:01, 77.09it/s]
88%|########7 | Permuting : 877/999 [00:11<00:01, 77.67it/s]
88%|########7 | Permuting : 879/999 [00:11<00:01, 76.74it/s]
88%|########8 | Permuting : 882/999 [00:11<00:01, 77.34it/s]
89%|########8 | Permuting : 885/999 [00:11<00:01, 77.90it/s]
89%|########8 | Permuting : 887/999 [00:11<00:01, 76.97it/s]
89%|########9 | Permuting : 890/999 [00:11<00:01, 77.56it/s]
89%|########9 | Permuting : 892/999 [00:11<00:01, 76.64it/s]
90%|########9 | Permuting : 895/999 [00:11<00:01, 77.24it/s]
90%|########9 | Permuting : 898/999 [00:11<00:01, 77.82it/s]
90%|######### | Permuting : 900/999 [00:11<00:01, 76.86it/s]
90%|######### | Permuting : 903/999 [00:11<00:01, 77.45it/s]
91%|######### | Permuting : 906/999 [00:11<00:01, 78.02it/s]
91%|######### | Permuting : 909/999 [00:11<00:01, 78.56it/s]
91%|#########1| Permuting : 911/999 [00:11<00:01, 77.58it/s]
91%|#########1| Permuting : 914/999 [00:11<00:01, 78.13it/s]
92%|#########1| Permuting : 916/999 [00:11<00:01, 77.17it/s]
92%|#########1| Permuting : 919/999 [00:11<00:01, 77.75it/s]
92%|#########2| Permuting : 921/999 [00:11<00:01, 76.80it/s]
92%|#########2| Permuting : 924/999 [00:11<00:00, 77.40it/s]
93%|#########2| Permuting : 926/999 [00:11<00:00, 76.47it/s]
93%|#########2| Permuting : 929/999 [00:11<00:00, 77.08it/s]
93%|#########3| Permuting : 932/999 [00:11<00:00, 77.67it/s]
93%|#########3| Permuting : 934/999 [00:11<00:00, 76.74it/s]
94%|#########3| Permuting : 936/999 [00:12<00:00, 75.86it/s]
94%|#########3| Permuting : 939/999 [00:12<00:00, 76.50it/s]
94%|#########4| Permuting : 941/999 [00:12<00:00, 75.62it/s]
94%|#########4| Permuting : 944/999 [00:12<00:00, 76.26it/s]
95%|#########4| Permuting : 946/999 [00:12<00:00, 75.39it/s]
95%|#########4| Permuting : 949/999 [00:12<00:00, 76.05it/s]
95%|#########5| Permuting : 952/999 [00:12<00:00, 76.67it/s]
95%|#########5| Permuting : 954/999 [00:12<00:00, 75.74it/s]
96%|#########5| Permuting : 957/999 [00:12<00:00, 76.38it/s]
96%|#########5| Permuting : 959/999 [00:12<00:00, 75.44it/s]
96%|#########6| Permuting : 962/999 [00:12<00:00, 76.11it/s]
96%|#########6| Permuting : 964/999 [00:12<00:00, 75.27it/s]
97%|#########6| Permuting : 966/999 [00:12<00:00, 74.45it/s]
97%|#########6| Permuting : 969/999 [00:12<00:00, 75.17it/s]
97%|#########7| Permuting : 971/999 [00:12<00:00, 74.37it/s]
97%|#########7| Permuting : 973/999 [00:12<00:00, 73.62it/s]
98%|#########7| Permuting : 976/999 [00:12<00:00, 74.38it/s]
98%|#########7| Permuting : 979/999 [00:12<00:00, 75.10it/s]
98%|#########8| Permuting : 981/999 [00:12<00:00, 74.30it/s]
98%|#########8| Permuting : 984/999 [00:12<00:00, 75.03it/s]
99%|#########8| Permuting : 986/999 [00:12<00:00, 74.24it/s]
99%|#########8| Permuting : 989/999 [00:12<00:00, 74.95it/s]
99%|#########9| Permuting : 991/999 [00:12<00:00, 74.16it/s]
99%|#########9| Permuting : 994/999 [00:12<00:00, 74.88it/s]
100%|#########9| Permuting : 996/999 [00:12<00:00, 74.10it/s]
100%|#########9| Permuting : 998/999 [00:12<00:00, 73.34it/s]
100%|##########| Permuting : 999/999 [00:12<00:00, 77.66it/s]
Note
Note how we only specified an adjacency for sensors! However,
because we used mne.stats.spatio_temporal_cluster_test(),
an adjacency for time points was automatically taken into
account. That is, at time point N, the time points N - 1 and
N + 1 were considered as adjacent (this is also called “lattice
adjacency”). This is only possible because we ran the analysis on
2D data (times × channels) per observation … for 3D data per
observation (e.g., times × frequencies × channels), we will need
to use mne.stats.combine_adjacency(), as shown further
below.
Note also that the same functions work with source estimates. The only differences are the origin of the data, the size, and the adjacency definition. It can be used for single trials or for groups of subjects.
Visualize clusters#
# We subselect clusters that we consider significant at an arbitrarily
# picked alpha level: "p_accept".
# NOTE: remember the caveats with respect to "significant" clusters that
# we mentioned in the introduction of this tutorial!
p_accept = 0.01
good_cluster_inds = np.where(p_values < p_accept)[0]
# configure variables for visualization
colors = {"Aud": "crimson", "Vis": 'steelblue'}
linestyles = {"L": '-', "R": '--'}
# organize data for plotting
evokeds = {cond: epochs[cond].average() for cond in event_id}
# loop over clusters
for i_clu, clu_idx in enumerate(good_cluster_inds):
# unpack cluster information, get unique indices
time_inds, space_inds = np.squeeze(clusters[clu_idx])
ch_inds = np.unique(space_inds)
time_inds = np.unique(time_inds)
# get topography for F stat
f_map = F_obs[time_inds, ...].mean(axis=0)
# get signals at the sensors contributing to the cluster
sig_times = epochs.times[time_inds]
# create spatial mask
mask = np.zeros((f_map.shape[0], 1), dtype=bool)
mask[ch_inds, :] = True
# initialize figure
fig, ax_topo = plt.subplots(1, 1, figsize=(10, 3))
# plot average test statistic and mark significant sensors
f_evoked = mne.EvokedArray(f_map[:, np.newaxis], epochs.info, tmin=0)
f_evoked.plot_topomap(times=0, mask=mask, axes=ax_topo, cmap='Reds',
vlim=(np.min, np.max), show=False,
colorbar=False, mask_params=dict(markersize=10))
image = ax_topo.images[0]
# remove the title that would otherwise say "0.000 s"
ax_topo.set_title("")
# create additional axes (for ERF and colorbar)
divider = make_axes_locatable(ax_topo)
# add axes for colorbar
ax_colorbar = divider.append_axes('right', size='5%', pad=0.05)
plt.colorbar(image, cax=ax_colorbar)
ax_topo.set_xlabel(
'Averaged F-map ({:0.3f} - {:0.3f} s)'.format(*sig_times[[0, -1]]))
# add new axis for time courses and plot time courses
ax_signals = divider.append_axes('right', size='300%', pad=1.2)
title = 'Cluster #{0}, {1} sensor'.format(i_clu + 1, len(ch_inds))
if len(ch_inds) > 1:
title += "s (mean)"
plot_compare_evokeds(evokeds, title=title, picks=ch_inds, axes=ax_signals,
colors=colors, linestyles=linestyles, show=False,
split_legend=True, truncate_yaxis='auto')
# plot temporal cluster extent
ymin, ymax = ax_signals.get_ylim()
ax_signals.fill_betweenx((ymin, ymax), sig_times[0], sig_times[-1],
color='orange', alpha=0.3)
# clean up viz
mne.viz.tight_layout(fig=fig)
fig.subplots_adjust(bottom=.05)
plt.show()
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
combining channels using "gfp"
Permutation statistic for time-frequencies#
Let’s do the same thing with the time-frequency decomposition of the data (see Frequency and time-frequency sensor analysis for a tutorial and Time-frequency on simulated data (Multitaper vs. Morlet vs. Stockwell vs. Hilbert) for a comparison of time-frequency methods) to show how cluster permutations can be done on higher-dimensional data.
decim = 4
freqs = np.arange(7, 30, 3) # define frequencies of interest
n_cycles = freqs / freqs[0]
epochs_power = list()
for condition in [epochs[k] for k in ('Aud/L', 'Vis/L')]:
this_tfr = tfr_morlet(condition, freqs, n_cycles=n_cycles,
decim=decim, average=False, return_itc=False)
this_tfr.apply_baseline(mode='ratio', baseline=(None, 0))
epochs_power.append(this_tfr.data)
# transpose again to (epochs, frequencies, times, channels)
X = [np.transpose(x, (0, 2, 3, 1)) for x in epochs_power]
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 1 out of 1 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 2 out of 2 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 3 out of 3 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 4 out of 4 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 102 out of 102 | elapsed: 0.8s finished
Not setting metadata
Applying baseline correction (mode: ratio)
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 1 out of 1 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 2 out of 2 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 3 out of 3 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 4 out of 4 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 102 out of 102 | elapsed: 0.7s finished
Not setting metadata
Applying baseline correction (mode: ratio)
Remember the note on the adjacency matrix from above: For 3D data, as here,
we must use mne.stats.combine_adjacency() to extend the
sensor-based adjacency to incorporate the time-frequency plane as well.
Here, the integer inputs are converted into a lattice and combined with the sensor adjacency matrix so that data at similar times and with similar frequencies and at close sensor locations are clustered together.
# our data at each observation is of shape frequencies × times × channels
tfr_adjacency = combine_adjacency(
len(freqs), len(this_tfr.times), adjacency)
Now we can run the cluster permutation test, but first we have to set a threshold. This example decimates in time and uses few frequencies so we need to increase the threshold from the default value in order to have differentiated clusters (i.e., so that our algorithm doesn’t just find one large cluster). For a more principled method of setting this parameter, threshold-free cluster enhancement may be used. See Statistical inference for a discussion.
# This time we don't calculate a threshold based on the F distribution.
# We might as well select an arbitrary threshold for cluster forming
tfr_threshold = 15.0
# run cluster based permutation analysis
cluster_stats = spatio_temporal_cluster_test(
X, n_permutations=1000, threshold=tfr_threshold, tail=1, n_jobs=None,
buffer_size=None, adjacency=tfr_adjacency)
stat_fun(H1): min=0.000000 max=45.333033
Running initial clustering …
Found 5 clusters
0%| | Permuting : 0/999 [00:00<?, ?it/s]
0%| | Permuting : 1/999 [00:00<01:46, 9.37it/s]
0%| | Permuting : 2/999 [00:00<01:29, 11.20it/s]
0%| | Permuting : 4/999 [00:00<04:04, 4.07it/s]
1%| | Permuting : 5/999 [00:01<03:28, 4.77it/s]
1%| | Permuting : 6/999 [00:01<03:04, 5.39it/s]
1%| | Permuting : 8/999 [00:01<02:22, 6.96it/s]
1%| | Permuting : 9/999 [00:01<02:09, 7.67it/s]
1%|1 | Permuting : 10/999 [00:01<02:02, 8.06it/s]
1%|1 | Permuting : 11/999 [00:01<01:53, 8.73it/s]
1%|1 | Permuting : 12/999 [00:01<01:44, 9.40it/s]
1%|1 | Permuting : 13/999 [00:01<01:37, 10.08it/s]
1%|1 | Permuting : 14/999 [00:01<01:35, 10.35it/s]
2%|1 | Permuting : 15/999 [00:01<01:29, 10.97it/s]
2%|1 | Permuting : 16/999 [00:01<01:28, 11.14it/s]
2%|1 | Permuting : 18/999 [00:01<01:19, 12.27it/s]
2%|2 | Permuting : 20/999 [00:01<01:13, 13.30it/s]
2%|2 | Permuting : 22/999 [00:01<01:08, 14.24it/s]
2%|2 | Permuting : 24/999 [00:01<01:02, 15.66it/s]
3%|2 | Permuting : 25/999 [00:02<01:28, 11.04it/s]
3%|2 | Permuting : 26/999 [00:02<01:24, 11.46it/s]
3%|2 | Permuting : 27/999 [00:02<01:22, 11.84it/s]
3%|2 | Permuting : 29/999 [00:02<01:16, 12.62it/s]
3%|3 | Permuting : 31/999 [00:02<01:10, 13.73it/s]
3%|3 | Permuting : 33/999 [00:02<01:06, 14.44it/s]
3%|3 | Permuting : 34/999 [00:02<01:07, 14.40it/s]
4%|3 | Permuting : 35/999 [00:02<01:07, 14.33it/s]
4%|3 | Permuting : 37/999 [00:02<01:02, 15.42it/s]
4%|3 | Permuting : 38/999 [00:02<01:01, 15.72it/s]
4%|4 | Permuting : 41/999 [00:02<00:56, 17.08it/s]
4%|4 | Permuting : 42/999 [00:02<00:56, 16.91it/s]
4%|4 | Permuting : 43/999 [00:03<00:55, 17.19it/s]
4%|4 | Permuting : 44/999 [00:03<00:56, 17.01it/s]
5%|4 | Permuting : 45/999 [00:03<00:56, 16.84it/s]
5%|4 | Permuting : 47/999 [00:03<00:54, 17.41it/s]
5%|4 | Permuting : 49/999 [00:03<00:52, 17.95it/s]
5%|5 | Permuting : 50/999 [00:03<01:10, 13.55it/s]
5%|5 | Permuting : 52/999 [00:03<01:05, 14.44it/s]
5%|5 | Permuting : 54/999 [00:03<01:01, 15.34it/s]
6%|5 | Permuting : 56/999 [00:03<00:58, 16.25it/s]
6%|5 | Permuting : 57/999 [00:03<00:58, 16.14it/s]
6%|5 | Permuting : 58/999 [00:04<00:58, 16.01it/s]
6%|5 | Permuting : 59/999 [00:04<00:59, 15.91it/s]
6%|6 | Permuting : 60/999 [00:04<00:59, 15.81it/s]
6%|6 | Permuting : 61/999 [00:04<00:59, 15.72it/s]
6%|6 | Permuting : 62/999 [00:04<00:58, 15.97it/s]
6%|6 | Permuting : 63/999 [00:04<00:59, 15.81it/s]
6%|6 | Permuting : 64/999 [00:04<00:58, 16.12it/s]
7%|6 | Permuting : 65/999 [00:04<00:58, 16.00it/s]
7%|6 | Permuting : 67/999 [00:04<00:54, 17.00it/s]
7%|6 | Permuting : 68/999 [00:04<00:55, 16.84it/s]
7%|6 | Permuting : 69/999 [00:04<00:54, 17.02it/s]
7%|7 | Permuting : 72/999 [00:04<00:51, 18.17it/s]
7%|7 | Permuting : 74/999 [00:04<00:48, 19.14it/s]
8%|7 | Permuting : 76/999 [00:04<00:47, 19.57it/s]
8%|7 | Permuting : 78/999 [00:05<01:00, 15.23it/s]
8%|7 | Permuting : 79/999 [00:05<01:00, 15.13it/s]
8%|8 | Permuting : 80/999 [00:05<01:01, 15.03it/s]
8%|8 | Permuting : 81/999 [00:05<01:00, 15.28it/s]
8%|8 | Permuting : 83/999 [00:05<00:56, 16.15it/s]
9%|8 | Permuting : 85/999 [00:05<00:53, 17.02it/s]
9%|8 | Permuting : 86/999 [00:05<00:54, 16.88it/s]
9%|8 | Permuting : 87/999 [00:05<00:54, 16.74it/s]
9%|9 | Permuting : 90/999 [00:05<00:49, 18.27it/s]
9%|9 | Permuting : 91/999 [00:05<00:50, 18.07it/s]
9%|9 | Permuting : 93/999 [00:05<00:47, 18.97it/s]
9%|9 | Permuting : 94/999 [00:05<00:47, 19.14it/s]
10%|9 | Permuting : 95/999 [00:06<00:48, 18.82it/s]
10%|9 | Permuting : 96/999 [00:06<00:47, 19.07it/s]
10%|9 | Permuting : 98/999 [00:06<00:46, 19.48it/s]
10%|9 | Permuting : 99/999 [00:06<00:46, 19.20it/s]
10%|# | Permuting : 100/999 [00:06<00:47, 18.93it/s]
10%|# | Permuting : 102/999 [00:06<01:00, 14.94it/s]
10%|# | Permuting : 104/999 [00:06<00:58, 15.43it/s]
11%|# | Permuting : 105/999 [00:06<00:58, 15.37it/s]
11%|# | Permuting : 107/999 [00:06<00:55, 16.19it/s]
11%|# | Permuting : 109/999 [00:07<00:53, 16.66it/s]
11%|#1 | Permuting : 110/999 [00:07<00:53, 16.54it/s]
11%|#1 | Permuting : 111/999 [00:07<00:54, 16.42it/s]
11%|#1 | Permuting : 113/999 [00:07<00:51, 17.28it/s]
11%|#1 | Permuting : 114/999 [00:07<00:51, 17.13it/s]
12%|#1 | Permuting : 116/999 [00:07<00:49, 18.00it/s]
12%|#1 | Permuting : 117/999 [00:07<00:48, 18.18it/s]
12%|#2 | Permuting : 120/999 [00:07<00:46, 19.04it/s]
12%|#2 | Permuting : 122/999 [00:07<00:45, 19.43it/s]
12%|#2 | Permuting : 123/999 [00:07<00:44, 19.58it/s]
12%|#2 | Permuting : 124/999 [00:07<00:45, 19.25it/s]
13%|#2 | Permuting : 125/999 [00:07<00:44, 19.48it/s]
13%|#2 | Permuting : 126/999 [00:08<00:58, 14.95it/s]
13%|#2 | Permuting : 127/999 [00:08<00:58, 14.91it/s]
13%|#2 | Permuting : 128/999 [00:08<00:58, 14.87it/s]
13%|#3 | Permuting : 130/999 [00:08<00:55, 15.69it/s]
13%|#3 | Permuting : 131/999 [00:08<00:55, 15.61it/s]
13%|#3 | Permuting : 132/999 [00:08<00:55, 15.52it/s]
13%|#3 | Permuting : 134/999 [00:08<00:52, 16.37it/s]
14%|#3 | Permuting : 135/999 [00:08<00:53, 16.26it/s]
14%|#3 | Permuting : 136/999 [00:08<00:52, 16.44it/s]
14%|#3 | Permuting : 138/999 [00:08<00:50, 16.93it/s]
14%|#3 | Permuting : 139/999 [00:08<00:50, 17.12it/s]
14%|#4 | Permuting : 140/999 [00:08<00:50, 16.92it/s]
14%|#4 | Permuting : 142/999 [00:08<00:49, 17.40it/s]
14%|#4 | Permuting : 143/999 [00:08<00:48, 17.60it/s]
15%|#4 | Permuting : 145/999 [00:09<00:47, 18.07it/s]
15%|#4 | Permuting : 146/999 [00:09<00:47, 17.86it/s]
15%|#4 | Permuting : 147/999 [00:09<00:47, 18.06it/s]
15%|#4 | Permuting : 149/999 [00:09<00:45, 18.52it/s]
15%|#5 | Permuting : 151/999 [00:09<00:44, 18.95it/s]
15%|#5 | Permuting : 152/999 [00:09<00:57, 14.71it/s]
15%|#5 | Permuting : 154/999 [00:09<00:55, 15.20it/s]
16%|#5 | Permuting : 156/999 [00:09<00:52, 16.01it/s]
16%|#5 | Permuting : 157/999 [00:09<00:51, 16.22it/s]
16%|#5 | Permuting : 158/999 [00:09<00:51, 16.48it/s]
16%|#5 | Permuting : 159/999 [00:09<00:50, 16.70it/s]
16%|#6 | Permuting : 160/999 [00:10<00:50, 16.58it/s]
16%|#6 | Permuting : 161/999 [00:10<00:50, 16.45it/s]
16%|#6 | Permuting : 162/999 [00:10<00:51, 16.34it/s]
16%|#6 | Permuting : 163/999 [00:10<00:51, 16.19it/s]
17%|#6 | Permuting : 165/999 [00:10<00:51, 16.20it/s]
17%|#6 | Permuting : 166/999 [00:10<00:51, 16.08it/s]
17%|#6 | Permuting : 167/999 [00:10<00:52, 15.93it/s]
17%|#6 | Permuting : 168/999 [00:10<00:52, 15.79it/s]
17%|#7 | Permuting : 170/999 [00:10<00:50, 16.29it/s]
17%|#7 | Permuting : 171/999 [00:10<00:49, 16.57it/s]
17%|#7 | Permuting : 173/999 [00:10<00:47, 17.49it/s]
17%|#7 | Permuting : 174/999 [00:11<00:59, 13.75it/s]
18%|#7 | Permuting : 175/999 [00:11<00:59, 13.75it/s]
18%|#7 | Permuting : 176/999 [00:11<00:59, 13.75it/s]
18%|#7 | Permuting : 178/999 [00:11<00:57, 14.28it/s]
18%|#7 | Permuting : 179/999 [00:11<00:57, 14.26it/s]
18%|#8 | Permuting : 181/999 [00:11<00:54, 15.11it/s]
18%|#8 | Permuting : 184/999 [00:11<00:49, 16.53it/s]
19%|#8 | Permuting : 186/999 [00:11<00:48, 16.62it/s]
19%|#8 | Permuting : 188/999 [00:11<00:46, 17.46it/s]
19%|#8 | Permuting : 189/999 [00:11<00:46, 17.30it/s]
19%|#9 | Permuting : 190/999 [00:11<00:47, 17.15it/s]
19%|#9 | Permuting : 191/999 [00:11<00:46, 17.39it/s]
19%|#9 | Permuting : 192/999 [00:11<00:46, 17.23it/s]
19%|#9 | Permuting : 194/999 [00:12<00:44, 18.07it/s]
20%|#9 | Permuting : 195/999 [00:12<00:44, 17.87it/s]
20%|#9 | Permuting : 196/999 [00:12<00:44, 18.07it/s]
20%|#9 | Permuting : 198/999 [00:12<00:43, 18.52it/s]
20%|## | Permuting : 200/999 [00:12<00:41, 19.44it/s]
20%|## | Permuting : 201/999 [00:12<00:41, 19.10it/s]
20%|## | Permuting : 203/999 [00:12<00:40, 19.51it/s]
20%|## | Permuting : 204/999 [00:12<00:52, 15.20it/s]
21%|## | Permuting : 206/999 [00:12<00:50, 15.69it/s]
21%|## | Permuting : 207/999 [00:12<00:49, 15.90it/s]
21%|## | Permuting : 208/999 [00:13<00:49, 16.13it/s]
21%|## | Permuting : 209/999 [00:13<00:48, 16.32it/s]
21%|##1 | Permuting : 210/999 [00:13<00:48, 16.14it/s]
21%|##1 | Permuting : 211/999 [00:13<00:49, 15.99it/s]
21%|##1 | Permuting : 214/999 [00:13<00:45, 17.07it/s]
22%|##1 | Permuting : 217/999 [00:13<00:44, 17.70it/s]
22%|##1 | Permuting : 219/999 [00:13<00:42, 18.55it/s]
22%|##2 | Permuting : 221/999 [00:13<00:40, 19.33it/s]
22%|##2 | Permuting : 222/999 [00:13<00:39, 19.48it/s]
22%|##2 | Permuting : 223/999 [00:13<00:39, 19.58it/s]
23%|##2 | Permuting : 225/999 [00:13<00:38, 19.96it/s]
23%|##2 | Permuting : 226/999 [00:13<00:40, 19.14it/s]
23%|##2 | Permuting : 227/999 [00:13<00:40, 18.83it/s]
23%|##2 | Permuting : 228/999 [00:13<00:40, 19.04it/s]
23%|##2 | Permuting : 229/999 [00:13<00:41, 18.77it/s]
23%|##3 | Permuting : 231/999 [00:13<00:39, 19.69it/s]
23%|##3 | Permuting : 232/999 [00:14<00:50, 15.32it/s]
23%|##3 | Permuting : 234/999 [00:14<00:48, 15.80it/s]
24%|##3 | Permuting : 236/999 [00:14<00:45, 16.61it/s]
24%|##3 | Permuting : 238/999 [00:14<00:43, 17.44it/s]
24%|##4 | Permuting : 241/999 [00:14<00:41, 18.38it/s]
24%|##4 | Permuting : 242/999 [00:14<00:41, 18.19it/s]
24%|##4 | Permuting : 243/999 [00:14<00:41, 18.40it/s]
25%|##4 | Permuting : 245/999 [00:14<00:40, 18.83it/s]
25%|##4 | Permuting : 247/999 [00:14<00:38, 19.68it/s]
25%|##4 | Permuting : 249/999 [00:14<00:36, 20.53it/s]
25%|##5 | Permuting : 250/999 [00:14<00:37, 20.15it/s]
25%|##5 | Permuting : 253/999 [00:15<00:35, 21.12it/s]
26%|##5 | Permuting : 255/999 [00:15<00:33, 21.97it/s]
26%|##5 | Permuting : 256/999 [00:15<00:33, 22.06it/s]
26%|##5 | Permuting : 257/999 [00:15<00:34, 21.56it/s]
26%|##5 | Permuting : 258/999 [00:15<00:35, 21.16it/s]
26%|##5 | Permuting : 259/999 [00:15<00:34, 21.28it/s]
26%|##6 | Permuting : 261/999 [00:15<00:34, 21.60it/s]
26%|##6 | Permuting : 263/999 [00:15<00:34, 21.37it/s]
27%|##6 | Permuting : 265/999 [00:15<00:32, 22.25it/s]
27%|##6 | Permuting : 266/999 [00:15<00:33, 21.83it/s]
27%|##6 | Permuting : 268/999 [00:15<00:33, 22.12it/s]
27%|##7 | Permuting : 270/999 [00:15<00:31, 23.00it/s]
27%|##7 | Permuting : 271/999 [00:16<00:43, 16.86it/s]
27%|##7 | Permuting : 273/999 [00:16<00:42, 17.28it/s]
27%|##7 | Permuting : 274/999 [00:16<00:41, 17.46it/s]
28%|##7 | Permuting : 275/999 [00:16<00:41, 17.66it/s]
28%|##7 | Permuting : 276/999 [00:16<00:40, 17.79it/s]
28%|##7 | Permuting : 277/999 [00:16<00:41, 17.56it/s]
28%|##7 | Permuting : 278/999 [00:16<00:41, 17.35it/s]
28%|##7 | Permuting : 279/999 [00:16<00:40, 17.60it/s]
28%|##8 | Permuting : 280/999 [00:16<00:40, 17.79it/s]
28%|##8 | Permuting : 281/999 [00:16<00:39, 18.03it/s]
28%|##8 | Permuting : 283/999 [00:16<00:38, 18.83it/s]
29%|##8 | Permuting : 286/999 [00:16<00:34, 20.43it/s]
29%|##8 | Permuting : 287/999 [00:16<00:34, 20.57it/s]
29%|##8 | Permuting : 289/999 [00:16<00:33, 20.92it/s]
29%|##9 | Permuting : 290/999 [00:16<00:34, 20.56it/s]
29%|##9 | Permuting : 292/999 [00:17<00:33, 20.91it/s]
29%|##9 | Permuting : 294/999 [00:17<00:32, 21.83it/s]
30%|##9 | Permuting : 295/999 [00:17<00:32, 21.94it/s]
30%|##9 | Permuting : 296/999 [00:17<00:31, 22.11it/s]
30%|##9 | Permuting : 297/999 [00:17<00:32, 21.66it/s]
30%|##9 | Permuting : 298/999 [00:17<00:43, 16.23it/s]
30%|### | Permuting : 300/999 [00:17<00:40, 17.07it/s]
30%|### | Permuting : 302/999 [00:17<00:38, 17.93it/s]
30%|### | Permuting : 304/999 [00:17<00:37, 18.35it/s]
31%|### | Permuting : 306/999 [00:17<00:36, 19.21it/s]
31%|### | Permuting : 308/999 [00:17<00:35, 19.59it/s]
31%|### | Permuting : 309/999 [00:17<00:34, 19.74it/s]
31%|###1 | Permuting : 310/999 [00:18<00:34, 19.93it/s]
31%|###1 | Permuting : 311/999 [00:18<00:35, 19.63it/s]
31%|###1 | Permuting : 312/999 [00:18<00:35, 19.33it/s]
31%|###1 | Permuting : 313/999 [00:18<00:35, 19.47it/s]
31%|###1 | Permuting : 314/999 [00:18<00:34, 19.71it/s]
32%|###1 | Permuting : 315/999 [00:18<00:35, 19.41it/s]
32%|###1 | Permuting : 316/999 [00:18<00:34, 19.58it/s]
32%|###1 | Permuting : 317/999 [00:18<00:34, 19.83it/s]
32%|###1 | Permuting : 318/999 [00:18<00:36, 18.90it/s]
32%|###2 | Permuting : 320/999 [00:18<00:35, 19.35it/s]
32%|###2 | Permuting : 321/999 [00:18<00:34, 19.52it/s]
32%|###2 | Permuting : 322/999 [00:19<00:46, 14.57it/s]
32%|###2 | Permuting : 323/999 [00:19<00:46, 14.54it/s]
32%|###2 | Permuting : 324/999 [00:19<00:46, 14.50it/s]
33%|###2 | Permuting : 325/999 [00:19<00:45, 14.79it/s]
33%|###2 | Permuting : 327/999 [00:19<00:42, 15.68it/s]
33%|###2 | Permuting : 328/999 [00:19<00:43, 15.59it/s]
33%|###2 | Permuting : 329/999 [00:19<00:42, 15.83it/s]
33%|###3 | Permuting : 330/999 [00:19<00:42, 15.65it/s]
33%|###3 | Permuting : 332/999 [00:19<00:41, 16.18it/s]
33%|###3 | Permuting : 334/999 [00:19<00:38, 17.10it/s]
34%|###3 | Permuting : 335/999 [00:19<00:38, 17.33it/s]
34%|###3 | Permuting : 337/999 [00:19<00:37, 17.81it/s]
34%|###3 | Permuting : 338/999 [00:19<00:36, 18.02it/s]
34%|###3 | Permuting : 339/999 [00:19<00:37, 17.75it/s]
34%|###4 | Permuting : 340/999 [00:19<00:36, 18.02it/s]
34%|###4 | Permuting : 341/999 [00:19<00:36, 18.28it/s]
34%|###4 | Permuting : 343/999 [00:20<00:34, 18.75it/s]
35%|###4 | Permuting : 345/999 [00:20<00:33, 19.73it/s]
35%|###4 | Permuting : 347/999 [00:20<00:32, 20.14it/s]
35%|###4 | Permuting : 349/999 [00:20<00:31, 20.36it/s]
35%|###5 | Permuting : 351/999 [00:20<00:30, 21.32it/s]
35%|###5 | Permuting : 354/999 [00:20<00:28, 23.03it/s]
36%|###5 | Permuting : 355/999 [00:20<00:28, 22.30it/s]
36%|###5 | Permuting : 356/999 [00:20<00:39, 16.13it/s]
36%|###5 | Permuting : 357/999 [00:20<00:40, 16.03it/s]
36%|###5 | Permuting : 359/999 [00:21<00:37, 16.86it/s]
36%|###6 | Permuting : 360/999 [00:21<00:37, 17.03it/s]
36%|###6 | Permuting : 361/999 [00:21<00:36, 17.28it/s]
36%|###6 | Permuting : 363/999 [00:21<00:35, 18.15it/s]
37%|###6 | Permuting : 365/999 [00:21<00:34, 18.14it/s]
37%|###6 | Permuting : 366/999 [00:21<00:34, 18.33it/s]
37%|###6 | Permuting : 367/999 [00:21<00:34, 18.57it/s]
37%|###6 | Permuting : 368/999 [00:21<00:33, 18.75it/s]
37%|###7 | Permuting : 370/999 [00:21<00:32, 19.13it/s]
37%|###7 | Permuting : 371/999 [00:21<00:33, 18.80it/s]
37%|###7 | Permuting : 372/999 [00:21<00:33, 18.50it/s]
37%|###7 | Permuting : 373/999 [00:21<00:33, 18.75it/s]
38%|###7 | Permuting : 375/999 [00:21<00:31, 19.70it/s]
38%|###7 | Permuting : 376/999 [00:21<00:31, 19.87it/s]
38%|###7 | Permuting : 377/999 [00:21<00:31, 19.56it/s]
38%|###7 | Permuting : 378/999 [00:21<00:31, 19.77it/s]
38%|###8 | Permuting : 381/999 [00:21<00:30, 20.33it/s]
38%|###8 | Permuting : 383/999 [00:21<00:28, 21.27it/s]
38%|###8 | Permuting : 384/999 [00:21<00:28, 21.40it/s]
39%|###8 | Permuting : 386/999 [00:22<00:28, 21.73it/s]
39%|###8 | Permuting : 388/999 [00:22<00:35, 17.18it/s]
39%|###9 | Permuting : 390/999 [00:22<00:34, 17.62it/s]
39%|###9 | Permuting : 391/999 [00:22<00:34, 17.45it/s]
39%|###9 | Permuting : 393/999 [00:22<00:33, 17.89it/s]
39%|###9 | Permuting : 394/999 [00:22<00:33, 18.10it/s]
40%|###9 | Permuting : 395/999 [00:22<00:32, 18.34it/s]
40%|###9 | Permuting : 397/999 [00:22<00:31, 19.22it/s]
40%|###9 | Permuting : 398/999 [00:22<00:31, 18.97it/s]
40%|###9 | Permuting : 399/999 [00:22<00:32, 18.72it/s]
40%|#### | Permuting : 400/999 [00:23<00:32, 18.49it/s]
40%|#### | Permuting : 401/999 [00:23<00:32, 18.23it/s]
40%|#### | Permuting : 403/999 [00:23<00:31, 18.67it/s]
40%|#### | Permuting : 404/999 [00:23<00:32, 18.43it/s]
41%|#### | Permuting : 405/999 [00:23<00:31, 18.65it/s]
41%|#### | Permuting : 406/999 [00:23<00:32, 18.41it/s]
41%|#### | Permuting : 407/999 [00:23<00:40, 14.54it/s]
41%|#### | Permuting : 408/999 [00:23<00:40, 14.51it/s]
41%|#### | Permuting : 409/999 [00:23<00:40, 14.48it/s]
41%|####1 | Permuting : 411/999 [00:24<00:38, 15.33it/s]
41%|####1 | Permuting : 412/999 [00:24<00:37, 15.57it/s]
41%|####1 | Permuting : 414/999 [00:24<00:36, 16.04it/s]
42%|####1 | Permuting : 416/999 [00:24<00:35, 16.53it/s]
42%|####1 | Permuting : 417/999 [00:24<00:35, 16.41it/s]
42%|####1 | Permuting : 418/999 [00:24<00:34, 16.63it/s]
42%|####1 | Permuting : 419/999 [00:24<00:34, 16.89it/s]
42%|####2 | Permuting : 420/999 [00:24<00:33, 17.08it/s]
42%|####2 | Permuting : 421/999 [00:24<00:33, 17.32it/s]
42%|####2 | Permuting : 422/999 [00:24<00:33, 17.13it/s]
42%|####2 | Permuting : 423/999 [00:24<00:33, 17.37it/s]
43%|####2 | Permuting : 425/999 [00:24<00:32, 17.88it/s]
43%|####2 | Permuting : 428/999 [00:24<00:30, 18.55it/s]
43%|####3 | Permuting : 430/999 [00:24<00:29, 19.48it/s]
43%|####3 | Permuting : 431/999 [00:24<00:28, 19.65it/s]
43%|####3 | Permuting : 433/999 [00:24<00:28, 20.02it/s]
44%|####3 | Permuting : 435/999 [00:25<00:36, 15.55it/s]
44%|####3 | Permuting : 436/999 [00:25<00:36, 15.47it/s]
44%|####3 | Permuting : 438/999 [00:25<00:34, 16.29it/s]
44%|####4 | Permuting : 440/999 [00:25<00:32, 17.12it/s]
44%|####4 | Permuting : 441/999 [00:25<00:32, 17.31it/s]
44%|####4 | Permuting : 442/999 [00:25<00:32, 17.11it/s]
44%|####4 | Permuting : 443/999 [00:25<00:32, 17.32it/s]
44%|####4 | Permuting : 444/999 [00:25<00:31, 17.48it/s]
45%|####4 | Permuting : 446/999 [00:25<00:30, 17.94it/s]
45%|####4 | Permuting : 448/999 [00:25<00:29, 18.83it/s]
45%|####5 | Permuting : 450/999 [00:25<00:28, 19.24it/s]
45%|####5 | Permuting : 452/999 [00:25<00:27, 20.13it/s]
45%|####5 | Permuting : 454/999 [00:26<00:26, 20.47it/s]
46%|####5 | Permuting : 455/999 [00:26<00:26, 20.59it/s]
46%|####5 | Permuting : 456/999 [00:26<00:26, 20.76it/s]
46%|####5 | Permuting : 458/999 [00:26<00:24, 21.69it/s]
46%|####5 | Permuting : 459/999 [00:26<00:25, 21.28it/s]
46%|####6 | Permuting : 461/999 [00:26<00:24, 22.22it/s]
46%|####6 | Permuting : 463/999 [00:26<00:23, 22.50it/s]
47%|####6 | Permuting : 465/999 [00:26<00:22, 23.25it/s]
47%|####6 | Permuting : 467/999 [00:26<00:22, 23.48it/s]
47%|####6 | Permuting : 468/999 [00:26<00:22, 23.54it/s]
47%|####6 | Permuting : 469/999 [00:26<00:22, 23.70it/s]
47%|####7 | Permuting : 471/999 [00:26<00:21, 24.65it/s]
47%|####7 | Permuting : 472/999 [00:26<00:28, 18.41it/s]
47%|####7 | Permuting : 473/999 [00:27<00:28, 18.15it/s]
47%|####7 | Permuting : 474/999 [00:27<00:29, 17.94it/s]
48%|####7 | Permuting : 475/999 [00:27<00:29, 17.76it/s]
48%|####7 | Permuting : 477/999 [00:27<00:28, 18.64it/s]
48%|####7 | Permuting : 479/999 [00:27<00:27, 19.05it/s]
48%|####8 | Permuting : 480/999 [00:27<00:27, 19.21it/s]
48%|####8 | Permuting : 481/999 [00:27<00:26, 19.43it/s]
48%|####8 | Permuting : 484/999 [00:27<00:25, 19.98it/s]
49%|####8 | Permuting : 486/999 [00:27<00:24, 20.87it/s]
49%|####8 | Permuting : 487/999 [00:27<00:24, 20.52it/s]
49%|####9 | Permuting : 490/999 [00:27<00:23, 22.11it/s]
49%|####9 | Permuting : 491/999 [00:27<00:23, 21.69it/s]
49%|####9 | Permuting : 494/999 [00:27<00:21, 23.24it/s]
50%|####9 | Permuting : 495/999 [00:27<00:21, 23.31it/s]
50%|####9 | Permuting : 496/999 [00:27<00:21, 23.45it/s]
50%|####9 | Permuting : 497/999 [00:27<00:21, 22.93it/s]
50%|####9 | Permuting : 499/999 [00:27<00:20, 23.86it/s]
50%|##### | Permuting : 500/999 [00:28<00:21, 23.30it/s]
50%|##### | Permuting : 501/999 [00:28<00:21, 22.76it/s]
50%|##### | Permuting : 503/999 [00:28<00:20, 23.70it/s]
50%|##### | Permuting : 504/999 [00:28<00:28, 17.30it/s]
51%|##### | Permuting : 507/999 [00:28<00:26, 18.31it/s]
51%|##### | Permuting : 508/999 [00:28<00:27, 18.12it/s]
51%|##### | Permuting : 509/999 [00:28<00:27, 17.92it/s]
51%|#####1 | Permuting : 511/999 [00:28<00:26, 18.76it/s]
51%|#####1 | Permuting : 513/999 [00:28<00:24, 19.61it/s]
52%|#####1 | Permuting : 516/999 [00:28<00:22, 21.10it/s]
52%|#####1 | Permuting : 517/999 [00:29<00:23, 20.74it/s]
52%|#####1 | Permuting : 518/999 [00:29<00:23, 20.41it/s]
52%|#####2 | Permuting : 520/999 [00:29<00:22, 21.28it/s]
52%|#####2 | Permuting : 522/999 [00:29<00:22, 21.58it/s]
52%|#####2 | Permuting : 523/999 [00:29<00:22, 21.62it/s]
53%|#####2 | Permuting : 525/999 [00:29<00:21, 21.91it/s]
53%|#####2 | Permuting : 527/999 [00:29<00:20, 22.71it/s]
53%|#####2 | Permuting : 528/999 [00:29<00:20, 22.83it/s]
53%|#####2 | Permuting : 529/999 [00:29<00:20, 22.90it/s]
53%|#####3 | Permuting : 530/999 [00:29<00:20, 22.34it/s]
53%|#####3 | Permuting : 532/999 [00:29<00:20, 22.52it/s]
53%|#####3 | Permuting : 533/999 [00:29<00:20, 22.61it/s]
53%|#####3 | Permuting : 534/999 [00:29<00:21, 22.03it/s]
54%|#####3 | Permuting : 537/999 [00:29<00:20, 23.00it/s]
54%|#####3 | Permuting : 538/999 [00:30<00:26, 17.29it/s]
54%|#####4 | Permuting : 540/999 [00:30<00:25, 17.71it/s]
54%|#####4 | Permuting : 543/999 [00:30<00:24, 18.68it/s]
54%|#####4 | Permuting : 544/999 [00:30<00:24, 18.48it/s]
55%|#####4 | Permuting : 546/999 [00:30<00:23, 19.29it/s]
55%|#####4 | Permuting : 548/999 [00:30<00:22, 19.64it/s]
55%|#####4 | Permuting : 549/999 [00:30<00:23, 19.39it/s]
55%|#####5 | Permuting : 551/999 [00:30<00:22, 20.21it/s]
55%|#####5 | Permuting : 552/999 [00:30<00:21, 20.33it/s]
55%|#####5 | Permuting : 553/999 [00:30<00:21, 20.49it/s]
55%|#####5 | Permuting : 554/999 [00:30<00:21, 20.61it/s]
56%|#####5 | Permuting : 556/999 [00:30<00:21, 20.95it/s]
56%|#####5 | Permuting : 557/999 [00:30<00:21, 20.57it/s]
56%|#####5 | Permuting : 558/999 [00:31<00:21, 20.24it/s]
56%|#####5 | Permuting : 559/999 [00:31<00:21, 20.38it/s]
56%|#####6 | Permuting : 560/999 [00:31<00:21, 20.60it/s]
56%|#####6 | Permuting : 562/999 [00:31<00:20, 21.53it/s]
56%|#####6 | Permuting : 564/999 [00:31<00:19, 22.32it/s]
57%|#####6 | Permuting : 565/999 [00:31<00:26, 16.50it/s]
57%|#####6 | Permuting : 566/999 [00:31<00:26, 16.39it/s]
57%|#####6 | Permuting : 569/999 [00:31<00:24, 17.80it/s]
57%|#####7 | Permuting : 570/999 [00:31<00:24, 17.63it/s]
57%|#####7 | Permuting : 572/999 [00:31<00:23, 18.47it/s]
57%|#####7 | Permuting : 573/999 [00:32<00:23, 18.27it/s]
58%|#####7 | Permuting : 576/999 [00:32<00:21, 19.75it/s]
58%|#####7 | Permuting : 577/999 [00:32<00:21, 19.48it/s]
58%|#####7 | Permuting : 578/999 [00:32<00:21, 19.22it/s]
58%|#####7 | Permuting : 579/999 [00:32<00:22, 18.97it/s]
58%|#####8 | Permuting : 581/999 [00:32<00:21, 19.85it/s]
58%|#####8 | Permuting : 582/999 [00:32<00:21, 19.56it/s]
58%|#####8 | Permuting : 583/999 [00:32<00:21, 19.72it/s]
58%|#####8 | Permuting : 584/999 [00:32<00:20, 19.95it/s]
59%|#####8 | Permuting : 586/999 [00:32<00:19, 20.88it/s]
59%|#####8 | Permuting : 587/999 [00:32<00:19, 21.02it/s]
59%|#####8 | Permuting : 589/999 [00:32<00:18, 21.97it/s]
59%|#####9 | Permuting : 590/999 [00:32<00:18, 22.08it/s]
59%|#####9 | Permuting : 591/999 [00:32<00:18, 22.27it/s]
59%|#####9 | Permuting : 593/999 [00:32<00:17, 23.26it/s]
60%|#####9 | Permuting : 595/999 [00:32<00:16, 24.07it/s]
60%|#####9 | Permuting : 596/999 [00:32<00:16, 24.18it/s]
60%|#####9 | Permuting : 598/999 [00:32<00:15, 25.17it/s]
60%|#####9 | Permuting : 599/999 [00:33<00:22, 17.75it/s]
60%|###### | Permuting : 600/999 [00:33<00:22, 17.58it/s]
60%|###### | Permuting : 601/999 [00:33<00:22, 17.77it/s]
60%|###### | Permuting : 602/999 [00:33<00:22, 18.03it/s]
60%|###### | Permuting : 603/999 [00:33<00:21, 18.23it/s]
60%|###### | Permuting : 604/999 [00:33<00:21, 18.47it/s]
61%|###### | Permuting : 606/999 [00:33<00:20, 19.43it/s]
61%|###### | Permuting : 607/999 [00:33<00:20, 19.14it/s]
61%|###### | Permuting : 608/999 [00:33<00:20, 18.87it/s]
61%|######1 | Permuting : 610/999 [00:33<00:19, 19.84it/s]
61%|######1 | Permuting : 613/999 [00:33<00:17, 21.55it/s]
61%|######1 | Permuting : 614/999 [00:33<00:17, 21.67it/s]
62%|######1 | Permuting : 615/999 [00:33<00:17, 21.87it/s]
62%|######1 | Permuting : 616/999 [00:33<00:17, 21.98it/s]
62%|######1 | Permuting : 617/999 [00:33<00:17, 22.18it/s]
62%|######1 | Permuting : 619/999 [00:33<00:16, 23.20it/s]
62%|######2 | Permuting : 621/999 [00:34<00:16, 23.46it/s]
62%|######2 | Permuting : 622/999 [00:34<00:16, 22.89it/s]
62%|######2 | Permuting : 623/999 [00:34<00:16, 22.36it/s]
62%|######2 | Permuting : 624/999 [00:34<00:23, 16.12it/s]
63%|######2 | Permuting : 625/999 [00:34<00:23, 15.98it/s]
63%|######2 | Permuting : 626/999 [00:34<00:22, 16.25it/s]
63%|######2 | Permuting : 628/999 [00:34<00:22, 16.35it/s]
63%|######2 | Permuting : 629/999 [00:34<00:22, 16.17it/s]
63%|######3 | Permuting : 630/999 [00:35<00:22, 16.45it/s]
63%|######3 | Permuting : 631/999 [00:35<00:22, 16.33it/s]
63%|######3 | Permuting : 632/999 [00:35<00:22, 16.21it/s]
63%|######3 | Permuting : 634/999 [00:35<00:21, 16.73it/s]
64%|######3 | Permuting : 636/999 [00:35<00:20, 17.64it/s]
64%|######3 | Permuting : 637/999 [00:35<00:20, 17.85it/s]
64%|######3 | Permuting : 638/999 [00:35<00:19, 18.12it/s]
64%|######3 | Permuting : 639/999 [00:35<00:19, 18.32it/s]
64%|######4 | Permuting : 640/999 [00:35<00:19, 18.59it/s]
64%|######4 | Permuting : 642/999 [00:35<00:18, 19.59it/s]
65%|######4 | Permuting : 645/999 [00:35<00:16, 21.34it/s]
65%|######4 | Permuting : 646/999 [00:35<00:16, 20.93it/s]
65%|######4 | Permuting : 648/999 [00:35<00:16, 21.92it/s]
65%|######5 | Permuting : 650/999 [00:35<00:15, 22.23it/s]
65%|######5 | Permuting : 651/999 [00:36<00:20, 16.76it/s]
65%|######5 | Permuting : 653/999 [00:36<00:20, 17.23it/s]
65%|######5 | Permuting : 654/999 [00:36<00:20, 17.05it/s]
66%|######5 | Permuting : 655/999 [00:36<00:20, 16.87it/s]
66%|######5 | Permuting : 657/999 [00:36<00:19, 17.32it/s]
66%|######5 | Permuting : 659/999 [00:36<00:18, 18.19it/s]
66%|######6 | Permuting : 660/999 [00:36<00:18, 18.37it/s]
66%|######6 | Permuting : 662/999 [00:36<00:17, 18.76it/s]
66%|######6 | Permuting : 663/999 [00:36<00:17, 18.95it/s]
67%|######6 | Permuting : 665/999 [00:36<00:16, 19.86it/s]
67%|######6 | Permuting : 668/999 [00:36<00:16, 20.43it/s]
67%|######7 | Permuting : 670/999 [00:36<00:15, 21.31it/s]
67%|######7 | Permuting : 671/999 [00:36<00:15, 21.45it/s]
67%|######7 | Permuting : 673/999 [00:36<00:14, 21.76it/s]
67%|######7 | Permuting : 674/999 [00:36<00:15, 21.33it/s]
68%|######7 | Permuting : 675/999 [00:37<00:15, 21.44it/s]
68%|######7 | Permuting : 677/999 [00:37<00:14, 21.76it/s]
68%|######7 | Permuting : 679/999 [00:37<00:14, 22.05it/s]
68%|######8 | Permuting : 681/999 [00:37<00:13, 22.96it/s]
68%|######8 | Permuting : 683/999 [00:37<00:13, 23.86it/s]
69%|######8 | Permuting : 685/999 [00:37<00:12, 24.76it/s]
69%|######8 | Permuting : 687/999 [00:37<00:12, 24.91it/s]
69%|######8 | Permuting : 688/999 [00:37<00:12, 24.93it/s]
69%|######8 | Permuting : 689/999 [00:37<00:12, 25.06it/s]
69%|######9 | Permuting : 691/999 [00:37<00:16, 18.44it/s]
69%|######9 | Permuting : 693/999 [00:38<00:16, 18.83it/s]
69%|######9 | Permuting : 694/999 [00:38<00:16, 18.61it/s]
70%|######9 | Permuting : 695/999 [00:38<00:16, 18.78it/s]
70%|######9 | Permuting : 697/999 [00:38<00:15, 19.16it/s]
70%|######9 | Permuting : 698/999 [00:38<00:15, 18.92it/s]
70%|######9 | Permuting : 699/999 [00:38<00:15, 19.09it/s]
70%|####### | Permuting : 700/999 [00:38<00:15, 19.31it/s]
70%|####### | Permuting : 701/999 [00:38<00:15, 19.05it/s]
70%|####### | Permuting : 704/999 [00:38<00:14, 20.62it/s]
71%|####### | Permuting : 705/999 [00:38<00:14, 20.79it/s]
71%|####### | Permuting : 706/999 [00:38<00:13, 20.94it/s]
71%|####### | Permuting : 707/999 [00:38<00:14, 20.58it/s]
71%|####### | Permuting : 708/999 [00:38<00:14, 20.77it/s]
71%|####### | Permuting : 709/999 [00:38<00:14, 20.41it/s]
71%|#######1 | Permuting : 711/999 [00:38<00:13, 21.37it/s]
71%|#######1 | Permuting : 712/999 [00:39<00:18, 15.77it/s]
71%|#######1 | Permuting : 714/999 [00:39<00:17, 16.25it/s]
72%|#######1 | Permuting : 715/999 [00:39<00:17, 16.14it/s]
72%|#######1 | Permuting : 716/999 [00:39<00:17, 16.03it/s]
72%|#######1 | Permuting : 718/999 [00:39<00:16, 16.88it/s]
72%|#######2 | Permuting : 720/999 [00:39<00:15, 17.74it/s]
72%|#######2 | Permuting : 723/999 [00:39<00:14, 19.22it/s]
72%|#######2 | Permuting : 724/999 [00:39<00:14, 19.38it/s]
73%|#######2 | Permuting : 726/999 [00:39<00:13, 19.76it/s]
73%|#######2 | Permuting : 727/999 [00:39<00:13, 19.48it/s]
73%|#######2 | Permuting : 729/999 [00:39<00:13, 20.37it/s]
73%|#######3 | Permuting : 730/999 [00:39<00:13, 20.04it/s]
73%|#######3 | Permuting : 731/999 [00:39<00:13, 20.21it/s]
73%|#######3 | Permuting : 733/999 [00:39<00:12, 20.58it/s]
73%|#######3 | Permuting : 734/999 [00:40<00:13, 20.24it/s]
74%|#######3 | Permuting : 735/999 [00:40<00:12, 20.45it/s]
74%|#######3 | Permuting : 736/999 [00:40<00:13, 20.04it/s]
74%|#######3 | Permuting : 737/999 [00:40<00:13, 19.60it/s]
74%|#######3 | Permuting : 739/999 [00:40<00:12, 20.00it/s]
74%|#######4 | Permuting : 740/999 [00:40<00:17, 15.19it/s]
74%|#######4 | Permuting : 741/999 [00:40<00:16, 15.46it/s]
74%|#######4 | Permuting : 742/999 [00:40<00:16, 15.34it/s]
74%|#######4 | Permuting : 743/999 [00:41<00:16, 15.23it/s]
74%|#######4 | Permuting : 744/999 [00:41<00:16, 15.09it/s]
75%|#######4 | Permuting : 746/999 [00:41<00:16, 15.60it/s]
75%|#######4 | Permuting : 747/999 [00:41<00:15, 15.76it/s]
75%|#######4 | Permuting : 748/999 [00:41<00:15, 16.00it/s]
75%|#######4 | Permuting : 749/999 [00:41<00:15, 16.26it/s]
75%|#######5 | Permuting : 750/999 [00:41<00:15, 16.08it/s]
75%|#######5 | Permuting : 751/999 [00:41<00:15, 16.32it/s]
75%|#######5 | Permuting : 752/999 [00:41<00:14, 16.61it/s]
75%|#######5 | Permuting : 753/999 [00:41<00:14, 16.42it/s]
75%|#######5 | Permuting : 754/999 [00:41<00:15, 16.29it/s]
76%|#######5 | Permuting : 755/999 [00:41<00:15, 16.17it/s]
76%|#######5 | Permuting : 757/999 [00:41<00:14, 17.15it/s]
76%|#######5 | Permuting : 758/999 [00:42<00:18, 12.74it/s]
76%|#######5 | Permuting : 759/999 [00:42<00:18, 13.01it/s]
76%|#######6 | Permuting : 760/999 [00:42<00:17, 13.31it/s]
76%|#######6 | Permuting : 762/999 [00:42<00:16, 14.14it/s]
76%|#######6 | Permuting : 763/999 [00:42<00:16, 14.08it/s]
76%|#######6 | Permuting : 764/999 [00:42<00:16, 14.34it/s]
77%|#######6 | Permuting : 766/999 [00:42<00:15, 15.19it/s]
77%|#######7 | Permuting : 770/999 [00:42<00:13, 17.24it/s]
77%|#######7 | Permuting : 772/999 [00:42<00:12, 18.13it/s]
77%|#######7 | Permuting : 773/999 [00:42<00:12, 18.04it/s]
78%|#######7 | Permuting : 775/999 [00:42<00:11, 18.94it/s]
78%|#######7 | Permuting : 776/999 [00:42<00:11, 19.11it/s]
78%|#######7 | Permuting : 777/999 [00:42<00:11, 19.29it/s]
78%|#######7 | Permuting : 779/999 [00:42<00:10, 20.23it/s]
78%|#######8 | Permuting : 781/999 [00:42<00:10, 21.18it/s]
78%|#######8 | Permuting : 782/999 [00:42<00:10, 21.34it/s]
78%|#######8 | Permuting : 783/999 [00:42<00:10, 21.46it/s]
78%|#######8 | Permuting : 784/999 [00:42<00:09, 21.68it/s]
79%|#######8 | Permuting : 786/999 [00:43<00:09, 22.01it/s]
79%|#######8 | Permuting : 787/999 [00:43<00:09, 22.16it/s]
79%|#######8 | Permuting : 788/999 [00:43<00:09, 22.27it/s]
79%|#######8 | Permuting : 789/999 [00:43<00:09, 22.47it/s]
79%|#######9 | Permuting : 791/999 [00:43<00:08, 23.51it/s]
79%|#######9 | Permuting : 792/999 [00:43<00:09, 22.92it/s]
79%|#######9 | Permuting : 794/999 [00:43<00:08, 23.89it/s]
80%|#######9 | Permuting : 795/999 [00:43<00:08, 24.05it/s]
80%|#######9 | Permuting : 796/999 [00:43<00:11, 17.45it/s]
80%|#######9 | Permuting : 798/999 [00:43<00:11, 17.92it/s]
80%|######## | Permuting : 800/999 [00:43<00:10, 18.37it/s]
80%|######## | Permuting : 802/999 [00:43<00:10, 19.28it/s]
80%|######## | Permuting : 804/999 [00:44<00:09, 19.68it/s]
81%|######## | Permuting : 805/999 [00:44<00:09, 19.84it/s]
81%|######## | Permuting : 806/999 [00:44<00:09, 20.07it/s]
81%|######## | Permuting : 807/999 [00:44<00:09, 20.30it/s]
81%|######## | Permuting : 809/999 [00:44<00:08, 21.26it/s]
81%|########1 | Permuting : 811/999 [00:44<00:08, 21.60it/s]
81%|########1 | Permuting : 812/999 [00:44<00:08, 21.10it/s]
81%|########1 | Permuting : 813/999 [00:44<00:08, 21.32it/s]
81%|########1 | Permuting : 814/999 [00:44<00:08, 20.91it/s]
82%|########1 | Permuting : 815/999 [00:44<00:08, 20.52it/s]
82%|########1 | Permuting : 816/999 [00:44<00:09, 20.09it/s]
82%|########1 | Permuting : 817/999 [00:44<00:08, 20.33it/s]
82%|########1 | Permuting : 819/999 [00:45<00:11, 15.93it/s]
82%|########2 | Permuting : 821/999 [00:45<00:10, 16.39it/s]
82%|########2 | Permuting : 822/999 [00:45<00:10, 16.63it/s]
82%|########2 | Permuting : 823/999 [00:45<00:10, 16.51it/s]
82%|########2 | Permuting : 824/999 [00:45<00:10, 16.73it/s]
83%|########2 | Permuting : 826/999 [00:45<00:09, 17.64it/s]
83%|########2 | Permuting : 828/999 [00:45<00:09, 18.10it/s]
83%|########2 | Permuting : 829/999 [00:45<00:09, 18.29it/s]
83%|########3 | Permuting : 830/999 [00:45<00:09, 18.55it/s]
83%|########3 | Permuting : 831/999 [00:45<00:09, 18.25it/s]
83%|########3 | Permuting : 832/999 [00:45<00:09, 18.51it/s]
83%|########3 | Permuting : 833/999 [00:45<00:08, 18.71it/s]
83%|########3 | Permuting : 834/999 [00:45<00:08, 18.90it/s]
84%|########3 | Permuting : 835/999 [00:45<00:08, 19.17it/s]
84%|########3 | Permuting : 836/999 [00:46<00:10, 15.00it/s]
84%|########3 | Permuting : 837/999 [00:46<00:10, 14.90it/s]
84%|########3 | Permuting : 838/999 [00:46<00:10, 15.20it/s]
84%|########3 | Permuting : 839/999 [00:46<00:10, 15.46it/s]
84%|########4 | Permuting : 840/999 [00:46<00:10, 15.76it/s]
84%|########4 | Permuting : 841/999 [00:46<00:09, 16.02it/s]
84%|########4 | Permuting : 842/999 [00:46<00:09, 16.33it/s]
85%|########4 | Permuting : 845/999 [00:46<00:08, 17.56it/s]
85%|########4 | Permuting : 846/999 [00:46<00:08, 17.34it/s]
85%|########4 | Permuting : 847/999 [00:46<00:08, 17.63it/s]
85%|########4 | Permuting : 849/999 [00:46<00:08, 18.63it/s]
85%|########5 | Permuting : 851/999 [00:46<00:07, 19.64it/s]
85%|########5 | Permuting : 853/999 [00:46<00:07, 20.63it/s]
85%|########5 | Permuting : 854/999 [00:46<00:06, 20.80it/s]
86%|########5 | Permuting : 855/999 [00:46<00:07, 20.41it/s]
86%|########5 | Permuting : 857/999 [00:46<00:06, 21.44it/s]
86%|########5 | Permuting : 858/999 [00:47<00:08, 15.86it/s]
86%|########6 | Permuting : 861/999 [00:47<00:08, 16.94it/s]
86%|########6 | Permuting : 862/999 [00:47<00:07, 17.14it/s]
86%|########6 | Permuting : 863/999 [00:47<00:07, 17.41it/s]
86%|########6 | Permuting : 864/999 [00:47<00:07, 17.23it/s]
87%|########6 | Permuting : 865/999 [00:47<00:07, 17.45it/s]
87%|########6 | Permuting : 866/999 [00:47<00:07, 17.72it/s]
87%|########6 | Permuting : 868/999 [00:47<00:07, 18.68it/s]
87%|########7 | Permuting : 870/999 [00:47<00:06, 19.64it/s]
87%|########7 | Permuting : 871/999 [00:47<00:06, 19.80it/s]
87%|########7 | Permuting : 873/999 [00:47<00:06, 20.78it/s]
88%|########7 | Permuting : 875/999 [00:47<00:05, 21.15it/s]
88%|########7 | Permuting : 876/999 [00:47<00:05, 21.33it/s]
88%|########7 | Permuting : 877/999 [00:47<00:05, 20.84it/s]
88%|########7 | Permuting : 878/999 [00:47<00:05, 21.07it/s]
88%|########7 | Permuting : 879/999 [00:48<00:05, 21.24it/s]
88%|########8 | Permuting : 880/999 [00:48<00:05, 20.74it/s]
88%|########8 | Permuting : 881/999 [00:48<00:05, 20.98it/s]
88%|########8 | Permuting : 882/999 [00:48<00:05, 19.94it/s]
88%|########8 | Permuting : 883/999 [00:48<00:05, 20.18it/s]
88%|########8 | Permuting : 884/999 [00:48<00:07, 15.52it/s]
89%|########8 | Permuting : 886/999 [00:48<00:07, 16.05it/s]
89%|########8 | Permuting : 887/999 [00:48<00:07, 15.90it/s]
89%|########8 | Permuting : 888/999 [00:48<00:06, 16.19it/s]
89%|########8 | Permuting : 889/999 [00:48<00:06, 16.03it/s]
89%|########9 | Permuting : 890/999 [00:49<00:06, 16.32it/s]
89%|########9 | Permuting : 891/999 [00:49<00:06, 16.62it/s]
89%|########9 | Permuting : 892/999 [00:49<00:06, 16.86it/s]
89%|########9 | Permuting : 893/999 [00:49<00:06, 17.17it/s]
90%|########9 | Permuting : 895/999 [00:49<00:05, 18.13it/s]
90%|########9 | Permuting : 896/999 [00:49<00:05, 18.41it/s]
90%|########9 | Permuting : 899/999 [00:49<00:05, 19.64it/s]
90%|######### | Permuting : 901/999 [00:49<00:04, 20.65it/s]
90%|######### | Permuting : 903/999 [00:49<00:04, 20.94it/s]
90%|######### | Permuting : 904/999 [00:49<00:04, 21.17it/s]
91%|######### | Permuting : 905/999 [00:49<00:05, 16.41it/s]
91%|######### | Permuting : 907/999 [00:49<00:05, 17.31it/s]
91%|######### | Permuting : 908/999 [00:49<00:05, 17.52it/s]
91%|######### | Permuting : 909/999 [00:49<00:05, 17.78it/s]
91%|#########1| Permuting : 911/999 [00:49<00:04, 18.25it/s]
91%|#########1| Permuting : 913/999 [00:50<00:04, 18.70it/s]
92%|#########1| Permuting : 915/999 [00:50<00:04, 19.05it/s]
92%|#########1| Permuting : 916/999 [00:50<00:04, 19.29it/s]
92%|#########1| Permuting : 917/999 [00:50<00:04, 18.95it/s]
92%|#########1| Permuting : 918/999 [00:50<00:04, 19.20it/s]
92%|#########1| Permuting : 919/999 [00:50<00:04, 19.38it/s]
92%|#########2| Permuting : 921/999 [00:50<00:03, 20.36it/s]
92%|#########2| Permuting : 923/999 [00:50<00:03, 20.74it/s]
93%|#########2| Permuting : 925/999 [00:50<00:03, 21.71it/s]
93%|#########2| Permuting : 928/999 [00:50<00:03, 22.78it/s]
93%|#########3| Permuting : 930/999 [00:50<00:02, 23.03it/s]
93%|#########3| Permuting : 931/999 [00:51<00:03, 17.79it/s]
93%|#########3| Permuting : 932/999 [00:51<00:03, 17.99it/s]
93%|#########3| Permuting : 934/999 [00:51<00:03, 18.85it/s]
94%|#########3| Permuting : 936/999 [00:51<00:03, 19.71it/s]
94%|#########3| Permuting : 937/999 [00:51<00:03, 19.38it/s]
94%|#########3| Permuting : 938/999 [00:51<00:03, 19.60it/s]
94%|#########4| Permuting : 940/999 [00:51<00:02, 19.98it/s]
94%|#########4| Permuting : 942/999 [00:51<00:02, 20.25it/s]
94%|#########4| Permuting : 943/999 [00:51<00:02, 20.47it/s]
94%|#########4| Permuting : 944/999 [00:51<00:02, 20.07it/s]
95%|#########4| Permuting : 945/999 [00:51<00:02, 20.29it/s]
95%|#########4| Permuting : 948/999 [00:51<00:02, 21.33it/s]
95%|#########4| Permuting : 949/999 [00:51<00:02, 21.45it/s]
95%|#########5| Permuting : 951/999 [00:51<00:02, 22.37it/s]
95%|#########5| Permuting : 953/999 [00:51<00:01, 23.30it/s]
95%|#########5| Permuting : 954/999 [00:51<00:01, 23.37it/s]
96%|#########5| Permuting : 956/999 [00:51<00:01, 24.32it/s]
96%|#########5| Permuting : 959/999 [00:51<00:01, 26.05it/s]
96%|#########6| Permuting : 960/999 [00:52<00:01, 19.97it/s]
96%|#########6| Permuting : 961/999 [00:52<00:01, 20.11it/s]
96%|#########6| Permuting : 962/999 [00:52<00:01, 20.32it/s]
96%|#########6| Permuting : 963/999 [00:52<00:01, 20.46it/s]
96%|#########6| Permuting : 964/999 [00:52<00:01, 20.62it/s]
97%|#########6| Permuting : 965/999 [00:52<00:01, 20.84it/s]
97%|#########6| Permuting : 967/999 [00:52<00:01, 21.19it/s]
97%|#########6| Permuting : 968/999 [00:52<00:01, 21.31it/s]
97%|#########6| Permuting : 969/999 [00:52<00:01, 21.49it/s]
97%|#########7| Permuting : 970/999 [00:52<00:01, 21.69it/s]
97%|#########7| Permuting : 971/999 [00:52<00:01, 21.81it/s]
97%|#########7| Permuting : 972/999 [00:52<00:01, 22.02it/s]
97%|#########7| Permuting : 974/999 [00:52<00:01, 22.97it/s]
98%|#########7| Permuting : 975/999 [00:52<00:01, 23.11it/s]
98%|#########7| Permuting : 977/999 [00:52<00:00, 24.14it/s]
98%|#########7| Permuting : 979/999 [00:52<00:00, 25.17it/s]
98%|#########8| Permuting : 980/999 [00:52<00:00, 23.55it/s]
98%|#########8| Permuting : 981/999 [00:53<00:00, 22.97it/s]
98%|#########8| Permuting : 982/999 [00:53<00:01, 16.79it/s]
99%|#########8| Permuting : 985/999 [00:53<00:00, 17.90it/s]
99%|#########8| Permuting : 986/999 [00:53<00:00, 18.09it/s]
99%|#########8| Permuting : 987/999 [00:53<00:00, 18.34it/s]
99%|#########8| Permuting : 989/999 [00:53<00:00, 18.77it/s]
99%|#########9| Permuting : 991/999 [00:53<00:00, 19.19it/s]
99%|#########9| Permuting : 992/999 [00:53<00:00, 19.36it/s]
99%|#########9| Permuting : 993/999 [00:53<00:00, 19.59it/s]
100%|#########9| Permuting : 996/999 [00:53<00:00, 21.23it/s]
100%|#########9| Permuting : 998/999 [00:53<00:00, 22.16it/s]
100%|##########| Permuting : 999/999 [00:53<00:00, 22.12it/s]
100%|##########| Permuting : 999/999 [00:53<00:00, 18.50it/s]
Finally, we can plot our results. It is difficult to visualize clusters in time-frequency-sensor space; plotting time-frequency spectrograms and plotting topomaps display time-frequency and sensor space respectively but they are difficult to combine. We will plot topomaps with the clustered sensors colored in white adjacent to spectrograms in order to provide a visualization of the results. This is a dimensionally limited view, however. Each sensor has its own significant time-frequencies, but, in order to display a single spectrogram, all the time-frequencies that are significant for any sensor in the cluster are plotted as significant. This is a difficulty inherent to visualizing high-dimensional data and should be taken into consideration when interpreting results.
F_obs, clusters, p_values, _ = cluster_stats
good_cluster_inds = np.where(p_values < p_accept)[0]
for i_clu, clu_idx in enumerate(good_cluster_inds):
# unpack cluster information, get unique indices
freq_inds, time_inds, space_inds = clusters[clu_idx]
ch_inds = np.unique(space_inds)
time_inds = np.unique(time_inds)
freq_inds = np.unique(freq_inds)
# get topography for F stat
f_map = F_obs[freq_inds].mean(axis=0)
f_map = f_map[time_inds].mean(axis=0)
# get signals at the sensors contributing to the cluster
sig_times = epochs.times[time_inds]
# initialize figure
fig, ax_topo = plt.subplots(1, 1, figsize=(10, 3))
# create spatial mask
mask = np.zeros((f_map.shape[0], 1), dtype=bool)
mask[ch_inds, :] = True
# plot average test statistic and mark significant sensors
f_evoked = mne.EvokedArray(f_map[:, np.newaxis], epochs.info, tmin=0)
f_evoked.plot_topomap(times=0, mask=mask, axes=ax_topo, cmap='Reds',
vlim=(np.min, np.max), show=False, colorbar=False,
mask_params=dict(markersize=10))
image = ax_topo.images[0]
# create additional axes (for ERF and colorbar)
divider = make_axes_locatable(ax_topo)
# add axes for colorbar
ax_colorbar = divider.append_axes('right', size='5%', pad=0.05)
plt.colorbar(image, cax=ax_colorbar)
ax_topo.set_xlabel(
'Averaged F-map ({:0.3f} - {:0.3f} s)'.format(*sig_times[[0, -1]]))
# remove the title that would otherwise say "0.000 s"
ax_topo.set_title("")
# add new axis for spectrogram
ax_spec = divider.append_axes('right', size='300%', pad=1.2)
title = 'Cluster #{0}, {1} spectrogram'.format(i_clu + 1, len(ch_inds))
if len(ch_inds) > 1:
title += " (max over channels)"
F_obs_plot = F_obs[..., ch_inds].max(axis=-1)
F_obs_plot_sig = np.zeros(F_obs_plot.shape) * np.nan
F_obs_plot_sig[tuple(np.meshgrid(freq_inds, time_inds))] = \
F_obs_plot[tuple(np.meshgrid(freq_inds, time_inds))]
for f_image, cmap in zip([F_obs_plot, F_obs_plot_sig], ['gray', 'autumn']):
c = ax_spec.imshow(f_image, cmap=cmap, aspect='auto', origin='lower',
extent=[epochs.times[0], epochs.times[-1],
freqs[0], freqs[-1]])
ax_spec.set_xlabel('Time (ms)')
ax_spec.set_ylabel('Frequency (Hz)')
ax_spec.set_title(title)
# add another colorbar
ax_colorbar2 = divider.append_axes('right', size='5%', pad=0.05)
plt.colorbar(c, cax=ax_colorbar2)
ax_colorbar2.set_ylabel('F-stat')
# clean up viz
mne.viz.tight_layout(fig=fig)
fig.subplots_adjust(bottom=.05)
plt.show()
Exercises#
References#
Total running time of the script: ( 1 minutes 23.428 seconds)
Estimated memory usage: 128 MB