Ctrl+K

Note

Go to the end to download the full example code

09. Manually storing empty room data#

This example demonstrates how to store empty room data “manually” in the BIDS format and how to retrieve them.

# Authors: Mainak Jas <mainakjas@gmail.com>
#
# License: BSD-3-Clause

We are dealing with MEG data, which is often accompanied by so-called “empty room” recordings for noise modeling. Below we show that we can use MNE-BIDS to also save such a recording with the just converted data.

Note

The steps described below should only be followed if you intend to store empty-room and experimental data in separate steps for some reason. Otherwise, we recommend you store both with a single call to mne_bids.write_raw_bids() by passing the empty-room raw data via the empty_room parameter, as demonstrated in 02. Convert MNE sample data to BIDS format. What is described in the example below is targeted towards advanced users only.

Let us first import mne_bids.

import os.path as op
import shutil
from datetime import datetime, timezone

import mne
from mne.datasets import sample

from mne_bids import BIDSPath, print_dir_tree, read_raw_bids, write_raw_bids

And define the paths and event_id dictionary.

data_path = sample.data_path()
raw_fname = op.join(data_path, "MEG", "sample", "sample_audvis_raw.fif")

bids_root = op.join(data_path, "..", "MNE-sample-data-bids")

To ensure the output path doesn’t contain any leftover files from previous tests and example runs, we simply delete it.

Warning

Do not delete directories that may contain important data!

if op.exists(bids_root):
    shutil.rmtree(bids_root)

Specify the raw file and write the BIDS data.

raw = mne.io.read_raw_fif(raw_fname)
raw.info["line_freq"] = 60  # specify power line frequency as required by BIDS

bids_path = BIDSPath(
    subject="01", session="01", task="audiovisual", run="01", root=bids_root
)
write_raw_bids(raw, bids_path, overwrite=True)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif...
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 25800 ... 192599 =     42.956 ...   320.670 secs
Ready.
Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif...
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 25800 ... 192599 =     42.956 ...   320.670 secs
Ready.
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/README'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.tsv'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_coordsystem.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_coordsystem.json'...
/home/circleci/project/examples/convert_empty_room.py:68: RuntimeWarning: No events found or provided. Please add annotations to the raw data, or provide the events and event_id parameters. For resting state data, BIDS recommends naming the task using labels beginning with "rest".
  write_raw_bids(raw, bids_path, overwrite=True)
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/dataset_description.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_meg.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_channels.tsv'...
Copying data files to sub-01_ses-01_task-audiovisual_run-01_meg.fif
Reserving possible split file sub-01_ses-01_task-audiovisual_run-01_split-01_meg.fif
Writing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_meg.fif
Closing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_meg.fif
[done]
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/sub-01_ses-01_scans.tsv'...
Wrote /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/sub-01_ses-01_scans.tsv entry with meg/sub-01_ses-01_task-audiovisual_run-01_meg.fif.

BIDSPath(
root: /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids
datatype: meg
basename: sub-01_ses-01_task-audiovisual_run-01_meg.fif)

Specify some empty room data and run BIDS conversion on it.

er_raw_fname = op.join(data_path, "MEG", "sample", "ernoise_raw.fif")
er_raw = mne.io.read_raw_fif(er_raw_fname)
er_raw.info["line_freq"] = 60  # specify power line frequency as req. by BIDS

# For empty room data we need to specify the recording date in the format
# YYYYMMDD for the session id.
er_date = er_raw.info["meas_date"].strftime("%Y%m%d")
print(er_date)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/ernoise_raw.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
20021206

The measurement date is

raw_date = raw.info["meas_date"].strftime("%Y%m%d")
print(raw_date)

20021203

We also need to specify that the subject ID is ‘emptyroom’, and that the task is ‘noise’ (these are BIDS rules).

er_bids_path = BIDSPath(
    subject="emptyroom", session=er_date, task="noise", root=bids_root
)
write_raw_bids(er_raw, er_bids_path, overwrite=True)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/ernoise_raw.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.tsv'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.json'...
Writing of electrodes.tsv is not supported for data type "meg". Skipping ...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/dataset_description.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/meg/sub-emptyroom_ses-20021206_task-noise_meg.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/meg/sub-emptyroom_ses-20021206_task-noise_channels.tsv'...
Copying data files to sub-emptyroom_ses-20021206_task-noise_meg.fif
Reserving possible split file sub-emptyroom_ses-20021206_task-noise_split-01_meg.fif
Writing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/meg/sub-emptyroom_ses-20021206_task-noise_meg.fif
Closing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/meg/sub-emptyroom_ses-20021206_task-noise_meg.fif
[done]
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/sub-emptyroom_ses-20021206_scans.tsv'...
Wrote /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021206/sub-emptyroom_ses-20021206_scans.tsv entry with meg/sub-emptyroom_ses-20021206_task-noise_meg.fif.

BIDSPath(
root: /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids
datatype: meg
basename: sub-emptyroom_ses-20021206_task-noise_meg.fif)

Just to illustrate, we can save more than one empty room file for different dates. Here, they will all contain the same data but in your study, they will be different on different days.

dates = ["20021204", "20021201", "20021001"]

for date in dates:
    er_bids_path.update(session=date)
    er_meas_date = datetime.strptime(date, "%Y%m%d")
    er_raw.set_meas_date(er_meas_date.replace(tzinfo=timezone.utc))
    write_raw_bids(er_raw, er_bids_path, overwrite=True)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/ernoise_raw.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.tsv'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.json'...
Writing of electrodes.tsv is not supported for data type "meg". Skipping ...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/dataset_description.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_meg.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_channels.tsv'...
Copying data files to sub-emptyroom_ses-20021204_task-noise_meg.fif
Reserving possible split file sub-emptyroom_ses-20021204_task-noise_split-01_meg.fif
Writing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_meg.fif
Closing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_meg.fif
[done]
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/sub-emptyroom_ses-20021204_scans.tsv'...
Wrote /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/sub-emptyroom_ses-20021204_scans.tsv entry with meg/sub-emptyroom_ses-20021204_task-noise_meg.fif.
Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/ernoise_raw.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.tsv'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.json'...
Writing of electrodes.tsv is not supported for data type "meg". Skipping ...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/dataset_description.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/meg/sub-emptyroom_ses-20021201_task-noise_meg.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/meg/sub-emptyroom_ses-20021201_task-noise_channels.tsv'...
Copying data files to sub-emptyroom_ses-20021201_task-noise_meg.fif
Reserving possible split file sub-emptyroom_ses-20021201_task-noise_split-01_meg.fif
Writing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/meg/sub-emptyroom_ses-20021201_task-noise_meg.fif
Closing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/meg/sub-emptyroom_ses-20021201_task-noise_meg.fif
[done]
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/sub-emptyroom_ses-20021201_scans.tsv'...
Wrote /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021201/sub-emptyroom_ses-20021201_scans.tsv entry with meg/sub-emptyroom_ses-20021201_task-noise_meg.fif.
Opening raw data file /home/circleci/mne_data/MNE-sample-data/MEG/sample/ernoise_raw.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.tsv'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/participants.json'...
Writing of electrodes.tsv is not supported for data type "meg". Skipping ...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/dataset_description.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/meg/sub-emptyroom_ses-20021001_task-noise_meg.json'...
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/meg/sub-emptyroom_ses-20021001_task-noise_channels.tsv'...
Copying data files to sub-emptyroom_ses-20021001_task-noise_meg.fif
Reserving possible split file sub-emptyroom_ses-20021001_task-noise_split-01_meg.fif
Writing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/meg/sub-emptyroom_ses-20021001_task-noise_meg.fif
Closing /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/meg/sub-emptyroom_ses-20021001_task-noise_meg.fif
[done]
Writing '/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/sub-emptyroom_ses-20021001_scans.tsv'...
Wrote /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021001/sub-emptyroom_ses-20021001_scans.tsv entry with meg/sub-emptyroom_ses-20021001_task-noise_meg.fif.

Let us look at the directory structure

print_dir_tree(bids_root)

|MNE-sample-data-bids/
|--- README
|--- dataset_description.json
|--- participants.json
|--- participants.tsv
|--- sub-01/
|------ ses-01/
|--------- sub-01_ses-01_scans.tsv
|--------- meg/
|------------ sub-01_ses-01_coordsystem.json
|------------ sub-01_ses-01_task-audiovisual_run-01_channels.tsv
|------------ sub-01_ses-01_task-audiovisual_run-01_meg.fif
|------------ sub-01_ses-01_task-audiovisual_run-01_meg.json
|--- sub-emptyroom/
|------ ses-20021001/
|--------- sub-emptyroom_ses-20021001_scans.tsv
|--------- meg/
|------------ sub-emptyroom_ses-20021001_task-noise_channels.tsv
|------------ sub-emptyroom_ses-20021001_task-noise_meg.fif
|------------ sub-emptyroom_ses-20021001_task-noise_meg.json
|------ ses-20021201/
|--------- sub-emptyroom_ses-20021201_scans.tsv
|--------- meg/
|------------ sub-emptyroom_ses-20021201_task-noise_channels.tsv
|------------ sub-emptyroom_ses-20021201_task-noise_meg.fif
|------------ sub-emptyroom_ses-20021201_task-noise_meg.json
|------ ses-20021204/
|--------- sub-emptyroom_ses-20021204_scans.tsv
|--------- meg/
|------------ sub-emptyroom_ses-20021204_task-noise_channels.tsv
|------------ sub-emptyroom_ses-20021204_task-noise_meg.fif
|------------ sub-emptyroom_ses-20021204_task-noise_meg.json
|------ ses-20021206/
|--------- sub-emptyroom_ses-20021206_scans.tsv
|--------- meg/
|------------ sub-emptyroom_ses-20021206_task-noise_channels.tsv
|------------ sub-emptyroom_ses-20021206_task-noise_meg.fif
|------------ sub-emptyroom_ses-20021206_task-noise_meg.json

To get an accurate estimate of the noise, it is important that the empty room recording be as close in date as the raw data. We can retrieve the basename corresponding to the empty room recording that is closest in time to the experimental measurement.

er_bids_path = bids_path.find_empty_room()
print(er_bids_path)

The MEG sidecar file does not contain an "AssociatedEmptyRoom" entry. Will try to find a matching empty-room recording based on the measurement date …
Opening raw data file /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_meg.fif...
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 25800 ... 192599 =     42.956 ...   320.670 secs
Ready.
/home/circleci/project/examples/convert_empty_room.py:117: RuntimeWarning: Did not find any events.tsv associated with sub-01_ses-01_task-audiovisual_run-01.

The search_str was "/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/**/meg/sub-01_ses-01*events.tsv"
  er_bids_path = bids_path.find_empty_room()
Reading channel info from /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-01/ses-01/meg/sub-01_ses-01_task-audiovisual_run-01_channels.tsv.
/home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_meg.fif

Finally, we can read the empty room file using

raw = read_raw_bids(bids_path=er_bids_path)
print(raw)

Opening raw data file /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_meg.fif...
Isotrak not found
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 19800 ... 85867 =     32.966 ...   142.965 secs
Ready.
Reading channel info from /home/circleci/mne_data/MNE-sample-data/../MNE-sample-data-bids/sub-emptyroom/ses-20021204/meg/sub-emptyroom_ses-20021204_task-noise_channels.tsv.
<Raw | sub-emptyroom_ses-20021204_task-noise_meg.fif, 315 x 66068 (110.0 s), ~296 kB, data not loaded>

Total running time of the script: (0 minutes 1.924 seconds)

Gallery generated by Sphinx-Gallery