04. Convert EEG data to BIDS format

In this example, we use MNE-BIDS to create a BIDS-compatible directory of EEG data. Specifically, we will follow these steps:

1. Download some EEG data from the PhysioBank database.

2. Load the data, extract information, and save it in a new BIDS directory.

3. Check the result and compare it with the standard.

4. Cite mne-bids.

# Authors: Stefan Appelhoff <stefan.appelhoff@mailbox.org>
#
# License: BSD (3-clause)

We are importing everything we need for this example:

import os.path as op
import shutil

import mne
from mne.datasets import eegbci

from mne_bids import write_raw_bids, BIDSPath, print_dir_tree
from mne_bids.stats import count_events

Download the data

First, we need some data to work with. We will use the EEG Motor Movement/Imagery Dataset available on the PhysioBank database.

The data consists of 109 volunteers performing 14 experimental runs each. For each subject, there were two baseline tasks (i) eyes open, (ii) eyes closed, as well as four different motor imagery tasks.

In this example, we will download the data for a single subject doing the baseline task “eyes closed” and format it to the Brain Imaging Data Structure (BIDS).

Conveniently, there is already a data loading function available with MNE-Python:

# Download the data for subject 1, for the 2 minutes of eyes closed rest task.
# From the online documentation of the data we know that run "2" corresponds
# to the "eyes closed" task.
subject = 1
run = 2
eegbci.load_data(subject=subject, runs=run, update_path=True)

Out:

['/Users/hoechenberger/mne_data/MNE-eegbci-data/files/eegmmidb/1.0.0/S001/S001R02.edf']

Let’s see whether the data has been downloaded using a quick visualization of the directory tree.

# get MNE directory with example data
mne_data_dir = mne.get_config('MNE_DATASETS_EEGBCI_PATH')
data_dir = op.join(mne_data_dir, 'MNE-eegbci-data')

print_dir_tree(data_dir)

Out:

|MNE-eegbci-data/
|--- files/
|------ eegmmidb/
|--------- 1.0.0/
|------------ S001/
|--------------- S001R02.edf
|--------------- S001R04.edf
|--------------- S001R08.edf
|--------------- S001R12.edf
|------------ S002/
|--------------- S002R04.edf
|--------------- S002R08.edf
|--------------- S002R12.edf

The data are in the European Data Format with the .edf extension, which is good for us because next to the BrainVision format, EDF is one of the recommended file formats for EEG data in BIDS format.

However, apart from the data format, we need to build a directory structure and supply meta data files to properly bidsify this data.

We will do exactly that in the next step.

Convert to BIDS

Let’s start with loading the data and extracting the events. We are reading the data using MNE-Python’s io module and the mne.io.read_raw_edf() function. Note that we must use the preload=False parameter, which is the default in MNE-Python. It prevents the data from being loaded and modified when converting to BIDS.

# Load the data from "2 minutes eyes closed rest"
edf_path = eegbci.load_data(subject=subject, runs=run)[0]
raw = mne.io.read_raw_edf(edf_path, preload=False)
raw.info['line_freq'] = 50  # specify power line frequency as required by BIDS

Out:

Extracting EDF parameters from /Users/hoechenberger/mne_data/MNE-eegbci-data/files/eegmmidb/1.0.0/S001/S001R02.edf...
EDF file detected
Setting channel info structure...
Creating raw.info structure...

For the sake of the example we will also pretend that we have the electrode coordinates for the data recordings. We will use a coordinates file from the MNE testing data in CapTrak format.

Note

The *electrodes.tsv and *coordsystem.json files in BIDS are intended to carry information about digitized (i.e., measured) electrode positions on the scalp of the research subject. Do not (!) use these files to store “template” or “idealized” electrode positions, like those that can be obtained from mne.channels.make_standard_montage()!

# Get the electrode coordinates
testing_data = mne.datasets.testing.data_path()
captrak_path = op.join(testing_data, 'montage', 'captrak_coords.bvct')
montage = mne.channels.read_dig_captrak(captrak_path)

# Rename the montage channel names only for this example, because as said
# before, coordinate and EEG data were not actually collected together
# Do *not* do this for your own data.
montage.rename_channels(dict(zip(montage.ch_names, raw.ch_names)))

# "attach" the electrode coordinates to the `raw` object
raw.set_montage(montage)

# show the electrode positions
raw.plot_sensors()

Out:

<Figure size 640x640 with 1 Axes>

With these steps, we have everything to start a new BIDS directory using our data.

To do that, we can use write_raw_bids()

Generally, write_raw_bids() tries to extract as much meta data as possible from the raw data and then formats it in a BIDS compatible way. write_raw_bids() takes a bunch of inputs, most of which are however optional. The required inputs are:

• raw

• bids_basename

• bids_root

… as you can see in the docstring:

print(write_raw_bids.__doc__)

Out:

Save raw data to a BIDS-compliant folder structure.

    .. warning:: * The original file is simply copied over if the original
                   file format is BIDS-supported for that datatype. Otherwise,
                   this function will convert to a BIDS-supported file format
                   while warning the user. For EEG and iEEG data, conversion
                   will be to BrainVision format; for MEG, conversion will be
                   to FIFF.

                 * ``mne-bids`` will infer the manufacturer information
                   from the file extension. If your file format is non-standard
                   for the manufacturer, please update the manufacturer field
                   in the sidecars manually.

    Parameters
    ----------
    raw : instance of mne.io.Raw
        The raw data. It must be an instance of `mne.io.Raw`. The data
        should not be loaded from disk, i.e., ``raw.preload`` must be
        ``False``.
    bids_path : BIDSPath
        The file to write. The `mne_bids.BIDSPath` instance passed here
        **must** have the ``.root`` attribute set. If the ``.datatype``
        attribute is not set, it will be inferred from the recording data type
        found in ``raw``.
        Example::

            bids_path = BIDSPath(subject='01', session='01', task='testing',
                                 acquisition='01', run='01', root='/data/BIDS')

        This will write the following files in the correct subfolder ``root``::

            sub-01_ses-01_task-testing_acq-01_run-01_meg.fif
            sub-01_ses-01_task-testing_acq-01_run-01_meg.json
            sub-01_ses-01_task-testing_acq-01_run-01_channels.tsv
            sub-01_ses-01_task-testing_acq-01_run-01_coordsystem.json

        and the following one if ``events_data`` is not ``None``::

            sub-01_ses-01_task-testing_acq-01_run-01_events.tsv

        and add a line to the following files::

            participants.tsv
            scans.tsv

        Note that the data type is automatically inferred from the raw
        object, as well as the extension. Data with MEG and other
        electrophysiology data in the same file will be stored as ``'meg'``.
    events_data : path-like | array | None
        Use this parameter to specify events to write to the ``*_events.tsv``
        sidecar file, additionally to the object's `mne.Annotations` (which
        are always written).
        If a path, specifies the location of an MNE events file.
        If an array, the MNE events array (shape: ``(n_events, 3)``).
        If a path or an array and ``raw.annotations`` exist, the union of
        ``event_data`` and ``raw.annotations`` will be written.
        Corresponding descriptions for all event IDs (listed in the third
        column of the MNE events array) must be specified via the ``event_id``
        parameter; otherwise, an exception is raised.
        If ``None``, events will only be inferred from the the raw object's
        `mne.Annotations`.

        .. note::
           If ``not None``, writes the union of ``events_data`` and
           ``raw.annotations``. If you wish to **only** write
           ``raw.annotations``, pass ``events_data=None``. If you want to
           **exclude** the events in ``raw.annotations`` from being written,
           call ``raw.set_annotations(None)`` before invoking this function.

        .. note::
           Descriptions of all event IDs must be specified via the ``event_id``
           parameter.

    event_id : dict | None
        Descriptions of all event IDs, if you passed ``events_data``.
        The descriptions will be written to the ``trial_type`` column in
        ``*_events.tsv``. The dictionary keys correspond to the event
        descriptions and the values to the event IDs. You must specify a
        description for all event IDs in ``events_data``.
    anonymize : dict | None
        If `None` (default), no anonymization is performed.
        If a dictionary, data will be anonymized depending on the dictionary
        keys: ``daysback`` is a required key, ``keep_his`` is optional.

        ``daysback`` : int
            Number of days by which to move back the recording date in time.
            In studies with multiple subjects the relative recording date
            differences between subjects can be kept by using the same number
            of ``daysback`` for all subject anonymizations. ``daysback`` should
            be great enough to shift the date prior to 1925 to conform with
            BIDS anonymization rules.

        ``keep_his`` : bool
            If ``False`` (default), all subject information next to the
            recording date will be overwritten as well. If True, keep subject
            information apart from the recording date.

    overwrite : bool
        Whether to overwrite existing files or data in files.
        Defaults to ``False``.

        If ``True``, any existing files with the same BIDS parameters
        will be overwritten with the exception of the ``*_participants.tsv``
        and ``*_scans.tsv`` files. For these files, parts of pre-existing data
        that match the current data will be replaced. For
        ``*_participants.tsv``, specifically, age, sex and hand fields will be
        overwritten, while any manually added fields in ``participants.json``
        and ``participants.tsv`` by a user will be retained.
        If ``False``, no existing data will be overwritten or
        replaced.
    verbose : bool
        If ``True``, this will print a snippet of the sidecar files. Otherwise,
        no content will be printed.

    Returns
    -------
    bids_path : BIDSPath
        The path of the created data file.

    Notes
    -----
    You should ensure that ``raw.info['subject_info']`` and
    ``raw.info['meas_date']`` are set to proper (not-``None``) values to allow
    for the correct computation of each participant's age when creating
    ``*_participants.tsv``.

    This function will convert existing `mne.Annotations` from
    ``raw.annotations`` to events. Additionally, any events supplied via
    ``events_data`` will be written too. To avoid writing of annotations,
    remove them from the raw file via ``raw.set_annotations(None)`` before
    invoking ``write_raw_bids``.

    To write events encoded in a ``STIM`` channel, you first need to create the
    events array manually and pass it to this function:

    ..
        events = mne.find_events(raw, min_duration=0.002)
        write_raw_bids(..., events_data=events)

    See the documentation of `mne.find_events` for more information on event
    extraction from ``STIM`` channels.

    See Also
    --------
    mne.io.Raw.anonymize
    mne.find_events
    mne.Annotations
    mne.events_from_annotations

We loaded S001R02.edf, which corresponds to subject 1 in the second run. In the second run of the experiment, the task was to rest with closed eyes.

# zero padding to account for >100 subjects in this dataset
subject_id = '001'

# define a task name and a directory where to save the data to
task = 'RestEyesClosed'
bids_root = op.join(mne_data_dir, 'eegmmidb_bids_eeg_example')

To ensure the output path doesn’t contain any leftover files from previous tests and example runs, we simply delete it.

Warning

Do not delete directories that may contain important data!

if op.exists(bids_root):
    shutil.rmtree(bids_root)

The data contains annotations; which will be converted to events automatically by MNE-BIDS when writing the BIDS data:

print(raw.annotations)

Out:

<Annotations | 1 segment: T0 (1)>

Finally, let’s write the BIDS data!

bids_path = BIDSPath(subject=subject_id, task=task, root=bids_root)
write_raw_bids(raw, bids_path, overwrite=True)

Out:

Extracting EDF parameters from /Users/hoechenberger/mne_data/MNE-eegbci-data/files/eegmmidb/1.0.0/S001/S001R02.edf...
EDF file detected
Setting channel info structure...
Creating raw.info structure...

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/README'...

References
----------
Appelhoff, S., Sanderson, M., Brooks, T., Vliet, M., Quentin, R., Holdgraf, C., Chaumon, M., Mikulan, E., Tavabi, K., Höchenberger, R., Welke, D., Brunner, C., Rockhill, A., Larson, E., Gramfort, A. and Jas, M. (2019). MNE-BIDS: Organizing electrophysiological data into the BIDS format and facilitating their analysis. Journal of Open Source Software 4: (1896). https://doi.org/10.21105/joss.01896

Pernet, C. R., Appelhoff, S., Gorgolewski, K. J., Flandin, G., Phillips, C., Delorme, A., Oostenveld, R. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. Scientific Data, 6, 103. https://doi.org/10.1038/s41597-019-0104-8


Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/participants.tsv'...

participant_id  age     sex     hand
sub-001 n/a     n/a     n/a

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/participants.json'...

{
    "participant_id": {
        "Description": "Unique participant identifier"
    },
    "age": {
        "Description": "Age of the participant at time of testing",
        "Units": "years"
    },
    "sex": {
        "Description": "Biological sex of the participant",
        "Levels": {
            "F": "female",
            "M": "male"
        }
    },
    "hand": {
        "Description": "Handedness of the participant",
        "Levels": {
            "R": "right",
            "L": "left",
            "A": "ambidextrous"
        }
    }
}
Writing electrodes file to...  /Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_electrodes.tsv
Writing coordsytem file to...  /Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_coordsystem.json

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_electrodes.tsv'...

name    x       y       z
Fc5.    -0.09731968723468239    -0.0021423686071579805  0.050226741225281264
Fc3.    -0.08856262271830136    0.03656538257252212     0.08223925927400842
Fc1.    -0.08409763256980056    0.0651383327975854      0.04885693263757247
Fcz.    -0.09271418986954857    -0.0008204160816824106  0.08669305289548136
Fc2.    -0.09514340411749506    0.033122249460160715    0.046783546930167653

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_coordsystem.json'...

{
    "EEGCoordinateSystem": "CapTrak",
    "EEGCoordinateUnits": "m",
    "EEGCoordinateSystemDescription": "The X-axis goes from the left preauricular point (LPA) through the right preauricular point (RPA). The Y-axis goes orthogonally to the X-axis through the nasion (NAS). The Z-axis goes orthogonally to the XY-plane through the vertex of the head. This corresponds to a \"RAS\" orientation with the origin of the coordinate system approximately between the ears. See Appendix VIII in the BIDS specification.",
    "AnatomicalLandmarkCoordinates": {
        "NAS": [
            -3.788238398497924e-18,
            0.11309931478694205,
            -3.0814879110195774e-33
        ],
        "LPA": [
            -0.09189697162389295,
            3.078070254157709e-18,
            0.0
        ],
        "RPA": [
            0.09240077493980713,
            -3.094945043100789e-18,
            -6.162975822039155e-33
        ]
    },
    "AnatomicalLandmarkCoordinateSystem": "CapTrak",
    "AnatomicalLandmarkCoordinateUnits": "m"
}

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_task-RestEyesClosed_events.tsv'...

onset   duration        trial_type      value   sample
0.0     60.2    T0      1       0

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/dataset_description.json'...

{
    "Name": " ",
    "BIDSVersion": "1.4.0",
    "DatasetType": "raw",
    "Authors": [
        "Please cite MNE-BIDS in your publication before removing this (citations in README)"
    ]
}
Reading 0 ... 9759  =      0.000 ...    60.994 secs...

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_task-RestEyesClosed_eeg.json'...

{
    "TaskName": "RestEyesClosed",
    "Manufacturer": "n/a",
    "PowerLineFrequency": 50,
    "SamplingFrequency": 160.0,
    "SoftwareFilters": "n/a",
    "RecordingDuration": 60.99375,
    "RecordingType": "continuous",
    "EEGReference": "n/a",
    "EEGGround": "n/a",
    "EEGPlacementScheme": "n/a",
    "EEGChannelCount": 64,
    "EOGChannelCount": 0,
    "ECGChannelCount": 0,
    "EMGChannelCount": 0,
    "MiscChannelCount": 0,
    "TriggerChannelCount": 0
}

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_task-RestEyesClosed_channels.tsv'...

name    type    units   low_cutoff      high_cutoff     description     sampling_frequency      status  status_description
Fc5.    EEG     µV      0.0     80.0    ElectroEncephaloGram    160.0   good    n/a
Fc3.    EEG     µV      0.0     80.0    ElectroEncephaloGram    160.0   good    n/a
Fc1.    EEG     µV      0.0     80.0    ElectroEncephaloGram    160.0   good    n/a
Fcz.    EEG     µV      0.0     80.0    ElectroEncephaloGram    160.0   good    n/a
Fc2.    EEG     µV      0.0     80.0    ElectroEncephaloGram    160.0   good    n/a
Copying data files to sub-001_task-RestEyesClosed_eeg.edf

Writing '/Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/sub-001_scans.tsv'...

filename        acq_time
eeg/sub-001_task-RestEyesClosed_eeg.edf 2009-08-12T16:15:00.000000Z
Wrote /Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example/sub-001/sub-001_scans.tsv entry with eeg/sub-001_task-RestEyesClosed_eeg.edf.

BIDSPath(
root: /Users/hoechenberger/mne_data/eegmmidb_bids_eeg_example
datatype: eeg
basename: sub-001_task-RestEyesClosed_eeg.edf)

What does our fresh BIDS directory look like?

print_dir_tree(bids_root)

Out:

|eegmmidb_bids_eeg_example/
|--- README
|--- dataset_description.json
|--- participants.json
|--- participants.tsv
|--- sub-001/
|------ sub-001_scans.tsv
|------ eeg/
|--------- sub-001_coordsystem.json
|--------- sub-001_electrodes.tsv
|--------- sub-001_task-RestEyesClosed_channels.tsv
|--------- sub-001_task-RestEyesClosed_eeg.edf
|--------- sub-001_task-RestEyesClosed_eeg.json
|--------- sub-001_task-RestEyesClosed_events.tsv

Finally let’s get an overview of the events on the whole dataset

counts = count_events(bids_root)
counts

	RestEyesClosed
trial_type	T0
subject
001	1

We can see that MNE-BIDS wrote several important files related to subject 1 for us:

• electrodes.tsv containing the electrode coordinates and coordsystem.json, which contains the metadata about the electrode coordinates.

• The actual EDF data file (now with a proper BIDS name) and an accompanying *_eeg.json file that contains metadata about the EEG recording.

• The *scans.json file lists all data recordings with their acquisition date. This file becomes more handy once there are multiple sessions and recordings to keep track of.

• And finally, channels.tsv and events.tsv which contain even further metadata.

Next to the subject specific files, MNE-BIDS also created several experiment specific files. However, we will not go into detail for them in this example.

Cite mne-bids

After a lot of work was done by MNE-BIDS, it’s fair to cite the software when preparing a manuscript and/or a dataset publication.

We can see that the appropriate citations are already written in the README file.

If you are preparing a manuscript, please make sure to also cite MNE-BIDS there.

readme = op.join(bids_root, 'README')
with open(readme, 'r', encoding='utf-8-sig') as fid:
    text = fid.read()
print(text)

Out:

References
----------
Appelhoff, S., Sanderson, M., Brooks, T., Vliet, M., Quentin, R., Holdgraf, C., Chaumon, M., Mikulan, E., Tavabi, K., Höchenberger, R., Welke, D., Brunner, C., Rockhill, A., Larson, E., Gramfort, A. and Jas, M. (2019). MNE-BIDS: Organizing electrophysiological data into the BIDS format and facilitating their analysis. Journal of Open Source Software 4: (1896). https://doi.org/10.21105/joss.01896

Pernet, C. R., Appelhoff, S., Gorgolewski, K. J., Flandin, G., Phillips, C., Delorme, A., Oostenveld, R. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. Scientific Data, 6, 103. https://doi.org/10.1038/s41597-019-0104-8

Now it’s time to manually check the BIDS directory and the meta files to add all the information that MNE-BIDS could not infer. For instance, you must describe EEGReference and EEGGround yourself. It’s easy to find these by searching for “n/a” in the sidecar files.

Remember that there is a convenient javascript tool to validate all your BIDS directories called the “BIDS-validator”, available as a web version and a command line tool:

Web version: https://bids-standard.github.io/bids-validator/

Command line tool: https://www.npmjs.com/package/bids-validator