04. Convert EEG data to BIDS format#
In this example, we use MNE-BIDS to create a BIDS-compatible directory of EEG data. Specifically, we will follow these steps:
Download some EEG data from the PhysioBank database.
Load the data, extract information, and save it in a new BIDS directory.
Check the result and compare it with the standard.
# Authors: Stefan Appelhoff <email@example.com> # # License: BSD-3-Clause
We are importing everything we need for this example:
Download the data#
First, we need some data to work with. We will use the EEG Motor Movement/Imagery Dataset available on the PhysioBank database.
The data consists of 109 volunteers performing 14 experimental runs each. For each subject, there were two baseline tasks (i) eyes open, (ii) eyes closed, as well as four different motor imagery tasks.
In this example, we will download the data for a single subject doing the baseline task “eyes closed” and format it to the Brain Imaging Data Structure (BIDS).
Conveniently, there is already a data loading function available with MNE-Python:
Let’s see whether the data has been downloaded using a quick visualization of the directory tree.
|MNE-eegbci-data\ |--- files\ |------ eegmmidb\ |--------- 1.0.0\ |------------ S001\ |--------------- S001R02.edf |--------------- S001R04.edf |--------------- S001R08.edf |--------------- S001R12.edf |------------ S002\ |--------------- S002R04.edf |--------------- S002R08.edf |--------------- S002R12.edf |------------ S004\ |--------------- S004R01.edf
However, apart from the data format, we need to build a directory structure and supply meta data files to properly bidsify this data.
We will do exactly that in the next step.
Convert to BIDS#
Let’s start with loading the data and extracting the events.
We are reading the data using MNE-Python’s
io module and the
Note that we must use the
preload=False parameter, which is the default
It prevents the data from being loaded and modified when converting to BIDS.
Extracting EDF parameters from C:\Users\stefan\mne_data\MNE-eegbci-data\files\eegmmidb\1.0.0\S001\S001R02.edf... EDF file detected Setting channel info structure... Creating raw.info structure...
For the sake of the example we will also pretend that we have the electrode coordinates for the data recordings. We will use a coordinates file from the MNE testing data in CapTrak format.
*coordsystem.json files in BIDS are
intended to carry information about digitized (i.e., measured)
electrode positions on the scalp of the research subject. Do not
(!) use these files to store “template” or “idealized” electrode
positions, like those that can be obtained from
# Get the electrode coordinates testing_data = mne.datasets.testing.data_path() captrak_path = op.join(testing_data, "montage", "captrak_coords.bvct") montage = mne.channels.read_dig_captrak(captrak_path) # Rename the montage channel names only for this example, because as said # before, coordinate and EEG data were not actually collected together # Do *not* do this for your own data. montage.rename_channels(dict(zip(montage.ch_names, raw.ch_names))) # "attach" the electrode coordinates to the `raw` object # Note that this only works for some channel types (EEG/sEEG/ECoG/DBS/fNIRS) raw.set_montage(montage) # show the electrode positions raw.plot_sensors()
<Figure size 640x640 with 1 Axes>
With these steps, we have everything to start a new BIDS directory using our data.
To do that, we can use
write_raw_bids() tries to extract as much
meta data as possible from the raw data and then formats it in a BIDS
write_raw_bids() takes a bunch of inputs, most of
which are however optional. The required inputs are:
… as you can see in the docstring:
Save raw data to a BIDS-compliant folder structure. .. warning:: * The original file is simply copied over if the original file format is BIDS-supported for that datatype. Otherwise, this function will convert to a BIDS-supported file format while warning the user. For EEG and iEEG data, conversion will be to BrainVision format; for MEG, conversion will be to FIFF. * ``mne-bids`` will infer the manufacturer information from the file extension. If your file format is non-standard for the manufacturer, please update the manufacturer field in the sidecars manually. Parameters ---------- raw : mne.io.Raw The raw data. It must be an instance of `mne.io.Raw` that is not already loaded from disk unless ``allow_preload`` is explicitly set to ``True``. See warning for the ``allow_preload`` parameter. bids_path : BIDSPath The file to write. The `mne_bids.BIDSPath` instance passed here **must** have the ``subject``, ``task``, and ``root`` attributes set. If the ``datatype`` attribute is not set, it will be inferred from the recording data type found in ``raw``. In case of multiple data types, the ``.datatype`` attribute must be set. Example:: bids_path = BIDSPath(subject='01', session='01', task='testing', acquisition='01', run='01', datatype='meg', root='/data/BIDS') This will write the following files in the correct subfolder ``root``:: sub-01_ses-01_task-testing_acq-01_run-01_meg.fif sub-01_ses-01_task-testing_acq-01_run-01_meg.json sub-01_ses-01_task-testing_acq-01_run-01_channels.tsv sub-01_ses-01_acq-01_coordsystem.json and the following one if ``events`` is not ``None``:: sub-01_ses-01_task-testing_acq-01_run-01_events.tsv and add a line to the following files:: participants.tsv scans.tsv Note that the extension is automatically inferred from the raw object. events : path-like | np.ndarray | None Use this parameter to specify events to write to the ``*_events.tsv`` sidecar file, additionally to the object's :class:`~mne.Annotations` (which are always written). If ``path-like``, specifies the location of an MNE events file. If an array, the MNE events array (shape: ``(n_events, 3)``). If a path or an array and ``raw.annotations`` exist, the union of ``events`` and ``raw.annotations`` will be written. Mappings from event names to event codes (listed in the third column of the MNE events array) must be specified via the ``event_id`` parameter; otherwise, an exception is raised. If :class:`~mne.Annotations` are present, their descriptions must be included in ``event_id`` as well. If ``None``, events will only be inferred from the raw object's :class:`~mne.Annotations`. .. note:: If specified, writes the union of ``events`` and ``raw.annotations``. If you wish to **only** write ``raw.annotations``, pass ``events=None``. If you want to **exclude** the events in ``raw.annotations`` from being written, call ``raw.set_annotations(None)`` before invoking this function. .. note:: Descriptions of all event codes must be specified via the ``event_id`` parameter. event_id : dict | None Descriptions or names describing the event codes, if you passed ``events``. The descriptions will be written to the ``trial_type`` column in ``*_events.tsv``. The dictionary keys correspond to the event description,s and the values to the event codes. You must specify a description for all event codes appearing in ``events``. If your data contains :class:`~mne.Annotations`, you can use this parameter to assign event codes to each unique annotation description (mapping from description to event code). anonymize : dict | None If `None` (default), no anonymization is performed. If a dictionary, data will be anonymized depending on the dictionary keys: ``daysback`` is a required key, ``keep_his`` is optional. ``daysback`` : int Number of days by which to move back the recording date in time. In studies with multiple subjects the relative recording date differences between subjects can be kept by using the same number of ``daysback`` for all subject anonymizations. ``daysback`` should be great enough to shift the date prior to 1925 to conform with BIDS anonymization rules. ``keep_his`` : bool If ``False`` (default), all subject information next to the recording date will be overwritten as well. If ``True``, keep subject information apart from the recording date. ``keep_source`` : bool Whether to store the name of the ``raw`` input file in the ``source`` column of ``scans.tsv``. By default, this information is not stored. format : 'auto' | 'BrainVision' | 'EDF' | 'FIF' | 'EEGLAB' Controls the file format of the data after BIDS conversion. If ``'auto'``, MNE-BIDS will attempt to convert the input data to BIDS without a change of the original file format. A conversion to a different file format will then only take place if the original file format lacks some necessary features. Conversion may be forced to BrainVision, EDF, or EEGLAB for (i)EEG, and to FIF for MEG data. symlink : bool Instead of copying the source files, only create symbolic links to preserve storage space. This is only allowed when not anonymizing the data (i.e., ``anonymize`` must be ``None``). .. note:: Symlinks currently only work with FIFF files. In case of split files, only a link to the first file will be created, and :func:`mne_bids.read_raw_bids` will correctly handle reading the data again. .. note:: Symlinks are currently only supported on macOS and Linux. We will add support for Windows 10 at a later time. empty_room : mne.io.Raw | BIDSPath | None The empty-room recording to be associated with this file. This is only supported for MEG data. If :class:`~mne.io.Raw`, you may pass raw data that was not preloaded (otherwise, pass ``allow_preload=True``); i.e., it behaves similar to the ``raw`` parameter. The session name will be automatically generated from the raw object's ``info['meas_date']``. If a :class:`~mne_bids.BIDSPath`, the ``root`` attribute must be the same as in ``bids_path``. Pass ``None`` (default) if you do not wish to specify an associated empty-room recording. .. versionchanged:: 0.11 Accepts :class:`~mne.io.Raw` data. allow_preload : bool If ``True``, allow writing of preloaded raw objects (i.e., ``raw.preload`` is ``True``). Because the original file is ignored, you must specify what ``format`` to write (not ``auto``). .. warning:: BIDS was originally designed for unprocessed or minimally processed data. For this reason, by default, we prevent writing of preloaded data that may have been modified. Only use this option when absolutely necessary: for example, manually converting from file formats not supported by MNE or writing preprocessed derivatives. Be aware that these use cases are not fully supported. montage : mne.channels.DigMontage | None The montage with channel positions if channel position data are to be stored in a format other than "head" (the internal MNE coordinate frame that the data in ``raw`` is stored in). acpc_aligned : bool It is difficult to check whether the T1 scan is ACPC aligned which means that "mri" coordinate space is "ACPC" BIDS coordinate space. So, this flag is required to be True when the digitization data is in "mri" for intracranial data to confirm that the T1 is ACPC-aligned. overwrite : bool Whether to overwrite existing files or data in files. Defaults to ``False``. If ``True``, any existing files with the same BIDS parameters will be overwritten with the exception of the ``*_participants.tsv`` and ``*_scans.tsv`` files. For these files, parts of pre-existing data that match the current data will be replaced. For ``*_participants.tsv``, specifically, age, sex and hand fields will be overwritten, while any manually added fields in ``participants.json`` and ``participants.tsv`` by a user will be retained. If ``False``, no existing data will be overwritten or replaced. events_data .. deprecated:: 0.11 Use ``events`` instead. verbose : bool | str | int | None Control verbosity of the logging output. If ``None``, use the default verbosity level. See the :ref:`logging documentation <tut-logging>` and :func:`mne.verbose` for details. Should only be passed as a keyword argument. Returns ------- bids_path : BIDSPath The path of the created data file. .. note:: If you passed empty-room raw data via ``empty_room``, the :class:`~mne_bids.BIDSPath` of the empty-room recording can be retrieved via ``bids_path.find_empty_room(use_sidecar_only=True)``. Notes ----- You should ensure that ``raw.info['subject_info']`` and ``raw.info['meas_date']`` are set to proper (not-``None``) values to allow for the correct computation of each participant's age when creating ``*_participants.tsv``. This function will convert existing `mne.Annotations` from ``raw.annotations`` to events. Additionally, any events supplied via ``events`` will be written too. To avoid writing of annotations, remove them from the raw file via ``raw.set_annotations(None)`` before invoking ``write_raw_bids``. To write events encoded in a ``STIM`` channel, you first need to create the events array manually and pass it to this function: .. events = mne.find_events(raw, min_duration=0.002) write_raw_bids(..., events=events) See the documentation of :func:`mne.find_events` for more information on event extraction from ``STIM`` channels. When anonymizing ``.edf`` files, then the file format for EDF limits how far back we can set the recording date. Therefore, all anonymized EDF datasets will have an internal recording date of ``01-01-1985``, and the actual recording date will be stored in the ``scans.tsv`` file's ``acq_time`` column. ``write_raw_bids`` will generate a ``dataset_description.json`` file if it does not already exist. Minimal metadata will be written there. If one sets ``overwrite`` to ``True`` here, it will not overwrite an existing ``dataset_description.json`` file. If you need to add more data there, or overwrite it, then you should call :func:`mne_bids.make_dataset_description` directly. When writing EDF or BDF files, all file extensions are forced to be lower-case, in compliance with the BIDS specification. See Also -------- mne.io.Raw.anonymize mne.find_events mne.Annotations mne.events_from_annotations
S001R02.edf, which corresponds to subject 1 in the second run.
In the second run of the experiment, the task was to rest with closed eyes.
To ensure the output path doesn’t contain any leftover files from previous tests and example runs, we simply delete it.
Do not delete directories that may contain important data!
The data contains annotations; which will be converted to events automatically by MNE-BIDS when writing the BIDS data:
<Annotations | 1 segment: T0 (1)>
Finally, let’s write the BIDS data!
Extracting EDF parameters from C:\Users\stefan\mne_data\MNE-eegbci-data\files\eegmmidb\1.0.0\S001\S001R02.edf... EDF file detected Setting channel info structure... Creating raw.info structure... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\README'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\participants.tsv'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\participants.json'... Writing 'C:/Users/stefan/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_space-CapTrak_electrodes.tsv'... Writing 'C:/Users/stefan/mne_data/eegmmidb_bids_eeg_example/sub-001/eeg/sub-001_space-CapTrak_coordsystem.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['T0'] Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\eeg\sub-001_task-RestEyesClosed_events.tsv'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\eeg\sub-001_task-RestEyesClosed_events.json'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\dataset_description.json'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\eeg\sub-001_task-RestEyesClosed_eeg.json'... Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\eeg\sub-001_task-RestEyesClosed_channels.tsv'... Copying data files to sub-001_task-RestEyesClosed_eeg.edf Writing 'C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\sub-001_scans.tsv'... Wrote C:\Users\stefan\mne_data\eegmmidb_bids_eeg_example\sub-001\sub-001_scans.tsv entry with eeg\sub-001_task-RestEyesClosed_eeg.edf. BIDSPath( root: C:/Users/stefan/mne_data/eegmmidb_bids_eeg_example datatype: eeg basename: sub-001_task-RestEyesClosed_eeg.edf)
What does our fresh BIDS directory look like?
|eegmmidb_bids_eeg_example\ |--- README |--- dataset_description.json |--- participants.json |--- participants.tsv |--- sub-001\ |------ sub-001_scans.tsv |------ eeg\ |--------- sub-001_space-CapTrak_coordsystem.json |--------- sub-001_space-CapTrak_electrodes.tsv |--------- sub-001_task-RestEyesClosed_channels.tsv |--------- sub-001_task-RestEyesClosed_eeg.edf |--------- sub-001_task-RestEyesClosed_eeg.json |--------- sub-001_task-RestEyesClosed_events.json |--------- sub-001_task-RestEyesClosed_events.tsv
Finally let’s get an overview of the events on the whole dataset
We can see that MNE-BIDS wrote several important files related to subject 1 for us:
electrodes.tsvcontaining the electrode coordinates and
coordsystem.json, which contains the metadata about the electrode coordinates.
The actual EDF data file (now with a proper BIDS name) and an accompanying
*_eeg.jsonfile that contains metadata about the EEG recording.
*scans.jsonfile lists all data recordings with their acquisition date. This file becomes more handy once there are multiple sessions and recordings to keep track of.
events.tsvwhich contain even further metadata.
Next to the subject specific files, MNE-BIDS also created several experiment specific files. However, we will not go into detail for them in this example.
After a lot of work was done by MNE-BIDS, it’s fair to cite the software when preparing a manuscript and/or a dataset publication.
We can see that the appropriate citations are already written in the
If you are preparing a manuscript, please make sure to also cite MNE-BIDS there.
References ---------- Appelhoff, S., Sanderson, M., Brooks, T., Vliet, M., Quentin, R., Holdgraf, C., Chaumon, M., Mikulan, E., Tavabi, K., Höchenberger, R., Welke, D., Brunner, C., Rockhill, A., Larson, E., Gramfort, A. and Jas, M. (2019). MNE-BIDS: Organizing electrophysiological data into the BIDS format and facilitating their analysis. Journal of Open Source Software 4: (1896). https://doi.org/10.21105/joss.01896 Pernet, C. R., Appelhoff, S., Gorgolewski, K. J., Flandin, G., Phillips, C., Delorme, A., Oostenveld, R. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. Scientific Data, 6, 103. https://doi.org/10.1038/s41597-019-0104-8
Now it’s time to manually check the BIDS directory and the meta files to add all the information that MNE-BIDS could not infer. For instance, you must describe EEGReference and EEGGround yourself. It’s easy to find these by searching for “n/a” in the sidecar files.
Web version: https://bids-standard.github.io/bids-validator/
Command line tool: https://www.npmjs.com/package/bids-validator
Total running time of the script: (0 minutes 0.990 seconds)