Note
Go to the end to download the full example code.
Updating BIDS datasets#
When working with electrophysiological data in the BIDS format, we usually
do not have all the metadata stored in the Raw mne-python object.
We can update the BIDS sidecar files via the update_sidecar_json function.
In this tutorial, we show how update_sidecar_json can be used to update and
modify BIDS-formatted data.
# Authors: The MNE-BIDS developers
# SPDX-License-Identifier: BSD-3-Clause
Imports#
We are importing everything we need for this example:
from mne.datasets import somato
from mne_bids import (
find_matching_paths,
make_report,
print_dir_tree,
read_raw_bids,
update_sidecar_json,
)
We will be using the MNE somato data, which is already stored in BIDS format. For more information, you can check out the respective example.
Download the somato BIDS dataset#
Download the data if it hasn’t been downloaded already, and return the path to the download directory. This directory is the so-called root of this BIDS dataset.
Using default location ~/mne_data for somato...
Fetching 1 file for the somato dataset ...
0%| | 0.00/610M [00:00<?, ?B/s]
0%|▏ | 2.42M/610M [00:00<00:25, 24.2MB/s]
2%|▋ | 10.7M/610M [00:00<00:10, 58.7MB/s]
3%|█ | 16.6M/610M [00:00<00:13, 42.5MB/s]
4%|█▌ | 25.4M/610M [00:00<00:10, 56.8MB/s]
6%|██▏ | 34.3M/610M [00:00<00:08, 66.6MB/s]
7%|██▌ | 41.4M/610M [00:00<00:08, 64.2MB/s]
8%|███ | 49.7M/610M [00:00<00:08, 69.5MB/s]
10%|███▋ | 58.7M/610M [00:00<00:07, 75.6MB/s]
11%|████▏ | 66.5M/610M [00:01<00:08, 61.3MB/s]
12%|████▌ | 73.8M/610M [00:01<00:08, 64.2MB/s]
13%|█████ | 81.2M/610M [00:01<00:07, 66.9MB/s]
15%|█████▌ | 90.3M/610M [00:01<00:07, 73.6MB/s]
16%|██████▏ | 99.6M/610M [00:01<00:06, 79.0MB/s]
18%|██████▉ | 108M/610M [00:01<00:06, 80.0MB/s]
19%|███████▍ | 117M/610M [00:01<00:06, 82.2MB/s]
21%|████████ | 126M/610M [00:01<00:05, 85.1MB/s]
22%|████████▌ | 135M/610M [00:01<00:05, 86.5MB/s]
24%|█████████▏ | 144M/610M [00:02<00:05, 82.6MB/s]
25%|█████████▊ | 153M/610M [00:02<00:05, 85.3MB/s]
27%|██████████▎ | 162M/610M [00:02<00:05, 87.3MB/s]
28%|██████████▉ | 171M/610M [00:02<00:04, 89.0MB/s]
30%|███████████▌ | 181M/610M [00:02<00:04, 89.9MB/s]
31%|████████████▏ | 190M/610M [00:02<00:04, 90.7MB/s]
33%|████████████▋ | 199M/610M [00:02<00:04, 91.3MB/s]
34%|█████████████▎ | 208M/610M [00:02<00:04, 91.7MB/s]
36%|█████████████▉ | 218M/610M [00:02<00:04, 91.8MB/s]
37%|██████████████▍ | 227M/610M [00:03<00:05, 66.9MB/s]
39%|███████████████ | 236M/610M [00:03<00:05, 71.3MB/s]
40%|███████████████▌ | 244M/610M [00:03<00:04, 73.4MB/s]
41%|████████████████ | 252M/610M [00:03<00:05, 68.7MB/s]
43%|████████████████▋ | 261M/610M [00:03<00:04, 73.6MB/s]
44%|█████████████████▏ | 270M/610M [00:03<00:04, 78.5MB/s]
46%|█████████████████▊ | 279M/610M [00:03<00:04, 82.2MB/s]
47%|██████████████████▍ | 288M/610M [00:03<00:03, 85.1MB/s]
49%|███████████████████ | 298M/610M [00:03<00:03, 87.2MB/s]
50%|███████████████████▌ | 306M/610M [00:03<00:03, 87.1MB/s]
52%|████████████████████▏ | 315M/610M [00:04<00:04, 72.3MB/s]
53%|████████████████████▋ | 324M/610M [00:04<00:03, 77.2MB/s]
54%|█████████████████████▎ | 332M/610M [00:04<00:03, 77.3MB/s]
56%|█████████████████████▊ | 340M/610M [00:04<00:03, 77.9MB/s]
57%|██████████████████████▎ | 349M/610M [00:04<00:03, 78.7MB/s]
58%|██████████████████████▊ | 357M/610M [00:04<00:03, 68.0MB/s]
60%|███████████████████████▍ | 366M/610M [00:04<00:03, 74.2MB/s]
61%|███████████████████████▉ | 374M/610M [00:04<00:03, 76.7MB/s]
63%|████████████████████████▌ | 383M/610M [00:05<00:02, 80.8MB/s]
64%|█████████████████████████ | 392M/610M [00:05<00:03, 68.2MB/s]
65%|█████████████████████████▌ | 399M/610M [00:05<00:02, 71.0MB/s]
67%|██████████████████████████ | 408M/610M [00:05<00:02, 74.2MB/s]
68%|██████████████████████████▋ | 417M/610M [00:05<00:02, 79.2MB/s]
70%|███████████████████████████▏ | 426M/610M [00:05<00:02, 82.9MB/s]
71%|███████████████████████████▊ | 435M/610M [00:05<00:02, 85.5MB/s]
73%|████████████████████████████▍ | 445M/610M [00:05<00:01, 87.6MB/s]
74%|████████████████████████████▉ | 454M/610M [00:05<00:02, 73.8MB/s]
76%|█████████████████████████████▌ | 463M/610M [00:06<00:01, 78.4MB/s]
77%|██████████████████████████████ | 471M/610M [00:06<00:02, 69.1MB/s]
79%|██████████████████████████████▋ | 480M/610M [00:06<00:01, 74.5MB/s]
80%|███████████████████████████████▎ | 489M/610M [00:06<00:01, 78.9MB/s]
82%|███████████████████████████████▊ | 498M/610M [00:06<00:01, 82.1MB/s]
83%|████████████████████████████████▍ | 507M/610M [00:06<00:01, 84.8MB/s]
85%|█████████████████████████████████ | 517M/610M [00:06<00:01, 86.7MB/s]
86%|█████████████████████████████████▌ | 526M/610M [00:06<00:00, 88.2MB/s]
88%|██████████████████████████████████▏ | 535M/610M [00:06<00:00, 89.0MB/s]
89%|██████████████████████████████████▊ | 544M/610M [00:07<00:00, 90.0MB/s]
91%|███████████████████████████████████▎ | 553M/610M [00:07<00:00, 90.9MB/s]
92%|███████████████████████████████████▉ | 563M/610M [00:07<00:00, 91.5MB/s]
94%|████████████████████████████████████▌ | 572M/610M [00:07<00:00, 89.8MB/s]
95%|█████████████████████████████████████▏ | 581M/610M [00:07<00:00, 87.8MB/s]
97%|█████████████████████████████████████▋ | 590M/610M [00:07<00:00, 83.2MB/s]
98%|██████████████████████████████████████▎| 599M/610M [00:07<00:00, 85.6MB/s]
100%|██████████████████████████████████████▊| 608M/610M [00:07<00:00, 87.5MB/s]
0%| | 0.00/610M [00:00<?, ?B/s]
100%|███████████████████████████████████████| 610M/610M [00:00<00:00, 2.53TB/s]
Download complete in 18s (581.8 MB)
Explore the dataset contents#
We can use MNE-BIDS to print a tree of all
included files and folders. We pass the max_depth parameter to
mne_bids.print_dir_tree() to the output to three levels of folders, for
better readability in this example.
print_dir_tree(bids_root, max_depth=3)
# We can generate a report of the existing dataset
print(make_report(bids_root))
|MNE-somato-data/
|--- CHANGES
|--- README
|--- dataset_description.json
|--- participants.json
|--- participants.tsv
|--- code/
|------ README
|------ convert_somato_data.py
|--- derivatives/
|------ freesurfer/
|--------- subjects/
|------ sub-01/
|--------- sub-01_task-somato-fwd.fif
|--- sub-01/
|------ sub-01_scans.tsv
|------ anat/
|--------- sub-01_T1w.json
|--------- sub-01_T1w.nii.gz
|------ meg/
|--------- sub-01_coordsystem.json
|--------- sub-01_task-somato_channels.tsv
|--------- sub-01_task-somato_events.tsv
|--------- sub-01_task-somato_meg.fif
|--------- sub-01_task-somato_meg.json
Summarizing participants.tsv /home/circleci/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/circleci/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (Elekta) sampled at
300.31 Hz with line noise at 50 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
Update the sidecar JSON dataset contents#
We can use MNE-BIDS to update all sidecar files for a matching
BIDSPath object. We then pass in a dictionary (or JSON file) to update
all matching metadata fields within the BIDS dataset.
# Search for all matching BIDSPaths in the root directory
bids_root = somato.data_path()
suffix = "meg"
extension = ".fif"
bids_paths = find_matching_paths(bids_root, suffixes=suffix, extensions=extension)
# We can now retrieve a list of all MEG-related files in the dataset:
print(bids_paths)
# Define a sidecar update as a dictionary
entries = {
"PowerLineFrequency": 60,
"Manufacturer": "MEGIN",
"InstitutionName": "Martinos Center",
}
# Note: ``update_sidecar_json`` will perform essentially a
# dictionary update to your sidecar json file, so be absolutely sure
# that the ``entries`` are defined with the proper fields specified
# by BIDS. For example, if you are updating the ``coordsystem.json``
# file, then you don't want to include ``PowerLineFrequency`` in
# ``entries``.
#
# Now update all sidecar fields according to our updating dictionary
bids_path = bids_paths[0]
sidecar_path = bids_path.copy().update(extension=".json")
update_sidecar_json(bids_path=sidecar_path, entries=entries)
[BIDSPath(
root: /home/circleci/mne_data/MNE-somato-data
datatype: meg
basename: sub-01_task-somato_meg.fif)]
Writing '/home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.json'...
Read the updated dataset#
# new line frequency is now 60 Hz
raw = read_raw_bids(bids_path=bids_path)
print(raw.info["line_freq"])
Opening raw data file /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.fif...
Range : 237600 ... 506999 = 791.189 ... 1688.266 secs
Ready.
Reading events from /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_events.tsv.
Reading channel info from /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_channels.tsv.
Not fully anonymizing info - keeping hand, his_id, sex of subject_info
60.0
Generate a new report based on the updated metadata.
# The manufacturer was changed to ``MEGIN``
print(make_report(bids_root))
Summarizing participants.tsv /home/circleci/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/circleci/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (MEGIN) sampled at
300.31 Hz with line noise at 60 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
We can revert the changes by updating the sidecar again.
# update the sidecar data to have a new PowerLineFrequency
entries["Manufacturer"] = "Elekta"
entries["PowerLineFrequency"] = 50
update_sidecar_json(bids_path=sidecar_path, entries=entries)
Writing '/home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.json'...
Now let us inspect the dataset again by generating the report again. Now that
update_sidecar_json was called, the metadata will be updated.
# The power line frequency should now change back to 50 Hz
raw = read_raw_bids(bids_path=bids_path)
print(raw.info["line_freq"])
# Generate the report with updated fields
print(make_report(bids_root))
Opening raw data file /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.fif...
Range : 237600 ... 506999 = 791.189 ... 1688.266 secs
Ready.
Reading events from /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_events.tsv.
Reading channel info from /home/circleci/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_channels.tsv.
Not fully anonymizing info - keeping hand, his_id, sex of subject_info
50.0
Summarizing participants.tsv /home/circleci/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/circleci/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (Elekta) sampled at
300.31 Hz with line noise at 50 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
Total running time of the script: (0 minutes 18.223 seconds)