Note
Go to the end to download the full example code.
Updating BIDS datasets#
When working with electrophysiological data in the BIDS format, we usually
do not have all the metadata stored in the Raw mne-python object.
We can update the BIDS sidecar files via the update_sidecar_json function.
In this tutorial, we show how update_sidecar_json can be used to update and
modify BIDS-formatted data.
# Authors: The MNE-BIDS developers
# SPDX-License-Identifier: BSD-3-Clause
Imports#
We are importing everything we need for this example:
from mne.datasets import somato
from mne_bids import (
find_matching_paths,
make_report,
print_dir_tree,
read_raw_bids,
update_sidecar_json,
)
We will be using the MNE somato data, which is already stored in BIDS format. For more information, you can check out the respective example.
Download the somato BIDS dataset#
Download the data if it hasn’t been downloaded already, and return the path to the download directory. This directory is the so-called root of this BIDS dataset.
Using default location ~/mne_data for somato...
0%| | 0.00/611M [00:00<?, ?B/s]
1%|▎ | 5.85M/611M [00:00<00:10, 58.5MB/s]
2%|▊ | 13.0M/611M [00:00<00:09, 66.0MB/s]
3%|█▎ | 20.1M/611M [00:00<00:08, 68.5MB/s]
4%|█▋ | 27.3M/611M [00:00<00:08, 69.7MB/s]
6%|██▏ | 34.4M/611M [00:00<00:08, 70.4MB/s]
7%|██▌ | 41.6M/611M [00:00<00:08, 70.7MB/s]
8%|███ | 48.7M/611M [00:00<00:07, 70.8MB/s]
9%|███▍ | 55.8M/611M [00:00<00:07, 70.9MB/s]
10%|███▉ | 62.9M/611M [00:00<00:07, 70.9MB/s]
11%|████▎ | 70.0M/611M [00:01<00:07, 70.9MB/s]
13%|████▊ | 77.1M/611M [00:01<00:07, 71.0MB/s]
14%|█████▏ | 84.2M/611M [00:01<00:07, 71.0MB/s]
15%|█████▋ | 91.4M/611M [00:01<00:07, 71.2MB/s]
16%|██████▏ | 98.5M/611M [00:01<00:07, 71.2MB/s]
17%|██████▊ | 106M/611M [00:01<00:07, 71.3MB/s]
18%|███████▏ | 113M/611M [00:01<00:08, 59.8MB/s]
20%|███████▋ | 120M/611M [00:01<00:07, 62.6MB/s]
21%|████████ | 127M/611M [00:01<00:07, 64.9MB/s]
22%|████████▌ | 134M/611M [00:01<00:07, 66.6MB/s]
23%|█████████ | 141M/611M [00:02<00:06, 67.9MB/s]
24%|█████████▍ | 148M/611M [00:02<00:06, 68.9MB/s]
25%|█████████▉ | 155M/611M [00:02<00:06, 69.6MB/s]
27%|██████████▍ | 163M/611M [00:02<00:06, 70.1MB/s]
28%|██████████▊ | 170M/611M [00:02<00:06, 70.4MB/s]
29%|███████████▎ | 177M/611M [00:02<00:06, 70.6MB/s]
30%|███████████▋ | 184M/611M [00:02<00:06, 70.8MB/s]
31%|████████████▏ | 191M/611M [00:02<00:05, 70.9MB/s]
32%|████████████▋ | 198M/611M [00:02<00:05, 70.8MB/s]
34%|█████████████ | 205M/611M [00:02<00:05, 70.8MB/s]
35%|█████████████▌ | 212M/611M [00:03<00:05, 70.9MB/s]
36%|██████████████ | 219M/611M [00:03<00:05, 70.8MB/s]
37%|██████████████▍ | 226M/611M [00:03<00:05, 70.8MB/s]
38%|██████████████▉ | 234M/611M [00:03<00:05, 70.9MB/s]
39%|███████████████▎ | 241M/611M [00:03<00:05, 70.9MB/s]
41%|███████████████▊ | 248M/611M [00:03<00:05, 70.8MB/s]
42%|████████████████▎ | 255M/611M [00:03<00:05, 70.9MB/s]
43%|████████████████▋ | 262M/611M [00:03<00:04, 70.9MB/s]
44%|█████████████████▏ | 269M/611M [00:03<00:04, 71.0MB/s]
45%|█████████████████▋ | 276M/611M [00:03<00:04, 70.6MB/s]
46%|██████████████████ | 283M/611M [00:04<00:04, 70.8MB/s]
48%|██████████████████▌ | 290M/611M [00:04<00:04, 70.9MB/s]
49%|███████████████████ | 297M/611M [00:04<00:04, 70.9MB/s]
50%|███████████████████▍ | 305M/611M [00:04<00:04, 71.0MB/s]
51%|███████████████████▉ | 312M/611M [00:04<00:04, 70.9MB/s]
52%|████████████████████▎ | 319M/611M [00:04<00:04, 70.9MB/s]
53%|████████████████████▊ | 326M/611M [00:04<00:04, 71.1MB/s]
55%|█████████████████████▎ | 333M/611M [00:04<00:03, 71.2MB/s]
56%|█████████████████████▋ | 340M/611M [00:04<00:03, 71.4MB/s]
57%|██████████████████████▏ | 347M/611M [00:04<00:03, 71.4MB/s]
58%|██████████████████████▋ | 355M/611M [00:05<00:03, 71.5MB/s]
59%|███████████████████████ | 362M/611M [00:05<00:03, 71.5MB/s]
60%|███████████████████████▌ | 369M/611M [00:05<00:03, 69.3MB/s]
62%|████████████████████████ | 376M/611M [00:05<00:03, 66.2MB/s]
63%|████████████████████████▍ | 383M/611M [00:05<00:03, 67.5MB/s]
64%|████████████████████████▉ | 390M/611M [00:05<00:03, 68.7MB/s]
65%|█████████████████████████▍ | 397M/611M [00:05<00:03, 69.6MB/s]
66%|█████████████████████████▊ | 404M/611M [00:05<00:02, 69.7MB/s]
67%|██████████████████████████▎ | 411M/611M [00:05<00:02, 69.9MB/s]
69%|██████████████████████████▋ | 418M/611M [00:05<00:02, 70.2MB/s]
70%|███████████████████████████▏ | 426M/611M [00:06<00:02, 70.9MB/s]
71%|███████████████████████████▋ | 433M/611M [00:06<00:02, 71.0MB/s]
72%|████████████████████████████ | 440M/611M [00:06<00:02, 71.4MB/s]
73%|████████████████████████████▌ | 447M/611M [00:06<00:02, 71.6MB/s]
74%|█████████████████████████████ | 454M/611M [00:06<00:02, 71.7MB/s]
76%|█████████████████████████████▍ | 462M/611M [00:06<00:02, 71.7MB/s]
77%|█████████████████████████████▉ | 469M/611M [00:06<00:01, 71.8MB/s]
78%|██████████████████████████████▍ | 476M/611M [00:06<00:02, 49.6MB/s]
79%|██████████████████████████████▊ | 482M/611M [00:07<00:03, 42.8MB/s]
80%|███████████████████████████████ | 487M/611M [00:07<00:03, 40.0MB/s]
80%|███████████████████████████████▍ | 491M/611M [00:07<00:03, 37.2MB/s]
81%|███████████████████████████████▊ | 497M/611M [00:07<00:02, 42.1MB/s]
82%|████████████████████████████████ | 502M/611M [00:07<00:02, 41.0MB/s]
83%|████████████████████████████████▍ | 508M/611M [00:07<00:02, 45.6MB/s]
84%|████████████████████████████████▉ | 515M/611M [00:07<00:01, 52.0MB/s]
86%|█████████████████████████████████▎ | 522M/611M [00:07<00:01, 57.2MB/s]
87%|█████████████████████████████████▊ | 529M/611M [00:08<00:01, 61.4MB/s]
88%|██████████████████████████████████▏ | 536M/611M [00:08<00:01, 54.4MB/s]
89%|██████████████████████████████████▌ | 542M/611M [00:08<00:01, 52.3MB/s]
90%|██████████████████████████████████▉ | 547M/611M [00:08<00:01, 53.1MB/s]
91%|███████████████████████████████████▎ | 553M/611M [00:08<00:01, 53.6MB/s]
92%|███████████████████████████████████▋ | 560M/611M [00:08<00:00, 58.2MB/s]
93%|████████████████████████████████████▏ | 566M/611M [00:08<00:00, 61.3MB/s]
94%|████████████████████████████████████▋ | 573M/611M [00:08<00:00, 63.8MB/s]
95%|█████████████████████████████████████ | 581M/611M [00:08<00:00, 65.9MB/s]
96%|█████████████████████████████████████▌ | 588M/611M [00:09<00:00, 67.5MB/s]
97%|██████████████████████████████████████ | 595M/611M [00:09<00:00, 68.8MB/s]
99%|██████████████████████████████████████▍| 602M/611M [00:09<00:00, 69.5MB/s]
100%|██████████████████████████████████████▉| 609M/611M [00:09<00:00, 69.9MB/s]
0%| | 0.00/611M [00:00<?, ?B/s]
100%|███████████████████████████████████████| 611M/611M [00:00<00:00, 3.47TB/s]
Download complete in 21s (582.2 MB)
Explore the dataset contents#
We can use MNE-BIDS to print a tree of all
included files and folders. We pass the max_depth parameter to
mne_bids.print_dir_tree() to the output to three levels of folders, for
better readability in this example.
print_dir_tree(bids_root, max_depth=3)
# We can generate a report of the existing dataset
print(make_report(bids_root))
|MNE-somato-data/
|--- CHANGES
|--- README
|--- dataset_description.json
|--- participants.json
|--- participants.tsv
|--- code/
|------ README
|------ convert_somato_data.py
|--- derivatives/
|------ freesurfer/
|--------- subjects/
|------ sub-01/
|--------- sub-01_task-somato-fwd.fif
|--- sub-01/
|------ sub-01_scans.tsv
|------ anat/
|--------- sub-01_T1w.json
|--------- sub-01_T1w.nii.gz
|------ meg/
|--------- sub-01_coordsystem.json
|--------- sub-01_task-somato_channels.tsv
|--------- sub-01_task-somato_events.tsv
|--------- sub-01_task-somato_meg.fif
|--------- sub-01_task-somato_meg.json
Summarizing participants.tsv /home/runner/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/runner/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (Elekta) sampled at
300.31 Hz with line noise at 50 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
Update the sidecar JSON dataset contents#
We can use MNE-BIDS to update all sidecar files for a matching
BIDSPath object. We then pass in a dictionary (or JSON file) to update
all matching metadata fields within the BIDS dataset.
# Search for all matching BIDSPaths in the root directory
bids_root = somato.data_path()
suffix = "meg"
extension = ".fif"
bids_paths = find_matching_paths(bids_root, suffixes=suffix, extensions=extension)
# We can now retrieve a list of all MEG-related files in the dataset:
print(bids_paths)
# Define a sidecar update as a dictionary
entries = {
"PowerLineFrequency": 60,
"Manufacturer": "MEGIN",
"InstitutionName": "Martinos Center",
}
# Note: ``update_sidecar_json`` will perform essentially a
# dictionary update to your sidecar json file, so be absolutely sure
# that the ``entries`` are defined with the proper fields specified
# by BIDS. For example, if you are updating the ``coordsystem.json``
# file, then you don't want to include ``PowerLineFrequency`` in
# ``entries``.
#
# Now update all sidecar fields according to our updating dictionary
bids_path = bids_paths[0]
sidecar_path = bids_path.copy().update(extension=".json")
update_sidecar_json(bids_path=sidecar_path, entries=entries)
[BIDSPath(
root: /home/runner/mne_data/MNE-somato-data
datatype: meg
basename: sub-01_task-somato_meg.fif)]
Writing '/home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.json'...
Read the updated dataset#
# new line frequency is now 60 Hz
raw = read_raw_bids(bids_path=bids_path)
print(raw.info["line_freq"])
Opening raw data file /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.fif...
Range : 237600 ... 506999 = 791.189 ... 1688.266 secs
Ready.
Reading events from /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_events.tsv.
Reading channel info from /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_channels.tsv.
Not fully anonymizing info - keeping his_id, sex, and hand info
60.0
Generate a new report based on the updated metadata.
# The manufacturer was changed to ``MEGIN``
print(make_report(bids_root))
Summarizing participants.tsv /home/runner/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/runner/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (MEGIN) sampled at
300.31 Hz with line noise at 60 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
We can revert the changes by updating the sidecar again.
# update the sidecar data to have a new PowerLineFrequency
entries["Manufacturer"] = "Elekta"
entries["PowerLineFrequency"] = 50
update_sidecar_json(bids_path=sidecar_path, entries=entries)
Writing '/home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.json'...
Now let us inspect the dataset again by generating the report again. Now that
update_sidecar_json was called, the metadata will be updated.
# The power line frequency should now change back to 50 Hz
raw = read_raw_bids(bids_path=bids_path)
print(raw.info["line_freq"])
# Generate the report with updated fields
print(make_report(bids_root))
Opening raw data file /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_meg.fif...
Range : 237600 ... 506999 = 791.189 ... 1688.266 secs
Ready.
Reading events from /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_events.tsv.
Reading channel info from /home/runner/mne_data/MNE-somato-data/sub-01/meg/sub-01_task-somato_channels.tsv.
Not fully anonymizing info - keeping his_id, sex, and hand info
50.0
Summarizing participants.tsv /home/runner/mne_data/MNE-somato-data/participants.tsv...
Summarizing scans.tsv files [PosixPath('/home/runner/mne_data/MNE-somato-data/sub-01/sub-01_scans.tsv')]...
The participant template found: comprised of 1 male and 0 female participants;
handedness were all unknown;
ages all unknown
The MNE-somato-data-bids dataset was created by Lauri Parkkonen and conforms to
BIDS version 1.2.0. This report was generated with MNE-BIDS
(https://doi.org/10.21105/joss.01896). The dataset consists of 1 participants
(comprised of 1 male and 0 female participants; handedness were all unknown;
ages all unknown) . Data was recorded using an MEG system (Elekta) sampled at
300.31 Hz with line noise at 50 Hz. There was 1 scan in total. Recording
durations ranged from 897.08 to 897.08 seconds (mean = 897.08, std = 0.0), for a
total of 897.08 seconds of data recorded over all scans. For each dataset, there
were on average 316.0 (std = 0.0) recording channels per scan, out of which
316.0 (std = 0.0) were used in analysis (0.0 +/- 0.0 were removed from
analysis).
Total running time of the script: (0 minutes 21.913 seconds)