Skip to content

What's new

v1.9.0

🆕 New features & enhancements

⚠ Behavior changes

  • All ICA HTML reports have been consolidated in the standard subject *_report.html file instead of producing separate files. (#899 by @larsoner)
  • Changed default for source_info_path_update to None. In _04_make_forward.py and _05_make_inverse.py, we retrieve the info from the file from which the noise_cov is computed. (#919 by @SophieHerbst)
  • The depth parameter doesn't accept None anymore. Please use 0 instead. (#915 by @hoechenberger)
  • When using automated bad channel detection, now indicate the generated *_bads.tsv files whether a channel had previously already been marked as bad in the dataset. Resulting entries in the TSV file may now look like: "pre-existing (before MNE-BIDS-pipeline was run) & auto-noisy" (previously: only "auto-noisy"). (#930 by @hoechenberger)
  • The ica_ctps_ecg_threshold has been renamed to ica_ecg_threshold. (#935 by @hoechenberger)
  • We changed the behavior when setting an EEG montage:
  • When applying the montage, we now also check for channel aliases (e.g. M1 -> TP9).
  • If the data contains a channel that is not present in the montage, we now abort with an exception (previously, we emitted a warning). This is to prevent silent errors. To proceed in this situation, select a different montage, or drop the respective channels via the drop_channels configuration option. (#960 by @hoechenberger)

📦 Requirements

  • The minimum required version of MNE-Python is now 1.7.0.
  • We dropped support for Python 3.9. You now need Python 3.10 or newer. (#908 by @hoechenberger)

📖 Documentation

  • We removed the Execution section from configuration options documentation and replaced it with new, more explicit sections (namely, Caching, Parallelization, Logging, and Error handling), and enhanced documentation. (#914 by @hoechenberger, #916 by @SophieHerbst)

🐛 Bug fixes

  • When running the pipeline with [find_bad_channels_meg][mne_bids_pipeline._config. find_bad_channels_meg] enabled, then disabling it and running the pipeline again, the pipeline would incorrectly still use automatically detected bad channels from the first pipeline run. Now, we ensure that the original bad channels would be used and the related section is removed from the report in this case. (#902 by @larsoner)
  • Fixed group-average decoding statistics were not updated in some cases, even if relevant configuration options had been changed. (#902 by @larsoner)
  • Fixed a compatibility bug with joblib 1.4.0. (#899 by @larsoner)
  • Fixed how "original" raw data is included in the report. Previously, bad channels, subject, and experimenter name would not be displayed correctly. (#930 by @hoechenberger)
  • In the report's table of contents, don't put the run numbers in quotation marks. (#933 by @hoechenberger)

⚕ Code health and infrastructure

  • Use GitHub's dependabot service to automatically keep GitHub Actions up-to-date. (#893 by @hoechenberger)

v1.8.0 (2024-03-20)

🆕 New features & enhancements

📦 Requirements

🐛 Bug fixes

  • Fix handling of Maxwell-filtered data in CSP (#890 by @larsoner)
  • Avoid recomputation / cache miss when the same empty-room file is matched to multiple subjects (#890 by @larsoner)

⚕ Code health

  • We removed the unused settings shortest_event and study_name. They were relics of early days of the pipeline and haven't been in use for a long time. (#888, #889 by @hoechenberger and @larsoner)

v1.7.0 (2024-03-13)

🆕 New features & enhancements

  • Improved logging message during cache invalidation: We now print the selected memory_file_method ("hash" or "mtime"). Previously, we'd always print "hash". (#876 by @hoechenberger)

🐛 Bug fixes

⚕ Code health

  • We enabled stricter linting to guarantee a consistently high code quality! (#872 by @hoechenberger)

v1.6.0 (2024-03-01)

🆕 New features & enhancements

  • Added regress_artifact to allow artifact regression (e.g., of MEG reference sensors in KIT systems) (#837 by @larsoner)
  • Chosen reject parameters are now saved in the generated HTML reports (#839 by @larsoner)
  • Added saving of clean raw data in addition to epochs (#840 by @larsoner)
  • Added saving of detected blink and cardiac events used to calculate SSP projectors (#840 by @larsoner)
  • Added noise_cov_method to allow for the use of methods other than "shrunk" for noise covariance estimation (#854 by @larsoner)
  • Added option to pass image_kwargs to [mne.Report.add_epochs] to allow adjusting e.g. "vmin" and "vmax" of the epochs image in the report via report_add_epochs_image_kwargs. This feature requires MNE-Python 1.7 or newer. (#848 by @SophieHerbst)
  • Split ICA fitting and artifact detection into separate steps. This means that now, ICA is split into a total of three consecutive steps: fitting, artifact detection, and the actual data cleaning step ("applying ICA"). This makes it easier to experiment with different settings for artifact detection without needing to re-fit ICA. (#865 by @larsoner)
  • The configuration used for the pipeline is now saved in a separate spreadsheet in the .xlsx log file (#869 by @larsoner)

📦 Requirements

🐛 Bug fixes

  • Fixed minor issues with path handling for cross-talk and calibration files (#834 by @larsoner)
  • Fixed EEG reject use for ch_types = ["meg", "eeg"] in epoch cleaning (#839 by @larsoner)
  • Fixed bug where implicit mf_reference_run could change across invocations of mne_bids_pipeline, breaking caching (#839 by @larsoner)
  • Fixed --no-cache behavior having no effect (#839 by @larsoner)
  • Fixed Maxwell filtering failures when find_noisy_channels_meg = False` is used (#847 by @larsoner)
  • Fixed raw, empty-room, and custom noise covariances calculation, previously they could errantly be calculated on data without ICA or SSP applied (#840 by @larsoner)
  • Fixed multiple channel type handling (e.g., MEG and EEG) in decoding (#853 by @larsoner)
  • Changed the default for ica_n_components from 0.8 (too conservative) to None to match MNE-Python's default (#853 by @larsoner)
  • Prevent events table for the average subject overflowing in reports (#854 by @larsoner)
  • Fixed split file behavior for Epochs when using ICA (#855 by @larsoner)
  • Fixed a bug where users could not set _components.tsv as it would be detected as a cache miss and overwritten on next pipeline run (#865 by @larsoner)

⚕ Code health

  • The package build backend has been switched from setuptools to hatchling. (#825 by @hoechenberger)
  • Removed dependencies on datalad and git-annex for testing (#867 by @larsoner)
  • Code formatting now uses ruff format instead of black (#834, #838 by @larsoner)
  • Code caching is now tested using GitHub Actions (#836 by @larsoner)
  • Steps in the documentation are now automatically parsed into flowcharts (#859 by @larsoner)
  • New configuration options are now automatically added to the docs (#863 by @larsoner)
  • Configuration options now have relevant steps listed in the docs (#866 by @larsoner)

v1.5.0 (2023-11-30)

This release contains a number of very important bug fixes that address problems related to decoding, time-frequency analysis, and inverse modeling. All users are encouraged to update.

🆕 New features & enhancements

⚠ Behavior changes

  • The default cache directory is now _cache within the derivatives folder when using memory_location=True, set memory_subdir="joblib" to get the behavior from v1.4 (#778 by @larsoner)
  • Before cleaning epochs via ICA, we used to reject any epochs execeeding the ica_reject criteria. However, this may lead to the unnecessary exclusion of epochs that could have been salvaged through ICA cleaning. Now, we only apply ica_reject to the epochs used for ICA fitting. After the experimental epochs have been cleaned with ICA (preprocessing/apply_ica step), any remaining large-amplitude artifacts can be removed via reject, which is used in the last preprocessing step, preprocessing/ptp_reject. (#806 by @hoechenberger)
  • MVPA / decoding used to be performed on un-cleaned epochs in the past. Now, cleaned epochs will be used by default (please also see the "Bug fixes" section below). (#796 by @hoechenberger)

⚕ Code health

🐛 Bug fixes

  • Fixed bug where cache would not invalidate properly based on output file changes and steps could be incorrectly skipped. All steps will automatically rerun to accommodate the new, safer caching scheme (#756 by @larsoner)
  • Fixed bug with parallelization across runs for Maxwell filtering (#761 by @larsoner)
  • Fixed bug where head position files were not written with a proper suffix and extension (#761 by @larsoner)
  • Fixed bug where default values for decoding_csp_times and decoding_csp_freqs were not set dynamically based on the config parameters (#779 by @larsoner)
  • Fixed bug where the MNE logger verbosity was not respected inside parallel jobs (#813 by @larsoner)
  • A number of processing steps erroneously always operated on un-cleaned epochs (sensor/decoding_full_epochs, sensor/decoding_time_by_time, sensor/decoding_csp); or operated on un-cleaned epochs (without PTP rejection) if no ICA or SSP was requested (sensor/ime_frequency, sensor/make_cov) The bug in sensor/make_cov could propagate to the source level, as the covariance matrix is used for inverse modeling. (#796 by @hoechenberger)
  • Bad channels may have been submitted to MVPA (full epochs decoding, time-by-time decoding, CSP-based decoding) when not using Maxwell filtering (i.e., usually only EEG data was affected). This has now been fixed and data from bad channels is omitted from decoding. (#817 by @hoechenberger)

v1.4.0 (2023-07-04)

🆕 New features & enhancements

⚕ Code health

  • Refactor code to deduplicate keyword-passing and improve internal logic (#746, #749 by @larsoner)

🐛 Bug fixes

v1.3.0 (released 2023/06/01)

🆕 New features & enhancements

  • Provide a more helpful log message if the CSP decoding step is being skipped (#734 by @hoechenberger)
  • Use rich for improved logging control and styling (#737 by @larsoner)

⚕ Code health

  • Avoid using deprecated openpyxl API when working with Excel spreadsheets (#735 by @larsoner)

🐛 Bug fixes

  • Fix pandas 2.0 compatibility (#732 by @larsoner)
  • Fix bug with mne.sys_info insertion in reports (#732 by @larsoner)
  • Always generate CSP decdoing grand average analysis if CSP decoding was used on the single-subject level (#733 by @hoechenberger)

v1.2.0 (released 2023/03/23)

🆕 New features & enhancements

  • We improved caching to reduce initialization time when large datasets are being processed. (#720 by @apmellot)

⚠ Behavior changes

  • MNE-BIDS-Pipeline now requires MNE-Python 1.2 or newer.

⚕ Code health

  • Replace legacy plot_psd() calls with compute_psd().plot() (#725 by @drammock)

v1.1.0 (released 2022/01/13)

🆕 New features & enhancements

v1.0.3 (released 2022/12/18)

⚠ Behavior changes

This backward-incompatible change was made before officially announcing the release, so we are bumping the patch version number rather than the major version number.

🐛 Bug fixes

  • The stable documentation deployment was fixed (#682 by @larsoner )

⚕ Code health

v1.0.2 (released 2022/12/07)

⚕ Code health

v1.0.1 (released 2022/12/06)

⚠ Behavior changes

These backward-incompatible changes have been made before officially announcing the release, so we are bumping the patch version number rather than the major version number.

v1.0.0 (released 2022/12/01)

Changes were only tracked starting April 15, 2021.

🆕 New features & enhancements

⚠ Behavior changes

  • The conditions setting will now be None by default. It is a required setting so it will raise an error if left as None. (#348 by @guiomar and @hoechenberger)
  • When creating epochs, only those epochs are kept that belong to the specified conditions. This means that only this subset of epochs will be passed to ICA and SSP (if used). Previously, we would created epochs based on all events found in the data, and only subset them to the requested conditions at the epoching stage. (#449 by @agramfort)
  • Epochs rejection based on peak-to-peak amplitude, as controlled via the reject setting, will now take place after ICA or SSP. In previous versions of the Pipeline, rejection was carried out before ICA and SSP. To exclude epochs from ICA fitting, use the new ica_reject setting.
  • We don't apply SSP by default anymore. (#315 by @hoechenberger)
  • The use_ssp and use_ica settings have been removed. Please use the new spatial_filter setting instead. (#315 by @hoechenberger)
  • The allow_maxshield setting has been removed. The Pipeline now automatically ensures that FIFF files of recordings with active shielding (MaxShield) can be imported. Later stages of the Pipeline will fail if Maxwell filtering of such data is disabled via use_maxwell_filter=False. (#318 by @hoechenberger)
  • The overlay plots that show the effects of ICA cleaning are now based on the baseline-corrected data to make it easier to spot the differences. (#320 by @hoechenberger)
  • bids_root and deriv_root are now converted to absolute paths to avoid running into issues caused by relative path specifications. (#322 by @hoechenberger)
  • Warn if using ICA and no EOG- or ECG-related ICs were detected. (#351 by @crsegerie)
  • Added the possibility to have different runs for different subjects. (#353 by @crsegerie))
  • Check that the baseline interval falls into [epochs_tmin, epochs_tmax]. (#361 by @crsegerie)
  • config.crop has been renamed to the more explicit config.crop_runs, as it only applies to individual runs and not the concatenated data. (#358 by @hoechenberger)
  • ica_reject now also applies to ECG and EOG epochs. (#373 by @crsegerie)
  • Added a new step freesurfer/coreg_surfaces that creates the scalp surfaces required for coregistration in MNE-Python. (#422 by @hoechenberger)
  • Enabling interactive mode by setting interactive to True now deactivates parallel processing. (#473 by @hoechenberger)
  • The resolution of the MRI slices for BEM visualalization has been reduced to 256 by 256 pixels (was 512 by 512 before), we now only plot every 8th slice (was ever 2nd before). This greatly speeds up BEM rendering and reduces the size of the generated report, while maintaining a sufficiently detailed visualization. (#488 by @hoechenberger)
  • In interactive mode, the Matplotlib Agg backend will be used. (#497 by @hoechenberger)
  • We added new configuration options to control the number of time points to use when creating topographic maps of evoked signals and brain plots for source estimates: report_evoked_n_time_points and report_stc_n_time_points, respectively. (#542 by @agramfort)
  • Add reader_extra_params parameter to pass parameters to read_raw_bids. (#585 by @agramfort)
  • Add task_is_rest parameter to be explicit that the data must be analyzed as resting state. (#585 by @agramfort)
  • Patch information is now incorporated when computing surface source spaces, which should slightly improve the surface normals (#588 by @larsoner)
  • Add ssp_meg option for MEG SSP computation. This defaults to 'auto', which will use ssp_meg='combined' for SSP computation when Maxwell filtering is used. (#595 by @larsoner)
  • Empty-room and resting-state data are processed by default if present, regardless of config.noise_cov value. This can be controlled by changing the default values from config.process_empty_room = True and config.process_rest = True (#633 by @larsoner)
  • Environment variables are no longer used to control execution and variables, use command-line switches instead (#663 by @larsoner )
  • Config validation of likely misspellings and (some) outdated variables are now checked (#665, #670 by @larsoner )

⚕ Code health

  • Each processing script now extracts the required subset of configuration options from the user config and operates solely on these. This helps make it clear which settings an individual script depends on. (#383 by @agramfort and @hoechenberger)
  • We laid the groundwork for applying the inverse operator on other data types than just mne.Evoked by introducing inverse_targets. (#452 by @apmellot)
  • All processing scripts have been renamed and turned into submodules, and the pipeline is now organized like a standard Python package with a command-line interface mne_bids_pipeline ... (#470, #611, #664 by @agramfort, @hoechenberger, and @larsoner)
  • For storing configuration values, we switched from using BunchConst to Python's built-in SimpleNamespace. (#472 by @agramfort)
  • The config.process_er variable was renamed config.process_empty_room for readability, and the default was changed to True for consistency (#633 by @larsoner)

🐛 Bug fixes

  • The FreeSurfer script could only be run if --n_jobs was passed explicitly (#287 by @MerlinDumeur)
  • Fix a problem with the FreeSurfer processing step that caused the error message Could not consume arg after completion (#301 by @hoechenberger)
  • Selecting the extended_infomax ICA algorithm caused a crash (#308 by @hoechenberger)
  • Correctly handle eog_channels = None setting after creation of bipolar EEG channels (#311 by @hoechenberger)
  • Added instructions on how to handle FileNotFoundError when loading the BEM model in the source steps (#304 by @MerlinDumeur)
  • When using find_noisy_channels_meg or find_flat_channels_meg, we now pass mf_head_origin to the respective bad channel detection algorithm to achieve better performance (#319 by @agramfort)
  • Baseline was not applied to epochs if neither ICA nor SSP was used (#319 by @hoechenberger)
  • Ensure we always use the cleaned epochs for constructing evoked data (#319 by @agramfort)
  • The summary report didn't use the cleaned epochs for showing the effects of ICA. (#320 by @hoechenberger)
  • The sanity check comparing the rank of the experimental data and the rank of the empty-room after Maxwell-filtering did not use the maxfiltered data. (#336 by @agramfort, @hoechenberger, and @crsegerie)
  • epochs_tmin and epochs_tmax were named incorrectly in some test config files. (#340 by @crsegerie)
  • We now reject bad epochs by using ica_reject before producing the "overlay" plots that show the evoked data before and after ICA cleaning in the proc-ica_report. (#385 by @crsegerie).
  • Passing subject, session, task, and run names to run.py the consist only of numbers doesn't throw an exception anymore, even if the values weren't double-quoted and hence converted to strings by the command-line interface toolkit we are using, Fire. (#375 by @hoechenberger)
  • Setting eeg_reference to a string (name of the reference channel) caused us to crash. (#391 by @hoechenberger)
  • Parameters retrieved from the BIDS dataset were not limited to the modality ("data type") we meant to process, sometimes leading to crashes. (#391 by @hoechenberger)
  • Fix forward computation for CTF data. (#427 by @agramfort)
  • Generated derivative epochs split files now follow the BIDS naming scheme. (#463)}} by @dengemann)
  • Report tags are now better sanitized. (#471 by @hoechenberger)
  • When creating epochs, we now ensure that the trigger codes provided via the *_events.tsv file are retained; previously, new trigger codes were generated in certain situations. (#471 by @hoechenberger)
  • ICA overlay plots in the report used the cleaned epochs, resulting in no visible ICA cleaning effects. This has been corrected. (#478 by @hoechenberger)
  • Ensure we don't over-estimate the rank of Maxwell-filtered data consisting of multiple runs. (#530 by @hoechenberger and @agramfort)
  • Don't drop bad channels from empty-room data on import. (#532 by @hoechenberger)
  • Time-by-time decoding now respects the decoding_n_splits setting. (#558 by @hoechenberger)
  • Time-by-time decoding now respects the random seed specified via random_state when generating the cross-validation splits. (#558 by @hoechenberger)
  • When automatically finding flat or noisy MEG channels was enabled, the data would always be 40-Hz-lowpass-filtered even if not explicitly requested by the user. This is now fixed. (#558 by @apmellot and @hoechenberger)
  • EEG channels couldn't be used as "virtual" EOG channels during ICA artifact detection. Reported by "fraenni" on the forum. Thank you! 🌻 (#572 by @hoechenberger)
  • Fix bug with handling of split files during preprocessing (#597 by @larsoner)
  • Fix bug where wrong command-line arguments to run.py were just ignored instead of raising an error (#605) by @larsoner)
  • During the Maxwell filtering step, all channels are now kept rather than being restricted to just MEG channels (#606 by @larsoner)
  • The default for analyze_channels is now 'ch_types' which restricts to the data channels of interest rather than 'all', which included all channels, and hence could include things like stimulus channels (#606 by @larsoner)
  • Fix bug where only the first run was used to compute SSP (#607 by @larsoner)
  • Fix bug with CPU oversubscription when using the Dask backend on macOS M1 (#638 by @larsoner)