Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH, MRG] Add EpochsSpectrumArray and SpectrumArray classes #11803

Merged
merged 62 commits into from
Sep 2, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
395a758
[ENH, MRG] Add EpochsSpectrumArray and SpectrumArray classes [skip ci
alexrockhill Jul 13, 2023
710cef4
update latest
alexrockhill Jul 13, 2023
e34bc38
fix refs
alexrockhill Jul 13, 2023
3fcb557
wrong versionadded
alexrockhill Jul 13, 2023
597c48c
Update mne/time_frequency/spectrum.py
alexrockhill Jul 14, 2023
6a54f68
Update mne/time_frequency/spectrum.py
alexrockhill Jul 14, 2023
125903e
epoch not epochs
alexrockhill Jul 14, 2023
e0fb07f
edit seealso [ci skip]
drammock Jul 14, 2023
8cee941
Dan review
alexrockhill Jul 14, 2023
738c8d3
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Jul 14, 2023
0cd6ec8
style
alexrockhill Jul 14, 2023
5d18ff3
Merge branch 'main' into defaults
alexrockhill Jul 14, 2023
27be8d8
cruft, test plot
alexrockhill Jul 14, 2023
145b6bb
style
alexrockhill Jul 14, 2023
71016d6
very picky style
alexrockhill Jul 14, 2023
1ccf8a4
add fixtures
alexrockhill Jul 18, 2023
ece2b39
Merge branch 'main' into defaults
alexrockhill Jul 18, 2023
ff203ff
Merge branch 'main' into defaults
alexrockhill Jul 19, 2023
725446d
Dan review
alexrockhill Jul 26, 2023
ecbd0a8
Merge branch 'main' of https://github.com/mne-tools/mne-python into d…
alexrockhill Jul 26, 2023
f8e98ce
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Jul 26, 2023
cb80688
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 26, 2023
fd830e9
style
alexrockhill Jul 26, 2023
e5bd6a9
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Jul 26, 2023
e1bcbd9
fix repr raising error
drammock Jul 26, 2023
4a76c9f
make repr more honest
drammock Jul 26, 2023
7ef0626
readability
drammock Jul 26, 2023
f031c8b
fix wrong docstring
drammock Jul 26, 2023
62e25f4
make docstrings parallel
drammock Jul 26, 2023
f8e74c9
add note about assuming power
drammock Jul 26, 2023
36df9b5
through version with bug fixes, plots were checked
alexrockhill Jul 27, 2023
147d110
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 27, 2023
ba24ebb
style'
alexrockhill Jul 27, 2023
026e8e6
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Jul 27, 2023
7340ea2
Merge branch 'main' into defaults
alexrockhill Jul 27, 2023
e4ca8ca
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Jul 27, 2023
4d48f42
style
alexrockhill Jul 27, 2023
b98d2b8
fix tests
alexrockhill Jul 28, 2023
d8fefc8
fix one last tests
alexrockhill Jul 28, 2023
3c4da24
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 28, 2023
9c2cd25
style
alexrockhill Jul 28, 2023
10a2667
Merge branch 'main' into defaults
alexrockhill Aug 1, 2023
92d8a9d
spelling
alexrockhill Aug 2, 2023
5a33651
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Aug 2, 2023
6404aaa
Merge branch 'main' of https://github.com/mne-tools/mne-python into d…
alexrockhill Aug 2, 2023
da45d5c
refactor tests
alexrockhill Aug 2, 2023
1a42bf6
oops switched psd, psd2
alexrockhill Aug 3, 2023
e71672a
style
alexrockhill Aug 3, 2023
04262f2
Merge branch 'main' of https://github.com/mne-tools/mne-python into d…
alexrockhill Aug 18, 2023
b099177
resolve conflicts
alexrockhill Aug 18, 2023
265f009
cruft
alexrockhill Aug 18, 2023
2427b9f
Merge branch 'main' into defaults
alexrockhill Aug 25, 2023
54488ed
try skip h5io
alexrockhill Aug 25, 2023
1a1e57f
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Aug 25, 2023
9f45402
Merge branch 'main' into defaults
alexrockhill Aug 29, 2023
8f49d05
remove complex support
alexrockhill Aug 29, 2023
831c1ee
Merge branch 'defaults' of https://github.com/alexrockhill/mne-python…
alexrockhill Aug 29, 2023
b6707f3
Merge branch 'main' into defaults
alexrockhill Aug 29, 2023
e178231
simplifications and fixes
drammock Sep 1, 2023
88c790e
Merge remote-tracking branch 'upstream/main' into defaults
drammock Sep 1, 2023
f735e12
oops missed this
drammock Sep 1, 2023
40627ac
Update tutorials/simulation/10_array_objs.py
drammock Sep 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/changes/latest.inc
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Enhancements
- Added :func:`mne.preprocessing.eyetracking.interpolate_blinks` to linear interpolate eyetrack signals during blink periods. (:gh:`11740` by `Scott Huberty`_)
- Added a section for combining eye-tracking and EEG data to the preprocessing tutorial "working with eye tracker data in MNE-Python" (:gh:`11770` by `Scott Huberty`_)
- Add :meth:`mne.Annotations.count` and :func:`mne.count_annotations` to count unique annotations (:gh:`11796` by `Clemens Brunner`_)
- Add :class:`mne.time_frequency.spectrum.EpochsSpectrumArray` and :class:`mne.time_frequency.spectrum.SpectrumArray` to allow for instantiating power spectra from scratch (:gh:`11803` by `Alex Rockhill`_)
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved

Bugs
~~~~
Expand Down
2 changes: 2 additions & 0 deletions doc/time_frequency.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ Time-Frequency
EpochsTFR
CrossSpectralDensity
Spectrum
SpectrumArray
EpochsSpectrum
EpochsSpectrumArray

Functions that operate on mne-python objects:

Expand Down
8 changes: 7 additions & 1 deletion mne/time_frequency/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@
)
from .ar import fit_iir_model_raw
from .multitaper import dpss_windows, psd_array_multitaper, tfr_array_multitaper
from .spectrum import EpochsSpectrum, Spectrum, read_spectrum
from .spectrum import (
EpochsSpectrum,
EpochsSpectrumArray,
Spectrum,
SpectrumArray,
read_spectrum,
)
from ._stft import stft, istft, stftfreq
from ._stockwell import tfr_stockwell, tfr_array_stockwell
128 changes: 128 additions & 0 deletions mne/time_frequency/spectrum.py
Original file line number Diff line number Diff line change
Expand Up @@ -1179,6 +1179,56 @@ def __getitem__(self, item):
return BaseRaw._getitem(self, item, return_times=False)


@fill_doc
class SpectrumArray(Spectrum):
"""Spectrum object from numpy array.
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved

Parameters
----------
data : array, shape (n_channels, n_freqs)
The channels' power spectral density for each epoch.
%(info_not_none)s
%(freqs_tfr)s
drammock marked this conversation as resolved.
Show resolved Hide resolved
%(method_psd)s
drammock marked this conversation as resolved.
Show resolved Hide resolved
%(verbose)s

See Also
--------
mne.create_info
mne.EpochsArray
mne.EvokedArray
mne.io.RawArray
EpochsSpectrumArray
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved

Notes
-----
.. versionadded:: 1.5.0
"""

@verbose
def __init__(
self,
data,
info,
freqs,
method=None,
*,
verbose=None,
):
self.__setstate__(
dict(
method=method,
data=data,
sfreq=info["sfreq"],
dims=("channel", "freq"),
freqs=freqs,
inst_type_str="Raw",
data_type="Average Power Spectrum",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the data comes from an array we don't know if it was computed on Raw or averaged (Evoked) data. Also we don't know if it's power/amplitude, real/complex, etc.

Suggested change
inst_type_str="Raw",
data_type="Average Power Spectrum",
inst_type_str="Array",
data_type="Unknown",

This will probably break some tests (assuming we're testing thoroughly) so hopefully those test failures can guide what needs to change elsewhere

Copy link
Contributor Author

@alexrockhill alexrockhill Jul 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So real/complex can be inferred from the data type. We ask specifically for power in the constructor so maybe we should require that and let the amplitude kwarg in Spectrum.plot handle that? That'd be my 2 cents but not highly opinionated, whatever is simplest

info=info,
)
)


@fill_doc
class EpochsSpectrum(BaseSpectrum, GetEpochsMixin):
"""Data object for spectral representations of epoched data.
Expand Down Expand Up @@ -1368,6 +1418,84 @@ def average(self, method="mean"):
return Spectrum(state, **defaults)


@fill_doc
class EpochsSpectrumArray(EpochsSpectrum):
"""EpochsSpectrum object from numpy array.
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved

Parameters
----------
data : array, shape (n_epochs, n_channels, n_freqs)
The channels' power spectral density for each epoch.
%(info_not_none)s
%(freqs_tfr)s
%(method_psd)s
events : None | array of int, shape (n_events, 3)
The events typically returned by the read_events function.
If some events don't match the events of interest as specified
by event_id, they will be marked as 'IGNORED' in the drop log.
If None (default), all event values are set to 1 and event time-samples
are set to range(n_epochs).
event_id : int | list of int | dict | None
The id of the event to consider. If dict,
the keys can later be used to access associated events. Example:
dict(auditory=1, visual=3). If int, a dict will be created with
the id as string. If a list, all events with the IDs specified
in the list are used. If None, all events will be used with
and a dict is created with string integer names corresponding
to the event id integers.
metadata : instance of pandas.DataFrame | None
See :class:`mne.Epochs` docstring for details.
%(selection)s
%(drop_log)s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering what makes the most sense for these. Would like opinions of other devs (@hoechenberger @larsoner @mscheltienne @agramfort) as well as @alexrockhill... Off the top of my head:

  • events I can see why folks might want to pass this in but is there any reason it needs to be a "proper" events array (N x 3)? Seems like only the last column is going to matter? it can be set after-the-fact so we could omit it from the constructor, but my gut is that this will be frequently used so maybe best to leave it here?
  • event_id again I can see why folks might want this (for __getitem__)... it can be set after-the-fact and probably doesn't hurt to allow that; not sure about forcing it to be after-the-fact if we're going to have events as a param though. Thoughts?
  • metadata can be set after instantiation so I'm inclined to leave it out of the constructor
  • selection seem like this shouldn't be needed if you're working with np.array data. Is there a use case you're thinking of where users would want to pass anything besides selection=np.arange(data.shape[0])? If not we can just set that internally
  • drop_log similar to selection; is there a use case you're thinking of where users would want to pass anything besides drop_log=tuple(tuple() for _ in range(data.shape[0]))? If not we can just set that internally

If we do end up keeping events and event_id as-is, the docstring should be deduped with EpochsArray (where it seems you copy-pasted these entries from)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on simplifying the constructor, as inferred, this was copy-pasted with the thought that it should just be similar to EpochsArray and without much more thought than that. I think events and event_id are helpful in the constructor so people don't lose expected functionality.

%(verbose)s

See Also
--------
mne.create_info
mne.EpochsArray
mne.EvokedArray
mne.io.RawArray
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
SpectrumArray

Notes
-----
.. versionadded:: 1.5.0
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
"""

@verbose
def __init__(
self,
data,
info,
freqs,
method=None,
events=None,
event_id=None,
metadata=None,
selection=None,
drop_log=None,
*,
verbose=None,
):
self.__setstate__(
dict(
method=method,
data=data,
sfreq=info["sfreq"],
dims=("epoch", "channel", "freq"),
freqs=freqs,
inst_type_str="Epochs",
data_type="Epochs Power Spectra",
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
info=info,
events=events,
event_id=event_id,
metadata=metadata,
selection=selection,
drop_log=drop_log,
)
)


def read_spectrum(fname):
"""Load a :class:`mne.time_frequency.Spectrum` object from disk.

Expand Down
29 changes: 29 additions & 0 deletions mne/time_frequency/tests/test_spectrum.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from mne import Annotations
from mne.time_frequency import read_spectrum
from mne.time_frequency.multitaper import _psd_from_mt
from mne.time_frequency.spectrum import SpectrumArray, EpochsSpectrumArray


def test_spectrum_errors(raw):
Expand Down Expand Up @@ -386,3 +387,31 @@ def test_spectrum_kwarg_triaging(raw):
raw.plot_psd(axes=axes)
# `ax` is the correct legacy param name
raw.plot_psd(ax=axes)


def test_spectrumarray_raw(raw):
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
"""Test SpectrumArray for Raw-derived spectra."""
spect = raw.compute_psd()
spect2 = SpectrumArray(
data=spect.get_data(), freqs=spect.freqs, info=spect.info, method=spect.method
)
assert_array_equal(spect.get_data(), spect2.get_data())
assert_array_equal(spect.freqs, spect2.freqs)
drammock marked this conversation as resolved.
Show resolved Hide resolved


def test_epochsspectrumaray(epochs):
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
"""Test EpochsSpectrumArray for Epochs-derived spectra."""
spect = epochs.compute_psd()
spect2 = EpochsSpectrumArray(
data=spect.get_data(),
freqs=spect.freqs,
info=spect.info,
method=spect.method,
events=epochs.events,
event_id=epochs.event_id,
metadata=epochs.metadata,
selection=epochs.selection,
drop_log=epochs.drop_log,
)
assert_array_equal(spect.get_data(), spect2.get_data())
assert_array_equal(spect.freqs, spect2.freqs)
38 changes: 3 additions & 35 deletions tutorials/simulation/10_array_objs.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,39 +224,7 @@

psd_ave = psd.mean(0)

# map to `~mne.time_frequency.Spectrum` class and explore API


def spectrum_from_array(
data: np.ndarray, # spectral features
freqs: np.ndarray, # frequencies
inst_info: mne.Info, # the meta data of MNE instance
) -> mne.time_frequency.Spectrum: # Spectrum object
"""Create MNE averaged power spectrum object from custom data"""
state = dict(
method="my_welch",
data=data,
sfreq=inst_info["sfreq"],
dims=("channel", "freq"),
freqs=freqs,
inst_type_str="Raw",
data_type="Averaged Power Spectrum",
info=inst_info,
)
defaults = dict(
method=None,
fmin=None,
fmax=None,
tmin=None,
tmax=None,
picks=None,
proj=None,
reject_by_annotation=None,
n_jobs=None,
verbose=None,
)
return mne.time_frequency.Spectrum(state, **defaults)


spectrum = spectrum_from_array(data=psd_ave, freqs=freqs, inst_info=info)
spectrum = mne.time_frequency.spectrum.SpectrumArray(
alexrockhill marked this conversation as resolved.
Show resolved Hide resolved
data=psd_ave, freqs=freqs, info=info, method="my_welch"
)
spectrum.plot(picks=[0, 1], spatial_colors=False, exclude="bads")
Loading