Skip to content

Quality Assessment and Rating

Jump to bottom
JohnSmallridge edited this page Jun 15, 2020 · 6 revisions

Quality Assessment

The quality assessment GUI gives an overview of each dataset that has been preprocessed.

  • The data is plotted as a heatmap (MATLABs imagesc plot) with time points on the x axis and channels on the y axis. The color of each data point reflects the voltage magnitude. As artifacts are typically of higher amplitude than the EEG signal of interest, red and blue data points that stand out of the mostly green data points typically indicate some sort of noise.
    • The color scaling (by default fixed to a range of -100 to 100 mV) can be changed with the scrollbar in the lower right corner. The optimal values might differ between recording systems.
    • The Average Reference button changes the data rendered to average referenced data.
    • Auto Bad channels shows the automatically detected bad channels in blue lines (this only applies to data before interpolation)
    • View Raw File shows a 1 Hz high pass filtered figure of your data, which can be used to compare the noise before and after preprocessing.
  • The right panel shows an automated rating of the quality of the current data set that is based on a series of Quality scores (explained later)
    • The user can override these quality scores (but we do not recommend to do so) and can manually select channels that have not been identified as bad by either of the methods. To manually select channels, click on interpolate, then click on Turn on in the Selection Mode. Now, to be interpolated channels can be selected by clicking in the heatmap. After the selection, click Turn Off to continue the visual inspection.
    • These selected channels can be subsequently interpolated.
  • In the top panel, you can make a selection of, for example only the datasets that are rated as Good. This allows to quickly scroll through the datasets by clicking on Previous or Next.
  • Clicking the EEGLAB plot opens a new figure that shows the current data set in the EEGLAB channel plot.
  • Clicking the Quality Rating button additionally opens the quality rating GUI.

Quality Rating

The quality classification GUI allows to categorize data into Good, Ok and Bad datasets. This should serve as an objective way of including / excluding datasets into further analyses. Currently four quality measures are computed which all reflect ratios (range from 0 to 1):

  1. The overall high amplitude (OHA) measure reflects the ratio of data points (i.e. channels x timepoints) that have a higher absolute voltage magnitude of x mV, where x reflects a vector of voltage magnitude thresholds (set in the configurations).
  2. The timepoints of high variance (THV) measure reflects the ratio of time points, where the standard deviation of the voltage measures across all channels exceeds x mV, where x reflects a vector of standard deviation thresholds (set in the configurations).
  3. The channels of high variance (CHV) measure reflects the ratio of channels for which the standard deviation of the voltage measures across all time points exceeds x mV, , where x reflects a vector of standard deviation thresholds (set in the configurations).
  4. The ratio of bad channels (RBS) reflects the ratio of interpolated bad channels.

Using a selection of the quality measures (OHA, THV, CHV, RBC) and specified thresholds, the datasets of a study can be classified into three categories Good, Ok, Bad, by applying cutoffs for each criterion (as shown in the Figure above). All the values for thresholds and category cutoffs can be modified and therefore the user can compile samples in accordance to own purposes. The thresholds can be changed in the respective dropdown menu on the right side. This influences the sensitivity of a quality measure. The cutoff for a category (Good, Ok, Bad) can be modified using the respective sliders. To make the effect of the modifications visible, the resulting ratings for the currently displayed dataset (in the quality assessment GUI) is adapted. In addition, the frequencies of each categories is plotted in the barplot, so a categorization can be also based on the resulting sample sizes. To restrict from compiling categories and running analyses until effects become true (i.e. P-hacking), a selected categorical assignment can be committed by pressing Commit rating, which appends the letters g for Good, o for Ok, and b for Bad to the file name of the preprocessed EEG dataset in the results folder. There is a Reset rating button to return the GUI parameters to their default positions (the files are not altered), but as a sincere researcher you should not press it and then commit these settings just to improve dataset quality ratings. Practically, this initial commit can be overwritten (a popup indicates “that this is not a good research practice”) and a new threshold can be applied and committed, which adds another prefix. Therefore, repeated commits are easily identifiable with double (or triple, quadruple) prefixes.

Combination of File Prefixes

np - preprocessed file not rated (no channels to interpolate / interpolated)
gp - preprocessed file rated as Good (no channels to interpolate / interpolated)
op - preprocessed file rated as OK (no channels to interpolate / interpolated)
bp - preprocessed file rated as Bad (no channels to interpolate / interpolated)
ip - preprocessed file some electrodes to interpolate or have been interpolated
gip - preprocessed file rated as Good and interpolated at least once
oip - preprocessed file rated as OK and interpolated at least once
bip - preprocessed file rated as Bad and interpolated at least once
ip - preprocessed file rated as Interpolated and interpolated at least once

Examples for Multiple Commits:

ggiip - committed twice, the rating remained
goiip - committed twice, the rating changed from ok to good
.
.
.

Clone this wiki locally