8.11: Exercise- Commonly Recorded Artifactual Potentials (C.R.A.P.)

General approach to C.R.A.P. detection in the ERP CORE experiments

In the ERP CORE experiments, our approach to C.R.A.P. detection was to apply both a simple voltage threshold and the moving window peak-to-peak algorithm to all EEG channels. The simple voltage threshold is particularly good for detecting large drifts in voltage, and a threshold of ±200 µV works well for most participants. The moving window peak-to-peak algorithm is good for detecting muscle bursts and idiosyncratic artifacts that involve sudden voltage changes. The threshold for this routine needs to be adjusted for each participant to ensure that large artifacts are detected without throwing out too many epochs. In most participants, we used a threshold between 100 and 150 µV.

Subject 1 from the ERP CORE MMN experiment does not have very interesting C.R.A.P. artifacts, so we will start by looking at the data from Subject 6. In addition, it's a lot easier to demonstrate these artifacts in the absence of blinks, so we will work with a copy of Subject 6's data in which ICA-based artifact correction has been applied to estimate and remove the blink-related voltages (using procedures that will be described in the next chapter).

Start by resetting everything with the Reset button, and then load the EEGset named 6_MMN_blinks-corrected_be using the EEGset panel. As you will see, this EEGset has already been epoched (as indicated by the _be suffix in the EEGset name). You will also see that there is a VEOG-bipolar channel in which the blinks have not been corrected. This channel is useful for telling us when blinks occurred. For example, you can see from this channel that there was a large blink in epoch 3 (which is easier to see if you select the Stack option in the Plot Settings panel). You can also see that there is no blink-related activity in the FP1 and FP2 channels in this epoch, indicating that the artifact correction worked reasonably well.

In this example, we will assume that the MMN will be measured as the mean amplitude from 125-225 ms in a cluster of electrode sites. Zhang and Kappenman (2024) carefully examined the best channels for scoring the amplitudes for each of the ERP CORE paradigms. They found that, in most cases, averaging across a cluster of electrodes around the site with the maximum effect was the best option. For the MMN, the maximum effect was at FCz, and they recommended scoring from the average of Fz, FCz, Cz, FC3, and FC4. You can average across sites after making the averaged ERPs (or even after scoring the values from each individual site). However, we will want to compute the aSME metric of data quality to see if artifact rejection is improving the data quality, and this requires creating the cluster (i.e., averaging across sites) in the EEG, prior to averaging. If you look at the current EEGset, you'll see that it includes a channel named Cluster (channel 17), which I created by averaging together the Fz, FCz, Cz, FC3, and FC4 channels. We will try to maximize the data quality for this cluster channel.

Our approach will be to test all EEG channels for C.R.A.P. and then exclude the entire epoch (all channels) if C.R.A.P. is found in any channel. You might think that it would be better to check for C.R.A.P. only in the channel(s) being used for measurement. That way, you won't reject trials with C.R.A.P. in other channels that won't impact data quality in the measurement channel(s). I had the same intuition. But when we tried this, we found that it did not lead to better data quality than excluding trials that had C.R.A.P. in any channel (Zhang et al., 2024c). The reason for this is that C.R.A.P. is usually present across a wide range of electrodes, but it might not exceed the threshold for rejection in all sites. As a result, if you check for C.R.A.P. only in the measurement electrode(s), you will sometimes fail to mark epochs for rejection when C.R.A.P. is present. However, there may be some cases where this is a reasonable approach.

If an artifact is occurring at only a single electrode site, and it is present throughout much of the recording session, then you should interpolate that site. If you have an artifact at a single site that occurs only infrequently, you can apply artifact detection to that site but then interpolate that site for those epochs rather than rejecting those epochs (click here for more information).

Finding rapid voltage changes with the moving window peak-to-peak algorithm

Let's first try to detect C.R.A.P. in the loaded EEGset using the moving window peak-to-peak algorithm and a threshold of 150 µV. Go to the Artifact detection panel and set it up as shown in Screenshot 8.8. Note that we are going to look for C.R.A.P. only in the EEG channels (1:29), not in the EOG channels. We're also going to prefilter at 30 Hz so that high-frequency noise doesn't cause epochs to be flagged (because this noise has minimal impact on mean amplitude scores). Also note that the box for Flag 2 should be checked. This will cause every epoch that is flagged for rejection to have Flag 2 set. When we later detect C.R.A.P. with the absolute voltage threshold algorithm, we will use Flag 3, and the different flags will allow us to determine whether a given epoch was flagged because of its peak-to-peak amplitude or its absolute voltage.

Click the Run button and name the resulting EEGset 6_MMN_blinks-corrected_be_mw150 (where mw150 indicates that you used the moving window peak-to-peak algorithm with a threshold of 150). You'll then see a summary table showing that only 17 epochs were marked for rejection (4 Deviants and 13 Standards). Is it even worthwhile to reject only 2% of epochs?

To answer that question, you should first scroll through the new EEGset and see what the EEG looks like in the marked epochs versus the unmarked epochs. For the first 35 epochs, the EEG looks very clean and no epochs are marked (i.e., no epochs have a yellow background), although there are some blinks and eye movements visible in the EOG channels. Epoch 37 seems to have some large voltage swings in several of the EEG channels, but these artifacts were not detected by the algorithm. However, the Cluster channel did not show a large voltage swing, so it's fine that this epoch was not marked for rejection.

If you keep scrolling, you'll eventually see that epochs 82-85 have some very large artifacts (> 100 µV) in the EEG channels, and they are marked for rejection. The Cluster channel didn't exceed the threshold for rejection, but this channel was quite noisy in those epochs, so it's good that those epochs were marked for rejection. This is an example of why it can be good to reject an epoch even if the artifact is flagged in a channel not being used for scoring the component of interest.

Skip ahead to Epoch 120. You'll see that it's marked for rejection, and the Cluster channel was one of the channels that exceeded the threshold for rejection. Epoch 121 has a very large swing in voltage, but it did not exceed the threshold. This is because the moving window peak-to-peak algorithm is looking for relatively sudden changes in voltage, not gradual drifts like this. In a moment, we will use the simple voltage threshold algorithm to detect drifts like this.

Once you feel that you've viewed enough epochs to see what kinds of voltage patterns exceeded the threshold, you should quantify the data quality. Let's first do this for the original EEGset, prior to artifact detection, so select 6_MMN_blinks-corrected_be in the EEGsets panel. Now go to the Compute Data Quality Metrics panel. It should still have Custom parameters selected from the last time you used it (to include the MMN measurement window of 125-225 ms). If not, take a look at Screenshot 8.5 in Section 8.9 to see how to set this up. Then click Run to see the data quality table.

Now repeat the process of computing the data quality with 6_MMN_blinks-corrected_be_mw150 selected in the EEGsets panel. If you look in the 125:225 column of the two tables you just created, you'll see that the aSME values in many channels are quite a bit lower after artifact rejection compared to before artifact rejection. In the Cluster channel, for example, the aSME went from 0.8167 before rejection to 0.6030 after rejection (in Bin 1). Even though we've rejected only about 2% of the epochs, we've reduced the aSME by over 25%. A few epochs with large C.R.A.P. can really impact your data quality!

Finding slow voltage drifts with the simple voltage threshold algorithm

Now let's see if we can improve the data quality further by rejecting epochs that had large voltage swings but were not marked for rejection by the moving window peak-to-peak algorithm. Make sure that 6_MMN_blinks-corrected_be_mw150 is still selected, because we want to keep those marks and add to them. Now go to the Artifact Detection panel and set it up to use the Simple voltage threshold with a threshold of -200 200 (as shown in Screenshot 8.9). Note that we're now going to set Flag 3 instead of Flag 2.

Click the Run button and name the resulting EEGset 6_MMN_blinks-corrected_be_mw150_svt200 (where svt200 indicates that you used a simple voltage threshold of 200). The summary table will show the number of epochs flagged for each bin by the previous algorithm in the F2 column (4 Deviants and 13 standards) and the number flagged by the simple voltage threshold in the F3 column (5 Deviants and 15 standards). However, many of the same epochs were flagged by both algorithms, so the total percentage rejected has gone up only slightly from 2.17% before to 2.55% now. That won't change the end result very much, so let's try a lower threshold that will flag more trials.

Go back to the Artifact Detection panel, change the Voltage limits to -100 100, click Run, and name the resulting EEGset 6_MMN_blinks-corrected_be_mw150_svt100. You'll see that the number of epochs flagged with F3 has gone up quite a bit, and 4.2% of epochs have now been marked for rejection.

If you scroll through the epochs, you'll see that Epochs 37, 86, and 121 are now marked for rejection. These epochs have large but gradual drifts that the moving window peak-to-peak algorithm missed but were detected by the simple voltage threshold algorithm once we made the threshold low enough. However, if you generate the data quality table for this EEGset, you'll see that the aSME values haven't changed much.

You could try lowering the threshold to -80 80, -60 60, or -50 50. I tried all of these thresholds, and I found that the percentage of epochs marked for rejection increased as the threshold decreased (not surprisingly). However, the aSME value for the Cluster channel remained about the same. This is because the fact that the remaining epochs were cleaner (which makes the aSME better) was balanced by the fact that there were fewer trials left for averaging (which makes the aSME worse). I also tried a threshold of -40 40, but this led to the rejection of over 30% of trials (which would cause the subject to be excluded given my lab's a priori exclusion criterion of 25%) and made the aSME worse.

As I was developing this exercise, I wanted to show you that the simple voltage threshold algorithm was useful, and I was initially annoyed that it didn't improve the data quality over and above the improvement produced by the moving window peak-to-peak algorithm. I even tried finding another subject for whom it was helpful, but I couldn't find one. Then I realized that this is exactly what I should have expected. The slow drifts that the absolute voltage threshold picks up aren't a big problem when you are measuring the voltage at relatively short latencies (such as the 125-255 ms time window used for the MMN). At these short latencies, the baseline correction procedure takes care of much of the drift; a slow drift doesn't have time to change the voltage much at these short latencies. We could probably just use the moving window peak-to-peak algorithm for detecting artifacts in this paradigm. However, it doesn't hurt to also reject trials with slow drifts using the simple voltage threshold algorithm as long as we don't use an overly strict threshold. And there are many paradigms where it will help.

Example of slow drift from the P3b paradigm

To see how the simple voltage threshold algorithm can be helpful when you are looking at later time windows, we're going to look at an example from the P3b paradigm (which was described in Section 6.2 of Chapter 6). Start by clearing out everything from the EEGsets panel by clicking Clear All, and then load 30_P3_blinks-corrected_be. I chose this EEGset because it contains tons of epochs with slow drifts. As in the previous examples, ICA-based artifact correction was used to minimize blink activity, but the EEGset also contains uncorrected VEOG-bipolar and HEOG-bipolar channels so you can see when the subject blinked or made an eye movement. I also added a channel labeled Cluster, which is the average of the Cz, CPz, Pz, P3, and P4 electrode sites (as recommended for scoring the P3b by Zhang and Kappenman, 2024).

Start by running the moving window peak-to-peak algorithm with a threshold of 150 (exactly as shown above in Screenshot 8.8). You can name the resulting EEGset 30_P3_blinks-corrected_be_mw150. The summary window will show that 22 of the epochs were marked for rejection, which is 11.89% of the relatively small number of trials in this paradigm. If you scroll through the data, you'll see that many epochs with slow drifts were not marked (e.g., Epochs 57 and 63, which have fairly large drifts in the Cluster channel). The marked epochs mostly have relatively sudden voltage changes (e.g., Epochs 81 and 88).

Now let's see whether rejecting the marked epochs improves the data quality. Go to the Compute Data Quality Metrics panel. It should still have Custom parameters selected from the last time you used it, but with the custom time period for the MMN. Change that custom time period to 300-600 ms (and change the name accordingly). You can consult Screenshot 8.5 in Section 8.9 if you need a reminder of how to do this.

Once you've updated the custom time period, let's see what the data quality would be like without any rejection. Previously, you accomplished this by going to the EEGsets panel and selecting the original EEGset. But there is a quicker way: In the Compute Data Quality Metrics panel, select the radio button labeled Include all epochs (ignore artifact detection). Then click Run to see the data quality table for this dataset without excluding the marked epochs. Now select the radio button labeled Exclude epochs marked during artifact detection, and then click Run again to see the data quality table for this dataset after excluding the marked epochs. By consulting the two data quality tables you just created, you can see that the aSME during the 300:600 time window was improved by excluding the marked epochs in most channels, including the Cluster channel (for both Bin 1 and Bin 2).

Now try adding the simple voltage threshold algorithm with a threshold of 200 (exactly as shown in Screenshot 8.9), naming the resulting EEGset 30_P3_blinks-corrected_be_mw150_svt200. You should see that 22.16% of trials are now marked (which is getting close to my lab's limit of 25%, above which we would need to exclude the entire subject from further analysis). If you look through the data, you'll see that epochs with large drifts are now marked (e.g., Epochs 57 and 63, which were not marked before). If you look at the data quality metrics, you'll see that the aSME for the Cluster site in the 300:600 time window is about the same as it was with just the moving window peak-to-peak algorithm in Bin 1, but the aSME is now much improved for this case in Bin 2.

Just like any other metric, the aSME can be better estimated for a bin with more trials, so you should pay more attention to aSME values from bins with more trials (such as Bin 2 in the present paradigm). Or, if you want to get really serious about quantifying data quality, you can estimate the aSME for the difference between two bins. In the P3b paradigm, for example, the ultimate score is the mean voltage from 300-600 ms in a rare-minus-frequent difference wave. As discussed in Section 4.15 of the chapter on filtering, you can obtain the aSME corresponding to a difference wave between conditions A and B using this formula: SME_A-B = sqrt(SME_A² + SME_B²). The overall aSME was sqrt(4.8086² + 5.3852²) = 7.2196 without any rejection, sqrt(3.5405² + 3.2736²) = 4.82199 after the moving window peak-to-peak rejection, and sqrt(3.6970² + 1.9024²) = 3.9459 after both the moving window peak-to-peak and simple voltage threshold rejection. Thus, adding the simple voltage threshold improved the aSME value corresponding to the rare-minus-frequent difference wave.

A very challenging example from the P3b paradigm

Up to this point, I've selected example EEGsets for which it's fairly easy to mark the artifacts and end up with better data quality. Now I want to show you a very difficult case so you can experience the failure you will occasionally encounter when you perform artifact detection on your own data. (For fans of Star Trek, this is like the Kobayashi Maru).

I recommend resetting everything with the Reset button before going further. Then load 9_P3_blinks-corrected_continuous using the EEGsets panel. I wanted you to start with the continuous EEG so that you better understand why the epoched data looks so bad and why artifact detection will be so difficult. If you scroll through the data, you'll see that it looks like a roller coaster ride, going up and down and up and down. This participant also blinked near the time of the buttonpress response (event code 201) on almost every trial. An example is shown in Screenshot 8.10. I have no idea what is causing these roller-coaster-like voltage deflections. They're too fast to be skin potentials, but they are too slow to be the kinds of movement artifacts we typically see in my lab (which are usually sudden movements). This is the sort of EEGset that led me to develop the acronym C.R.A.P. for miscellaneous artifacts.

Once you've looked through the continuous EEG, load the epoched EEG file,  9_P3_blinks-corrected_be. If you scroll through the data, you'll see these same upward and downward deflections, which often go up and then down (or down and then up) within a single epoch. These deflections are often over 50 µV, which is very large relative to the ERP waveforms we obtained in this experiment. Moreover, these large deflections are going to produce tremendous trial-to-trial variation in the mean voltage between 300 and 600 ms, making it very difficult to obtain a precise estimate of this subject's P3b amplitude. In fact, this subject's aSME without any rejection is about twice as high as the aSME from the previous example without any rejection and about four times as high as the average ASME across all participants after rejection.

Rather than taking you step by step through the artifact detection process for this subject, I'm going to describe the various artifact detection parameters I tried. I've provided the EEGsets for each of these parameters. However, if you're up for the challenge, you can try to come up with your own artifact detection parameters before reading about how I came up with my parameters.

I started by applying the moving window peak-to-peak algorithm with a threshold of 150 (see the file named 9_P3_blinks-corrected_be_mw150). Not a single epoch was flagged by this (which you can see for yourself by loading the file, going to the Artifact Info & Tools panel, and clicking Artifact Summary). That makes sense, because there weren't any large, sudden voltage changes in the EEG channels of this EEGset.

Next I applied the simple voltage threshold algorithm with a threshold of 200 (see 9_P3_blinks-corrected_be_mw150_svt200). Only two epochs were flagged. That makes sense, because there weren't many large drifts that went consistently upward or consistently downward within a single epoch. Excluding the two flagged epochs had no real impact on the aSME.

My usual approach to C.R.A.P. detection is to start with these two steps and then make adjustments to the thresholds if necessary. This was a case where adjustments were definitely necessary. I started by reducing the simple voltage threshold to 100 (9_P3_blinks-corrected_be_mw150_svt100). That flagged 27% of trials, which is above my lab's maximum of 25%, which would have led this subject to be excluded. Also, it only reduced the aSME by a modest amount. And, as you can see in Screenshot 8.11, it mainly flagged epochs that had a consistent upward or downward drift (like Epoch 11), and it failed to flag epochs that had large up-and-down shifts within the epoch (like Epoch 14).

I next tried raising the threshold to 120 to avoid going over my lab's limit of 25% rejection (9_P3_blinks-corrected_be_mw150_svt120). That reduced the number of flagged epochs to 15%, but now it more frequently failed to flag high-noise trials, and the aSME was hardly any better than what I observed without any rejection.

Next, I went back to the simple voltage threshold of 200 and added the moving window peak-to-peak algorithm with a lower threshold of 100 (9_P3_blinks-corrected_be_mw150_svt200_mw100). This marked 11.5% of the epochs. It successfully flagged epochs with sudden changes in voltage (e.g., Epoch 24 in Screenshot 8.12), but it failed to mark epochs that had large but more slowly changing voltage fluctuations (e.g., Epoch 25 in Screenshot 8.12). Also, the aSME was now even worse than it had been without any rejection.

I tried decreasing the moving window threshold even further, down to 80, but this put me over 25% of epochs flagged, and the aSME was still bad (9_P3_blinks-corrected_be_mw150_svt200_mw80). The problem was that the algorithm wasn't flagging epochs like Epoch 25 in Screenshot 8.12. The voltage in those epochs changed too slowly to produce a large voltage deflection within the 200 ms sliding window, but because the artifacts went both upward and downward within a single epoch, the simple voltage threshold algorithm also failed to flag them.

Then I realized that I might be able to detect these artifacts if I increased the width of the sliding window. I tried a few different window widths, but what ended up working the best was a window width of 995 ms (which is the entire width of the epoch). When I tried a 995 ms moving window width with a threshold of 100, it did a great job of flagging the epochs that had large voltage deflections (9_P3_blinks-corrected_be_mw150_svt200_mw100-995). And it brought the aSME down quite a bit, from 8.5124 without rejection to 5.7999 with rejection (for the rare trials in Bin 1). But that's still not great; the average aSME for rare trials in the P3b paradigm was less than 2.0 (Zhang & Luck, 2023). Also, the P3b on rare trials in this paradigm had an amplitude of about 15 µV, and a standard error of about 5.7999 relative to an amplitude of 15 µV is pretty poor. Moreover, nearly 50% of trials were marked for rejection, greatly exceeding my lab's rule for excluding subjects.

The fact that the aSME was improved by over 30% by rejecting trials in this manner, even though the number of trials was reduced by half, makes it clear that the rejected trials really were adding a lot of noise. In other words, the peak-to-peak amplitude over a 995 ms window with a threshold of 100 was flagging bad trials and wasn't flagging good trials. But even the unflagged trials contained a lot of noise, so the aSME still remained very high.

I tried several other artifact detection parameters, but everything I tried led to a poor aSME or over 25% of trials rejected (or both). I had to admit defeat. There was no way that this subject could be included in the final analyses of the ERP CORE P3b experiment. In fact, this subject was excluded in the published paper because the person who set the artifact detection paradigm, Jaclyn Farrens, was also unable to find parameters that worked for this subject's data. Sometimes, you just have to throw out some of the participants in a study because of poor data.

I'd like to end this set of exercises by noting that there is nothing magic about excluding subjects for whom more than 25% of trials were rejected. That seems to work well in my lab's basic science studies using college students as the participants. But in my research on schizophrenia, where the participants are much more expensive to recruit and test, and the data are noisier, we use a threshold of 50% instead of 25%. You will need to choose a reasonable threshold for your own research, balancing the benefits of throwing out participants with C.R.A.P.P.Y. data with the costs of needing to recruit and test extra participants. Just make sure that you set this criterion a priori so that you don't bias your results (as discussed in Section 8.4).