9.11: Exercise- Does Artifact Correction+Rejection Improve Data Quality?

Details of Our Artifact Rejection Procedure

Here are the details of the artifact rejection. First, we wanted to exclude trials on which participants blinked at a time that interfered with seeing the target word. For this purpose, we applied the moving window peak-to-peak algorithm during the period from -25 to +225 ms with a threshold of 100 µV. We found that these parameters effectively detected blinks that occurred while the stimulus was visible (from 0 to 200 ms). There were hardly any blinks during this time window, because the participants knew when the targets would be presented. As a result, this part of the process had virtually no impact on the averaged ERP waveforms or the SME values. But I still like doing it—it’s an example of trying to chase perfection. Note that when an artifact is detected in a given epoch, there are several different flags that can be set. Flag 1 is set for any artifact, but you can set additional flags to indicate what kind of artifact was detected for a given epoch. The present script set flag F2 when a blink was detected at the time of the stimulus.

A more important part of the process was detecting and rejecting trials with extreme voltages that were unlikely to reflect brain activity (i.e., C.R.A.P.). As described in Chapter 8, we used two different algorithms to detect these extreme voltages. First, we flagged epochs with flag F3 if the voltage went beyond a range of -100 to +100 µV at any time. This was mainly used to detect relatively slow changes in voltage over the course of the epoch. Second, the moving window peak-to-peak algorithm was used to set flag F4 for epochs in which the peak-to-peak voltage exceeded 100 µV within any 200-ms window. This was mainly used to detect sudden changes in voltage, like those produced by head movements. The scripts were set up so that the artifact detection parameters are read from a spreadsheet, making it possible to specify different parameters for different participants, but the same parameters worked well for all 20 of the present participants.

For all of the artifact detection tests used here, I enabled the option for prefiltering the data at 30 Hz to avoid getting “fooled” by high-frequency noise. We did not do that in the pipeline used in our journal article (Zhang et al., 2024), but I wish we had. It allowed me to use a lower rejection threshold that flagged more trials with real C.R.A.P. without flagging trials that merely had high-frequency noise. This kind of noise doesn’t impact the mean amplitude scores used here, and it can be filtered out when other scoring methods are used, so I recommend using this prefiltering in most studies.

Assessing the Effects of Artifact Correction+Rejection on Data Quality

Let’s first look at the effects of artifact correction and rejection on the data from Subject 29, who had a moderately large number of trials flagged for C.R.A.P. First reset everything with the Reset button. Then click Load ERP in the ERPsets panel, navigate to N400_Data > 29, and open these three ERPsets:

• 29_N400_correction_plus_rejection.erp
• 29_N400_correction_without_rejection.erp
• 29_N400_no_correction_or_rejection.erp

Now let’s see how many epochs were marked for rejection. Select 29_N400_correction_plus_rejection in the ERPsets panel and then open the ERP & Bin Information panel and click the Bin Info Table button. This will pop up a window that includes a table with the details of the artifacts that were flagged. You will probably need to widen the window (or use the scroll bar) to see all the relevant flags. The table is shown in Screenshot 9.27. Below the table, you can see that 9.4% of epochs were flagged with artifacts. Flag F2 was set for one epoch in both bin 1 and bin 2, indicating that the participant blinked while the target word was visible for one epoch in each bin. Flag F3 was set for four epochs in each bin, indicating the number of epochs that exceeded the simple voltage threshold. Flag F4 was set for four epochs in bin 1 and two epochs in bin 2, indicating the number of epochs that exceeded the threshold for the moving window peak-to-peak amplitude algorithm. Some epochs were flagged with both F3 and F4, so the total number of flagged epochs for a given bin is less than the sum of the F2, F3, and F4 flags.

Now select all three ERPsets in the ERPsets panel, go to the View Data Quality Metrics panel, and click Show in a Table. You’ll see the usual data quality table. Because you selected all three ERPsets before creating this table, the Selected ERPset popup menu at the top of the window will allow you to display the data quality for any of the ERPsets. Select 29_N400_no_correction_or_rejection first. If you look at the aSME for bin 1 during the measurement window (300-500 ms) at the measurement channel (CPz), you’ll see an aSME value of 2.670. This is our data quality without any correction or rejection. Now look at the value for 29_N400_correction_without_rejection.  It’s very slightly better (2.5388). Now look at the value for 29_N400_correction_plus_rejection.  It’s even better (2.3149). If you repeat this process for bin 2, you’ll see the same pattern, with aSME values of 3.1710 for no correction or rejection, 3.1286 for correction without rejection, and 2.1534 for correction plus rejection. In other words, artifact correction improved the data quality slightly (i.e., slightly reduced the aSME), and combining correction with rejection improved the data quality somewhat more.

However, that was just for this one participant. The script named N400_correction_rejection_phase5.m summarizes the SME values for all 20 participants. If you run this script, you’ll see that it ends by printing the following to the Command Window:

Mean percentage of trials marked for rejection: 3.12

Summary of aSME values:
RMS(aSME) = 2.308 without correction or rejection
RMS(aSME) = 2.255 after correction but no rejection
RMS(aSME) = 2.055 after correction and rejection

You can see that we’re not rejecting a lot of trials on average (just 3.12%), but we’re rejecting as many as 15% of trials in some participants. It turns out that a few noisy participants can really reduce your statistical power, even if the data are super clean in most participants.

The script’s output shows that the SME for the study as a whole is worst when we don’t perform either correction or rejection, a little better when we correct blinks (and eye movements in a few participants), and even better when we also exclude trials with large C.R.A.P. (and a few trials with eye movements at the time of the stimulus). Note that we’re summarizing the SME across participants using the root mean square (RMS) of the single-participant values. The RMS is similar to the mean, but it gives greater weight to larger values; this reflects the fact that a few noisy participants have an outsized impact on your statistical power. Consequently, the RMS of the single-subject SME values is more directly related to statistical power than the mean of those values (Luck et al., 2021).

In the 20 participants included in this analysis, the combination of artifact correction and rejection reduced the SME by only 11%. That’s an improvement, but it’s not huge. However, those 20 participants were not a random sample, and when my lab included all 40 participants in our journal article, we found that artifact correction plus rejection cut the SME by over 50% (Zhang et al., 2024). When we looked at the other ERP CORE paradigms, we found an SME reduction of 14% for the N170 and the ERN, a reduction of 16% for the MMN, a reduction of 16% for the P3b, and a reduction of 36% for the MMN. Thus, the effects varied across paradigms. In all cases, however, the combination of artifact correction and rejection improved the data quality.

These percentage improvement values were generated by analyses of mean amplitude, and we found similar reductions in SME for peak amplitude. Interestingly, artifact correction and rejection had little or no effect on data quality for latency measures in the ERP CORE paradigms.

Conclusions About the Value of Artifact Correction and Rejection

The bottom line is that, at least for studies like the ERP CORE experiments, the combination of artifact correction and rejection described in this chapter can improve your data quality between a little and a lot, at least for amplitude measurements. Moreover, the artifact correction component of this pipeline can reduce or eliminate confounds, which is important for both amplitude and latency measurements.

In Chapter 7, I noted that the fundamental goal of EEG preprocessing is to accurately answer the scientific question that the experiment was designed to address. In Chapter 8, I further noted that artifacts interfere with answering scientific questions in 3 ways: they can reduce statistical power, they can be confounds, and they can indicate a problem with the sensory input. You’ve now seen rigorous, quantitative evidence that the combination of artifact correction and rejection can improve data quality (and therefore statistical power), minimize confounds (e.g., differences in blink voltages between conditions), and avoid sensory input problems (e.g., the eyes being closed when the stimulus appears). This means that the procedures described here can help you accurately answer the scientific question that your experiments are designed to address.