6.7: Exercise - The Signal-to-Noise Ratio

In the previous exercise, we saw that there was a lot of high-frequency noise in the right-hemisphere electrodes, especially F8. In the present exercise, we’re going to quantify the impact of this high-frequency noise on the data quality and also see how the data quality depends on the number of trials being averaged together. We'll use the 12_P3 ERPset you created in the previous exercise. If you don't still have it loaded in the ERPsets panel, go ahead and load it now from the Chapter_6 folder.

Take a look at the analytic standardized measurement error (aSME) values for this ERPset by going to the View Data Quality Metrics panel and clicking Show in a table. There’s a ton of information in the table, and I find that it helps to select the Color heatmap option. Interestingly, the aSME values from the F8 electrode are not particularly high. The worst values are at PO4.

This disconnect between the aSME values and the high-frequency noise you can see in the waveforms occurs because these aSME values tell you about the precision of the measurements of the mean voltage over 100-ms periods. As we discussed in Chapter 4, high-frequency noise has very little impact on the mean voltage over a 100-ms period, because the upward and downward noise deflections (the uppies and downies) cancel out. Low-frequency noise, by contrast, has a big impact on the mean voltage over a given time period. You can't easily see low-frequency noise in an averaged ERP, so let's go back to the EEG tab and take a look at the epoched EEG file from the previous exercise (12_P3_corrected_elist_bins_be; there's a copy in the Extra Files folder if you don't still have it loaded).

The PO4 channel should be a yellow waveform near the bottom of the display (see Screenshot 6.7). If you scroll through the epochs, you'll see that this channel often drifts upward or downward across an epoch, much more than the other channels. Screenshot 6.7 shows an example of epochs 105-109. This drift in the signal is why PO4 has the worst (largest) aSME value. If we were to quantify P3b amplitude as the mean voltage between 300 and 500 ms (as recommended by Kappenman et al., 2021), the random upward and downward low-frequency drifts in PO4 would mean that our P3b amplitude score could be quite far from the participant’s true score for that channel (i.e., the score we would obtain with an infinite number of trials). This is what aSME represents: the extent to which the measured value is likely to differ randomly from the true value. By contrast, the high-frequency noise in F8 wouldn’t have much impact on our ability to get the true value.

If we instead quantified P3b amplitude as the peak voltage between 300 and 500 ms, the high-frequency noise would be a bigger problem. Computing the standardized measurement error for peak amplitude is more complicated, so we’re not going to look at it now, but if we did, I’m sure that the high-frequency noise in F8 would produce a large SME value. The take-home message is that the effect of noise on your ability to precisely measure the amplitude or latency of a given ERP component depends on both the nature of the noise (e.g., high-frequency versus low-frequency) and the nature of the method used to quantify the amplitude or latency (e.g., mean amplitude versus peak amplitude).

Now let’s look at how the data quality differs between the Rare and Frequent averages. A standard idea in the ERP literature is that the signal-to-noise ratio (SNR) of an averaged ERP increases according to the square root of the number of trials (all else being equal). I have to admit that I didn’t understand exactly what was meant by "signal" and "noise" in the signal-to-noise ratio until a few years ago, when we started developing the SME metric of data quality. The "signal" part of the signal-to-noise ratio is the true value of an amplitude score (i.e., the amplitude we would obtain with an infinite number of trials).

The noise is quantified as the standard error of the amplitude score. Let's unpack that. Imagine that our score is the mean voltage between 300 and 400 ms in the averaged ERP waveform. This is equivalent to taking the mean voltage between 300 and 400 ms on each individual trial and averaging them together. The standard error of this score can be estimated using the standard analytic formula for the standard error of the mean: SD ÷ sqrt(N). That is, we compute the mean voltage between 300 and 400 ms on each trial, take the standard deviation (SD) of these voltages, and divide by the square root of the number of trials. Because the denominator is sqrt(N), this standard error gets smaller if we increase the number of trials according to sqrt(N). So, the denominator of the signal-to-noise ratio decreases according to the square root of the number of trials, causing the overall SNR to increase according to the square root of the number of trials.

In our P3b oddball experiment, 20% of the trials were oddballs, so there were 4 times as many Frequent trials as Rare trials. Consequently, sqrt(N) was twice as great for the Frequent condition as for the Rare condition (because sqrt(4) = 2). And this implies that the standard error should be half as large for the Frequent condition as for the Rare condition. The SME value is a generalized version of the standard error; it gives you the standard error for any amplitude or latency measure that is obtained from an averaged ERP waveform (see the box below for more details). When we are scoring the amplitude as the mean voltage during a specific time period, we can use the simple formula for the standard error [SD ÷ sqrt(N)] to estimate the SME, so we call this the analytic standardized measurement error or aSME.

Take a look at the aSME values for the Rare and Frequent conditions. You should see that the values are approximately half as large for the Frequent condition (Bin 2) as for the Rare condition (Bin 1) for any given combination of time period and electrode site. That is, the noise (quantified as the SME) is about half as big in the condition with four times as many trials. This is exactly what we would expect from the idea that the signal-to-noise ratio varies according to the square root of the number of trials.

The aSME is just an estimate, so we wouldn’t expect it to be perfectly predicted by the number of trials in a finite data set. To get a more robust estimate, I made an average across all the time ranges and channels, and I found a mean of 0.617 for the Frequent condition and 1.372 for the Rare condition. This isn’t quite a 1:2 ratio. But there’s a good explanation for this: although there were 4 times as many Frequent trials as Rare trials in the experiment, we ended up excluding more Rare trials from the averages because of incorrect responses. Take a look at the actual number of trials in Bins 1 and 2—does the difference between the average aSME values for the Rare and Frequent trials make sense given the actual sqrt(N) for the Rare and Frequent trials?

---