From LibreTexts, under CC BY-NC-SA 4.0, as the source states it; its equations are written in TeX, which the PDF shows typeset. This page at LibreTexts.
Explain how the brain uses place and timing to decide which signals belong together.
Describe the ventriloquism effect, the superadditive effect, and the principle of inverse effectiveness.
Explain how the vestibulo-ocular reflex keeps vision steady.
Picture yourself on Main Street in Coalinga. You hear a horn, a conversation across the street, smell food from a taquería, and spot a friend waving, all at once. Somehow, your brain sorts out which sights, sounds, and smells go together.
Which Signals Belong Together?
Your brain uses two main clues: signals that come from the same place at the same time probably come from the same event. Usually that works. But it can be tricked. In the ventriloquism effect, you hear the voice coming from the puppet’s moving mouth instead of the performer’s. Vision is better at pinpointing location, so it wins.
Scientists separate two kinds of effects. In multimodal effects, senses combine to help you perceive one event, like watching lips in a noisy room. In crossmodal effects, one sense changes what you perceive in another, like the ventriloquism effect.
Stronger Together
Some neurons respond to more than one sense. In the superior colliculus, certain neurons fire much more to a sight and sound together than you’d expect from adding up their responses to each one alone (Meredith & Stein, 1983). This is the superadditive effect.
Combining senses helps most when each sense alone is weak. This is the principle of inverse effectiveness. In a quiet library, you can hear a friend fine without watching their mouth. In a loud cafeteria, watching their lips helps a lot. In one classic study, seeing the speaker’s face helped people understand words in noise about as much as making the speech 15 decibels louder relative to the noise (Sumby & Pollack, 1954).
From a Peer
“Where I’m living right now, there’s noise all day: TVs, doors, people talking over each other. I catch myself watching people’s mouths more than I ever used to.”
Vision and Balance
Optional: You don't need to move your head or your finger. The text right after this describes what people typically notice, and reading it is enough. Skip the movement if it causes discomfort or dizziness.
Try this. Hold your head still and wave a finger back and forth in front of your eyes. It looks blurry. Now hold your finger still and shake your head side to side. Your finger stays sharp. That’s the vestibulo-ocular reflex: your inner ear senses your head moving, and your eyes automatically move the opposite way to keep the image steady.
Close to Home
During wildfire smoke season in the Valley, some people say food tastes flat or “off.” Much of what we call flavor actually comes from smell, so smoky air and irritated noses can change how food tastes.
Check Your Understanding
What two clues help your brain decide that signals belong to the same event?
In the ventriloquism effect, which sense wins for location?
What is the principle of inverse effectiveness?
Which reflex keeps your vision steady when your head moves?