From LibreTexts, under CC BY-NC-SA 4.0, as the source states it; its equations are written in TeX, which the PDF shows typeset. This page at LibreTexts.
Define synesthesia and multimodal perception and give an example of each.
Imagine always seeing the letter A as red and the number 3 as yellow, even when they’re printed in black ink. You might assume everyone sees it that way, until a friend gives you a strange look.
That experience is called synesthesia, when a stimulus automatically evokes an additional, consistent experience, such as a color linked to a letter or sound. In grapheme-color synesthesia, letters or numbers have their own consistent colors. Estimates vary, but one large study found that about 4 in 100 people have some type of synesthesia (Simner et al., 2006). It isn’t a disorder. It’s a reminder that perception happens in the brain. The artistic image below shows four types of synesthesia.
Image description: An artistic image of a young woman with her eyes closed and a glowing brain, surrounded by swirls of color. Four labels name types of synesthesia: “Grapheme-color synesthesia: letters and numbers are always seen in specific colors,” beside floating colored letters and numbers; “Sound-color synesthesia: sounds trigger the experience of color,” beside musical notes; “Sound-taste synesthesia: certain sounds automatically evoke tastes,” beside a lemon, chocolate, a strawberry, and coffee; and “Touch-color synesthesia: touching textures or surfaces produces colors,” beside a hand giving off color. The text at the bottom reads “Synesthesia is a reminder that perception is a construct of the brain, not just what the senses detect.” This is an artistic image, not a scientific diagram.
Credit: AI-generated image created by Michael Miguel with ChatGPT (OpenAI).
From a Peer
“For me, the weekdays have colors. Monday is blue. Wednesday is orange. It’s not something I decide. It’s just there. I didn’t know it was unusual until I was 20.”
Sensation and Perception
Sensation is your sense organs detecting energy, like light hitting your retina. Perception is your brain organizing and interpreting those signals into something meaningful, like recognizing a friend’s face. The same sensation can lead to different perceptions in different people.
One Event, Many Senses
Say you saw a car crash. You’d see the cars hit, hear the bang, and maybe smell burning rubber. But you wouldn’t think, “I saw a crash and heard a crash.” You’d just think, “There was a crash.” Your brain blends the senses into one event. This is multimodal perception: combining information from more than one sense into a single experience.
Scientists used to study the senses one at a time, as if each worked alone. Research now shows they usually work together. In the rest of this module, you’ll see how your brain decides which signals belong together, why combining senses helps, and how mismatched signals can fool you.
Close to Home
Some of this research happens close by. Dr. Lorin Lachs, a psychology professor at Fresno State, studies how people combine seeing and hearing speech. He also wrote one of the open textbook chapters this module is adapted from.