Making Sense of the Senses, Part 2

By Wil Forbis

Jan 1, 2011

The Our Wacky Brain collection:

In part one of this article, we explored the physiological architecture of the sensory organs such as the eyes, the ears, the nose etc.. We learned that sensory receptors such as the retinal cells of the eyes or tastebuds of the tongue get activated by elements of the environment such as light waves or chemicals. These receptors send information to a nerve which goes to the brain. Once inside the brain, this information is passed to the different modules of the brain responsible for "interpreting" the specific sensory information. Visual information gets passed to the visual cortex at the back of the brain, for example.

That provides a 50,000 foot view of how the sensory organs work. However, we need to understand that these organs are "merely" input devices. They collect information which is routed the brain, but they do not process the information. To use a computer metaphor: the mouse, keyboard and network cables that plug into your laptop are like the eyes, ears and nose "plugged into" your brain.The brain is the processor, RAM and hard drive. A mouse disconnected from a computer is just as worthless as an eyeball and dangling optic nerve disconnected from a brain.

The question, "what does the brain contribute to our ability to sense the world?" is what this article is about. But first, let me tell you a story about lizards.

I live in San Diego, California, and it is lizard country. During the summers, it's not uncommon to see dozens of chameleonlike six-inch reptiles scampering out of the way as you walk down the sidewalk. Several months ago, while hiking up a mountain near my residence, I stumbled across a giant rattlesnake laying in the sun, a noticeable bulge in his belly indicating that he had just made a meal of a mouse or rabbit.

About a month ago, I was rummaging around in my room looking for a particular paper binder. I spied it under my clothes dresser, pulled it onto my lap, opened it and found myself face-to-face with... what? It looked like some kind of iguana. But my brain struggled at the possibility. I vaguely recalled someone, at some point in my life, giving me a fake rubber lizard. Was that what this was? Seconds passed, and I continued to stare at this thing, dumbfounded, until I finally realized that it was indeed a live lizard. I yelled something like "yeeaahhh!" and tossed the folder and lizard across the room. (Eventually I decided the the lizard was not going to leave of his own accord so I managed to grab him by the tail and deposit him outside.)

Later, I reflected on that moment where my brain was struggling to figure out just what it was I was looking at. All the information was there at my disposal --- my eyes were working fine --- but, for a half second or so, I simply couldn't understand what I was seeing. This makes an interesting point about the senses. It's not enough to merely sense input --- it's not enough to be able to see waves of light, hear sound vibrations, smell chemicals etc. For the senses to really work, we need to correlate the incoming information to our internal understanding of the world. When I saw that lizard, my eyeballs did their job correctly, but my brain struggled with the act of mapping that information to the concept of "lizard." (Why did I struggle? In part, because we recognize what we expect to see. If we're walking around the neighborhood and see our neighbor John, there's nothing really surprising about the experience. If we happen to be in Osaka Japan, and look over to see someone who looks a lot like John, we do a double take. What is John doing here? (Or, what is this lizard doing here?) Again, our eyes are not to blame --- it's the expectations imposed on the world by our brain that confounds us.)

The neuroscientist and writer Richard Restak has referred to this internal understanding of the world as "a subjective perceptual model created for us by our brain."[1] How did we learned to map our external reality (as sensed by our senses) to our internal perceptual model? That's what learning is. You learned what a dog was when your mom pointed at one in a picture book and said "dog." When you saw a real dog and he barked, you associated that sound with dogs. When he ran up to you and started to lick your face, you associated that behavior, and the tactile sensation of having your face licked, with your concept of "dog." Repeat this process for hundreds of thousands of objects, as well as ideas and concepts, and you start to get a pretty good sense of the world.

[1] The Modular Brain, page 133

We tend to take for granted the process by which our brain maps sensory information to objects. Most of us started out with functioning senses, and spent our formative years developing our understanding of the objects in the world. But what happens when a person has a working subjective perceptual model, and then "turns on" a sense? The neurologist Oliver Sacks recounted such a scenario in his New Yorker article "To See and Not See" which described a 50-year-old man who had been blind most of his life, and via cataract surgery, became sighted. One would think the restoration of sight would be a magic gift, but as Sacks noted, for the patient Virgil, this was not the case. With the cataracts gone the outside visual world flowed into Virgil's brain, but he could not map what he saw to objects he had only experienced with his other senses. During Virgil's initial moment of sight...

... he had no idea what he was seeing. There was light, there was movement, there was color, all mixed up, all meaningless, a blur. Then out of the blur came a voice that said, "Well?" Then, and only then, [Virgil] said, did he finally realized that this chaos and light and shadow was a face --- and, indeed, the face of the surgeon.

Sacks contemplated the dilemma of this moment.

... when Virgil opened his eye, after being blind for 45 years --- having had little more than an infant's visual experience, and this long forgotten --- there were no visual memories to support a perception, there was no world of experience and meaning awaiting him. He saw, but what he saw had no coherence. His retina and optic nerve were active, transmitting impulses, but his brain could make no sense of them.

After regaining sight, Virgil struggled with seemingly basic components of seeing. He could see all the elements of a tree --- the leaves, the roots, the branches --- but had difficulty combining them into a single object. He struggled to understand shapes. Movement baffled him. He had to practice looking at household objects from different angles to gain the understanding that they were one single thing. And his eyes would fatigue much faster than a normal person. Eventually, Virgil lost his vision a second time, though the exact cause for this is unclear.

Virgil's case is a thought-provoking illustration of how integral the brain is to our ability to know the world. The sensory organs provide the raw data, but the brain does the computation and data crunching. Your eyes may indicate that there is a dog to your left but it is your brain that ties that to your past experience with dogs, your emotions about dogs (some people just aren't "dog people") and your predictions about what this dog might do (is he going to lick your face or attack?) Your brain also ties the information coming from all your sensory organs together. If you turn your head away from the dog, and hear a bark, it's your brain that decides that the sound is probably coming from the dog. If you smell a musty odor, it's your brain that associates that smell to the dog. And your brain doesn't just contextualize information about the outside world, but the world beneath your skin. If you're late for a meeting and feel sweaty and slightly nauseous, your brain understands this to be anxiety. (For more discussion of this point, see my article "what is emotion.")

Necker CubeHowever, not all contextualization of sensory information is learned -- quite a bit of it appears to be innate, programmed into our DNA. For example, let's consider the German gestalt psychologists of the early 1900s who noticed that the brain applies particular rules when interpreting visual data. For example, when looking at the famous Necker cube (a wireframe illustration of a box), a person's brain translates the flat two-dimensional image into either a three-dimensional box projecting either upwards towards the right or downwards towards the left. And, if you stare at the cube long enough, you can see it flip back and forth between the two interpretations. (Another great gestalt style visual illusion: the "turning the tables" graphic, in which different views of the same illustration of a table appear to have different widths. Additional crazy optical illusions can be found here.) Gestalt theoreticians determined that what we see is affected by hardcoded rules in the brain having to do with motion, shadow (and the presumed light source) and grouping. (It was by "faking out" these rules that the artist MC Escher came up with his memorable illustrations.)

Gestalt style rules apply to our sense of hearing as well. For example, when listening to a symphony we can pick out the trombones from the violins --- even if they're playing the same melody --- but we can't pick out the individual violins in the violin section. We are "programmed" to group similar sounds into one big "Omni-sound." And on a more basic level, all music (for that matter, all sound) is the result of brain interpretation. Air molecules vibrate 440 times a second, and we, for some reason, hear that as the musical note A. We sense the vibration of air molecules (using the cochlea as described in part one of this article) but the experience of the sound is purely psychological. Higher and lower pitches of sound --- dependent on faster or slower rates of vibration --- are also psychological constructs. So is our sense of loudness which corresponds to the amplitude with which air molecules vibrate. In this sense, if a tree falls in the forest and there is no one around to hear it, it does not make a sound. (For a deeper look at how our brain interprets music, I recommend reading chapters to and three of Daniel Levitan's book "This Is Your Brain on Music.")

(Why our brain evolved to impose the rules it does is a complex question. This review of the book "The Vision Revolution" includes interesting discussion on the topic. So does this article which examines our preferences for color grouping in art.)

The unfortunate case of blind Virgil makes clear that the functions of our neural circuitry are just as important to perceiving the world as our our sensory inputs. However, that neural circuitry is surprisingly malleable. The brains of blind people have been shown to use part of the visual cortex for sound perception. (Hence, deaf people often seem to be able to "hear better" than sighted people.) Recent observations have noted that after experiencing a stroke, a person's brain can (briefly) return to levels of plasticity it had during the learning intensive early years of life.

Of course, this neural circuitry can go amok. Phantom limb pain, or any sensation of excessive pain (possibly fibromyalgia) are examples of this. So are the visual and auditory hallucinations of people suffering from conditions like schizophrenia and epilepsy. But, on a more positive note, people can use their brains to "overwrite" some negative sensations. Think of the meditative mystic who can walk on hot coals and silence his mind to the screams of his scalding feet.

This implies an interesting question: can the brain be re-wired (via technology, psychiatry or meditation/therapy) to experience the world in new and different ways? Could you register light waves as tastes, or sounds as colors? People with the condition known as synesthesia already exhibit these behaviors, though it's due to the unusual way their brain was wired, not pure force of will. But perhaps technology will someday make the experience synesthesia available to everyone. It's also possible that people will be able to add new sensory organs (recall the woman mentioned in part one of this article who had affixed magnets to her fingertips) and the brain will wire itself to interpret these augmented abilities. For the hundred thousand plus years of man's history, we've been limited to the senses provided by nature. But things could radically change in the next hundred thousand years.


Next: what is creativity?

Columns - Features - Interviews - Fiction - GuestBook - Blogs
View for more sin and wackiness!!!

Email Publisher