We experience the jerky saccades whenever we move our eyes from one object to another, but if we’re visually following something in motion our eye movement is as smooth as a waxed bowling ball. This makes evolutionary sense; if you’re tracking a moving object in nature it’s usually prey or a threat, so you’d need to keep focused on it constantly. But we can do it only when there’s something moving that we can track. Once this object leaves our field of vision, our eyes jerk right back to where they were via saccades, a process termed the Optokinetic reflex. Overall, it means the brain can move our eyes smoothly, it just often doesn’t.
But why when we move our eyes do we not perceive the world around us as moving? After all, it all looks the same as far as images on the retina are concerned. Luckily, the brain has a quite ingenious system for dealing with this issue. The eye muscles receive regular inputs from the balance and motion systems in our ears, and use these to differentiate between eye motion and motion in or of the world around us. It means we can also maintain focus on an object when we’re in motion. It’s a system that can be confused though, as the motion-detection systems can sometimes end up sending signals to the eyes when we’re not moving, resulting in involuntary eye movements called nystagmus. Health professionals look out for these when assessing the health of the visual system, because when your eyes are twitching for no reason, that’s not great. It’s suggestive of something gone awry in the fundamental systems that control your eyes. Nystagmus is to doctors and optometrists what a rattling in the engine is to a mechanic; might be something fairly harmless, or it might not, but either way it’s not meant to be happening.
This is what your brain does just working out where to point the eyes. We haven’t even started on how the visual information is processed.
Visual information is mostly relayed to the visual cortex in the occipital lobe, at the back of the brain. Have you ever experienced the phenomenon of hitting your head and “seeing stars”? One explanation for this is that impact causes your brain to rattle around in your skull like a hideous bluebottle trapped in an egg cup, so the back of your brain bounces off your skull. This causes pressure and trauma to the visual processing areas, briefly scrambling them, and as a result we see sudden weird colors and images resembling stars, for want of a better description.
The visual cortex itself is divided into several different layers, which are themselves often subdivided into further layers.
The primary visual cortex, the first place the information from the eyes arrives in, is arranged in neat “columns,” like sliced bread. These columns are very sensitive to orientation, meaning they respond only to the sight of lines of a certain direction. In practical terms, this means we recognize edges. The importance of this can’t be overstressed: edges mean boundaries, which means we can recognize individual objects and focus on them, rather than on the uniform surface that makes up much of their form. And it means we can track their movements as different columns fire in response to changes. We can recognize individual objects and their movement, and dodge an oncoming soccer ball, rather than just wonder why the white blob is getting bigger. The discovery of this orientation sensitivity is so integral that when David Hubel and Torsten Wiesel discovered it in 1981, they ended up with a Nobel Prize.9
The secondary visual cortex is responsible for recognizing colors, and is extra impressive because it can work out color constancy. A red object in bright light will look, on the retina, very different from a red object in dark light, but the secondary visual cortex can seemingly take the amount of light into account, and work out what color the object is “meant” to be. This is great, but it’s not 100 percent reliable. If you’ve ever argued with someone over what color something is (such as whether a car is dark blue or black) you’ve experienced first hand what happens when the secondary visual cortex gets confused.
It goes on like this, the visual-processing areas spreading out further into the brain, and the further they spread from the primary visual cortex the more specific they get regarding what it is they process. It even crosses over into other lobes, such as the parietal lobe containing areas that process spatial awareness, to the inferior temporal lobe processing recognition of specific objects and (going back to the start) faces. We have parts of the brain that are dedicated to recognizing faces, so we see them everywhere. Even if they’re not there, because it’s just a piece of toast.
These are just some of the impressive facets of the visual system. But perhaps the one that is most fundamental is the fact that we can see in three dimensions, or “3D” as the kids are calling it. It’s a big ask, because the brain has to create a rich 3D impression of the environment from a patchy 2D image. The retina itself is technically a “flat” surface, so it can’t support 3D images any more than a blackboard can. Luckily, the brain has a few tricks to get around this.
Firstly, having two eyes helps. They may be close together on the face, but they’re far enough apart to supply subtly different images to the brain, and the brain uses this difference to work out depth and distance in the final image we end up perceiving.
It doesn’t just rely on the parallax resulting from ocular disparity (that’s the technical way of saying what I just said) though, as this requires two eyes to be working in unison, but when you close or cover one eye, the world doesn’t instantly convert to a flat image. This is because the brain can also use aspects of the image delivered by the retina to work out depth and distance. Things like occlusion (objects covering other objects), texture (fine details in a surface if it’s close but not if it’s far away) and convergence (things up close tend to be much further apart than things in the distance; imagine a long road receding to a single point) and more. While having two eyes is the most beneficial and effective way to work out depth, the brain can get by fine with just one, and can even keep performing tasks that involve fine manipulation. I once knew a successful dentist who could see out of only one eye; if you can’t manage depth perception, you don’t last long in that job.