How the Brain Sees 3D (Even when we don't want it to)

Our visual system faces a major challenge: a 3D world is projected onto a 2D mosaic of photoreceptors on our retinas, and must be reconstructed into a 3D image for our perception. How does this happen? 

There are two kinds of informational cues your brain uses to discern a 3D image from a 2D image.  The first kind of cue is "pictorial" and comes in three types:
1) Perspective (using the relative sizes of objects you already know about to determine relative depth)
2) Shading (using your knowledge that shadows indicate that an object is 3D)
3) Occlusion (using your knowledge that if one object is blocking another, one must be in front of the other)

You'll notice that all three types of pictorial cues require that we have prior knowledge about the world.  Neuroscientists are still unsure of exactly how the brain uses this information, but what is most important to note is that pictorial cues don't actually invoke the spatial computation functions of the brain; they are more like assumptions our brain makes based on our previous experiences. Thus, our brain can sometimes trick us into viewing a 2D image as 3D.  These pictorial cues are what artists make use of to render an apparent 3D image on a 2D surface.

The more valid cue the brain uses to interpret the world's 3D nature is called a "geometrical" cue, in which we view in object from more than one vantage point.  We owe the ability to do this in an instantaneous fashion to the physical separation of our two eyes.  Usually, an object in space will be projected onto a different location of each eye's retina, and the measure of how much the image is separated is called the "disparity".  There is a small area of our visual field called the "geometric horopter" where objects in space fall on the same location in each eye's retina, and thus there is zero disparity and zero ability to use geometrical cues to discern 3D space.

There is a small region of the visual cortex in the brain called MT that contains neurons which are tuned for different degrees of disparity.  When an object in space registers a certain disparity, the neurons tuned for that preferred disparity will fire at a maximum intensity.  It is this process that informs the brain of 3D space.

This, incidentally, is how 3D glasses work in a movie theater.  The screen actually presents two different images taken from slightly different positions simultaneously, and the glasses filter the images in such a way (usually by each lens being a different color) so that one eye receives one image and the other eye receives the other image.  This artificially facilitates disparity and our perception of 3D.

Point: Our brain is not always a reliable carrier of information from the outside world to our conscious perception.