Visual space is the experience of space by an aware observer. It is the subjective counterpart of the space of physical objects. There is a long history in philosophy, and later psychology of writings describing visual space, and its relationship to the space of physical objects. A partial list would include René Descartes, Immanuel Kant, Hermann von Helmholtz, William James, to name just a few.

Object Space and Visual Space.

Space of physical objects

The location and shape of physical objects can be accurately described with the tools of geometry. For practical purposes the space we occupy is Euclidean. It is three-dimensional and measurable using tools such as rulers. It can be quantified using co-ordinate systems like the Cartesian x,y,z, or polar coordinates with angles of elevation, azimuth and distance from an arbitrary origin.

Space of visual percepts

Percepts, the counterparts in the aware observer's conscious experience of objects in physical space, constitute an ordered ensemble or, as Ernst Cassirer explained,^[1] Visual Space can not be measured with rulers. Historically philosophers used introspection and reasoning to describe it. With the development of Psychophysics, beginning with Gustav Fechner, there has been an effort to develop suitable experimental procedures which allow objective descriptions of visual space, including geometric descriptions, to be developed and tested. An example illustrates the relationship between the concepts of object and visual space. Two straight lines are presented to an observer who is asked to set them so that they appear parallel. When this has been done, the lines are parallel in visual space A comparison is then possible with the actual measured layout of the lines in physical space. Good precision can be achieved using these and other psychophysical procedures in human observers or behavioral ones in trained animals.^[2]

Visual Space and the Visual Field

The visual field, the area or extent of physical space that is being imaged on the retina, should be distinguished from the perceptual space in which visual percepts are located, which we call visual space. Confusion is caused by the use of Sehraum in the German literature for both. There is no doubt that Ewald Hering and his followers meant visual space in their writings.^[3]

Spaces: formal, physical, perceptual

The fundamental distinction was made by Rudolf Carnap between three kinds of space which he called formal, physical and perceptual.^[4] Mathematicians, for example, deal with ordered structures, ensembles of elements for which rules of logico-deductive relationships hold, limited solely by being not self-contradictory. These are the formal spaces. According to Carnap, studying physical space means examining the relationship between empirically determined objects. Finally, there is the realm of what students of Kant know as Anschauungen, immediate sensory experiences, often awkwardly translated as "apperceptions", which belong to perceptual spaces.

Visual space and geometry

Geometry is the discipline devoted to the study of space and the rules relating the elements to each other. For example, in Euclidean space the Pythagorean theorem provides a rule to compute distances from Cartesian coordinates. In a two-dimensional space of constant curvature, like the surface of a sphere, the rule is somewhat more complex but applies everywhere. On the two-dimensional surface of a football, the rule is more complex still and has different values depending on location. In well-behaved spaces such rules used for measurement, called metrics, are classically handled by the mathematics invented by Riemann. Object space belongs to that class.

To the extent that it is reachable by scientifically acceptable probes, visual space as defined is also a candidate for such considerations. The first and remarkably prescient analysis was published by Ernst Mach^[5] in 1901. Under the heading On Physiological as Distinguished from Geometrical Space Mach states that "Both spaces are threefold manifoldnesses" but the former is "...neither constituted everywhere and in all directions alike, nor infinite in extent, nor unbounded." A notable attempt at a rigorous formulation was made in 1947 by Rudolf Luneburg, who preceded his essay on mathematical analysis of vision^[6] by a profound analysis of the underlying principles. When features are sufficiently singular and distinct, there is no problem about a correspondence between an individual item A in object space and its correlate A' in visual space. Questions can be asked and answered such as "If visual percepts A',B',C' are correlates of physical objects A,B,C, and if C lies between A and B, does C' lie between A' and B' ?" In this manner, the possibility of visual space being metrical can be approached. If the exercise is successful, a great deal can be said about the nature of the mapping of the physical space on the visual space.

On the basis of fragmentary psychophysical data of previous generations, Luneburg concluded that visual space was hyperbolic with constant curvature, meaning that elements can be moved throughout the space without changing shape. One of Luneburg's major arguments is that, in accord with a common observation, the transformation involving hyperbolic space renders infinity into a dome (the sky). The Luneburg proposition gave rise to discussions and attempts at corroborating experiments, which on the whole did not favor it.^[7]

Basic to the problem, and underestimated by Luneburg the mathematician, is the likely success of a mathematically viable formulation of the relationship between objects in physical space and percepts in visual space. Any scientific investigation of visual space is colored by the kind of access we have to it, and the precision, repeatability and generality of measurements. Insightful questions can be asked about the mapping of visual space to object space ^[8] but answers are mostly limited in the range of their validity. If the physical setting that satisfies the criterion of, say, apparent parallelism varies from observer to observer, or from day to day, or from context to context, so does the geometrical nature of, and hence mathematical formulation for, visual space.

All these arguments notwithstanding, there is a major concordance between the locations of items in object space and their correlates in visual space. It is adequately veridical for us to navigate very effectively in the world, deviations from such a situation are sufficiently notable to warrant special consideration. visual space agnosia is a recognized neurological condition, and the many common distortions, called geometrical-optical illusions, are widely demonstrated but of minor consequence.

Neural representation of space

Fechner's inner and outer psychophysics

Its founder, Gustav Theodor Fechner defined the mission of the discipline of psychophysics as the functional relationship between the mental and material worlds—in this particular case, the visual and object spaces—but he acknowledged an intermediate step, which has since blossomed into the major enterprise of modern neuroscience. In distinguishing between inner and outer psychophysics, Fechner recognized that a physical stimulus generates a percept by way of an effect on the organism's sensory and nervous systems. Hence, without denying that its essence is the arc between object and percept, the inquiry can concern itself with the neural substrate of visual space.^{[citation needed]}

Retinotopy and beyond

Two major concepts dating back to the middle of the 19th century set the parameters of the discussion here. Johannes Müller emphasized that what matters in a neural path is the connection it makes,^{[citation needed]} and Hermann Lotze, from psychological considerations, enunciated the principle of local sign^[clarify].^{[citation needed]} Put together in modern neuroanatomical terms they mean that a nerve fiber from a fixed retinal location instructs its target neurons in the brain about the presence of a stimulus in the location in the eye's visual field that is imaged there. The orderly array of retinal locations is preserved in the passage from the retina to the brain, and provides what is aptly called a "retinotopic" mapping in the primary visual cortex. Thus in the first instance brain activity retains the relative spatial ordering of the objects and lays the foundations for a neural substrate of visual space.

Unfortunately simplicity and transparency ends here. Right at the outset, visual signals are analyzed not only for their position, but also, separately in parallel channels, for many other attributes such as brightness, color, orientation, depth. No single neuron or even neuronal center or circuit represents both the nature of a target feature and its accurate location. The unitary mapping of object space into the coherent visual space without internal contradictions or inconsistencies that we as observer automatically experience, demands concepts of conjoint activity in several parts of the nervous system that is at present beyond the reach of neurophysiological research.

Place cells

Though the details of the process by which the experience of visual space emerges remain opaque, a startling finding gives hope for future insights. Neural units have been demonstrated in the brain structure called hippocampus that show activity only when the animal is in a specific place in its environment.^[10]

Space and its content

Only on an astronomical scale are physical space and its contents interdependent, This major proposition of the general theory of relativity is of no concern in vision. For us, distances in object space are independent of the nature of the objects.

But this is not so simple in visual space. At a minim an observer judges the relative location of a few light points in an otherwise dark visual field, a simplistic extension from object space that enabled Luneburg to make some statements about the geometry of visual space. In a more richly textured visual world, the various visual percepts carry with them prior perceptual associations which often affect their relative spatial disposition. Identical separations in physical space can look quite different (are quite different in visual space) depending on the features that demarcate them. This is particularly so in the depth dimension because the apparatus by which values in the third visual dimension are assigned is fundamentally different from that for the height and width of objects.

Even in monocular vision, which physiologically has only two dimensions, cues of size, perspective, relative motion etc. are used to assign depth differences to percepts. Looked at as a mathematical/geometrical problem, expanding a 2-dimensional object manifold into a 3-dimensional visual world is "ill-posed," i.e., not capable of a rational solution, but is accomplished quite effectively by the human observer.

The problem becomes less ill-posed when binocular vision allows actual determination of relative depth by stereoscopy, but its linkage to the evaluation of distance in the other two dimensions is uncertain (see: stereoscopic depth rendition). Hence, the uncomplicated three-dimensional visual space of every-day experience is the product of many perceptual and cognitive layers superimposed on the physiological representation of the physical world of objects.