Introduction

In 1952, Barlow wrote a paper in which he spelled out the implications of the diffraction limit for compound eyes, where the small size of each lens makes diffraction a much more severe problem than it is in eyes like our own with a single large lens. This paper, modestly entitled ‘The size of ommatidia in apposition eyes’, contained a sentence which seemed to me when I first read it as a graduate student wonderfully immodest. He wrote, ‘Imagine the problems concerned in designing an eye for an insect’, and then went on to work out the relation between the resolution of a compound eye and its size, and the size of its component parts. People in those days rarely thought like that; it was one thing to try to sort out an organ's function, but quite another to set about its design. A generation before genetic engineering made such ideas almost commonplace, Barlow's assertion that one could understand natural structures at that kind of level seemed exciting and pleasingly impious.

It is because we understand the behaviour of light so well that it is possible to entertain such ideas about eyes. For livers and kidneys, or even ears and noses, no correspondingly exact body of knowledge exists to permit a thorough analysis of the physical constraints on their design. Thus it is possible to attribute function to structure with more precision in the eye than in any other sense organ; and to a comparative physiologist one of the beauties of studying eyes is the confidence one has that differences between eyes are important. As Gordon Walls put it, ‘everything in the vertebrate eye means something’ (Walls, 1942).

The study of binocular mechanisms in visual cortex is just one of the many areas of vision influenced by Horace Barlow. How do cortical neurons utilise positional disparities to signal depth (Barlow, Blakemore & Pettigrew, 1967) and what developmental processes regulate the binocularity of cortical neurons (e.g. Barlow, 1975)? Although cortex is the first site in the geniculo-striate pathway in which binocular neurons are found, the fundamental basis of binocular vision is established much earlier in the visual pathway. It is the existence of bilateral retinal projections that ensures one point in visual space projects to one locus in the brain and so achieves the binocular congruence that is exploited by cortical neurons.

The binocular representation of each visual hemifield on the opposite side of the brain depends on the partial decussation of the retinal projections. For visual congruence, not only must the retinal decussation line coincide with the representation of the vertical meridian of the visual field but also the mapping rules of the resulting crossed and uncrossed projections must differ (Thompson, 1979, 1984). As Sperry (1963) recognised, there has to be bilateral symmetry in the representation of the nasotemporal retinal axis and a developmental mechanism that ensures retinal ganglion cells from two different retinal regions, one nasal and one temporal, terminate in a common locus in the target nucleus. These rather stringent conditions for adult retinal projections are not fulfilled in the developing animal. In this chapter, I shall describe the features of retinal projections that permit binocular mappings, the development of some of these features and how they can be influenced by early experimental manipulations.

On the basis of clinical experience with treatment of amblyopia, it had been long been suspected that the central visual pathways possessed a degree of plasticity in early life that did not exist in adulthood (Duke-Elder & Wybar, 1973). The first insight into the site and nature of this plasticity was obtained by Wiesel & Hubel (1963) in their pioneering study of the effects of early monocular deprivation on the kitten visual cortex. Whereas periods of deprivation imposed in the first months of life caused very substantial shifts of ocular dominance of cortical cells towards the nondeprived eye, similar periods of deprivation imposed on adult cats caused no measurable changes of ocular dominance. The time course of the sensitive period for the effects of monocular deprivation was documented more precisely in subsequent investigations that examined in a more systematic fashion the effects of short periods of deprivation imposed on both cats (Hubel & Wiesel, 1970; Olson & Freeman, 1980; Cynader, Timney & Mitchell, 1980; Jones, Spear & Tong, 1984) and monkeys (e.g. Hubel, Wiesel & LeVay, 1977; Blakemore, Garey & Vital-Durand, 1978; Von Noorden & Crawford, 1979; LeVay, Wiesel & Hubel, 1980) of different ages. While there is general consensus that the visual cortex of the cat is most sensitive to monocular deprivation during the fourth week of postnatal life, there is substantial disagreement concerning the rate of decline of susceptibility beyond this point.

Background

Development of the visual projection

The neural connections between the retinal ganglion cells and the higher order visual neurons in the midbrain (e.g. the optic tectum) or in the forebrain (e.g. lateral geniculate nucleus) develop in coherent topographic patterns. To establish the visual projections, individual ganglion cells send out their axons (the optic fibers) which grow towards the optic disc near the centre of the retina. The axons of ganglion cells exit the eyeball at its posterior pole and form a thick ensheathed bundle, the optic nerve. The growing tips of these axons select particular routes to reach their appropriate target zones along the visual pathways and eventually form synapses with higher order visual neurons.

Since the complexities of the visual pathways vary from one species to another we will discuss here a relatively simple example of visual projections in the goldfish. The optic fibers of goldfish make a complete cross at the optic chiasm and invade only the contralateral lobe of the optic tectum in the midbrain. Before they enter the rostral pole of the tectum the ingrowing axons from the ventral retina segregate from those originating from the dorsal area of the retina. The former select the medial branch of the optic tract and the latter the lateral branch. The optic fibers within the tectal tissue course through a superficial layer (the stratum opticum) towards the caudal pole along the spheroidal circumferences of the optic tectum. Most growing tips of retinal ganglion cell axons terminate in the subjacent layer (the stratum fibrosum et griseum superficiale) and form synapses with visual neurons of the tectum.

Introduction

In many electro-optical displays or photographic recordings visual detection is limited by some kind of pictorial noise. Practically all modern medical and military observation devices suffer from this problem occasionally or regularly. In this paper some psychophysical measurements will be described and discussed in order to evaluate the possible gain of noise cleaning techniques. In textbooks on image processing the justification of the various techniques usually is a matter of face-validity. Pictures are shown before and after image processing and then obviously look much better. However, there is, as far as we know, little experimental evidence of improved visual performance.

The problem of target detection in the presence of visible pictorial noise is the disentanglement of the two. Signal and noise must be separated on the basis of a priori knowledge about their differences. In this respect it is most helpful to analyse the noise into components that can be ordered according to their perceptual distance from the target. Obviously, spatial frequency analysis is attractive in this respect, both, because of its mathematical transparency and because of the present preference in the literature on visual performance. In line with this tradition visual performance is expressed in this paper in terms of contrast sensitivity for sine wave test gratings, measured as a function of spatial frequency in a number of pictorial noise conditions. Sine wave gratings may not be the most natural stimuli for the visual system, and it cannot be claimed beforehand that this angle of approach leads to a clear and complete insight into the interference of pictorial noise with the detection of arbitrary targets.

Introduction

Colour vision is a means of encoding the spectral reflectance of a surface. Thus the contours, edges or patterns which allow us to distinguish an object against its background may be seen both by virtue of their intensity variation and on the basis of their colour variation. Over the past three decades a relatively new approach to the investigation of colour vision has been emerging. This involves creating a stimulus from which the variations in intensity have been removed and so allowing it to be distinguished solely on the basis of its colour differences. Such stimuli are often termed ‘isoluminant’. In this chapter I aim to examine some of the difficulties associated with the use of colour-only stimuli and to assess their contribution to the understanding of the spatial coding of colour vision.

The problems of colour-only stimuli

Chromatic aberrations

There are considerable optical difficulties inherent in the use of chromatic stimuli, which have to be overcome in order to remove all luminance artifacts and ensure that the stimulus can be detected only on the basis of its colour variation. These arise largely from the two types of chromatic aberration of the eye. The chromatic difference of focus, the increasing power of the eye for shorter wavelengths of light, affects the relative contrasts of the sinusoidal component colours in a chromatic grating. (A chromatic grating can be considered as the sum of two luminance modulated gratings added in antiphase.) This is often corrected by using an achromatizing lens, a small pupil or by correcting the relative amplitudes of the component wavelengths. The second aberration is the chromatic difference of magnification.

Introduction: theories of perception

If you examine the history of ideas on perception during the last century or so you will notice that there have been three major trends in thinking:

1. Perception as unconscious inference. This view emphasizes that the visual image is inherently ambiguous and that the perceptual apparatus resolves ambiguities by using ‘intelligent’ processes that resemble conscious reasoning. The idea was originally put forward by Helmholtz (1867) and has more recently been revived by Gregory (1970) and Rock (1983).

2. Direct perception. Emphasizes the richness of the visual input and argues that ambiguity exists only in contrived laboratory situations but not in the ‘real world’. A vast amount of information is implicit in the visual image and perception is achieved not by making this information explicit but by direct ‘resonance’. Unfortunately, it is never clearly specified how or where in the brain this resonance is supposed to occur.

3. […]

Improving the efficiency of vision brings distinct advantages. Improved spatial resolution and the better detection of small intensity differences allows an animal to resolve more objects and to see objects at a greater distance. These improvements in accuracy and resolution extend both the range and the richness of perception, so providing a greater return for an animal's investment in eye and brain. It follows that coding efficiency, that is the accuracy and fidelity with which information is gathered by the eye, and transmitted and processed by neurons, is an important biological factor. Consequently, the need for efficiency must shape visual processing, and considerations of efficiency can guide our understanding of vision (e.g. Hecht, Shlaer & Pirenne, 1942; Rose, 1972; Barlow, 1964). The dictates of coding efficiency are illustrated by our work on the blowfly retina (Laughlin & Hardie, 1978; Laughlin, 1981; Srinivasan, Laughlin & Dubs, 1982; Laughlin, Howard & Blakeslee, 1987). Both our experimental-theoretical approach to retinal coding in the blowfly, and our major conclusions have recently been reviewed (Laughlin, 1987; 1989). In this article I will briefly summarise our arguments and explore some of the general implications of the finding that retinal circuits are designed to promote coding efficiency.

The constraints imposed upon vision by the properties of natural images and the construction of neurons are readily apparent in the retina. A detailed optical image is projected onto the photoreceptors and transformed into the first neural image – the distribution of receptor potentials across the photoreceptor mosaic. The properties of natural signals and photoreceptors severely limit the amplitude of the signal in this electrical “image”.

Introduction

Horace Barlow makes only occasional forays into the field of colour vision (Barlow, 1958,1982), but when he does, he always leaves us with much to think about. In his 1982 paper ‘What causes trichromacy?’, he gave us a novel way of considering the information content of a coloured spectrum: he expressed the detailed structure of the colour spectrum in terms of its Fourier components and he treated the three photopigments (Fig. 11.1) as low-pass filters that would differentially attenuate the different Fourier components. Owing to the broad bandwidth of the filters, the visual system is insensitive to the fine structure of the colour spectrum; that is to say, if the amplitude of a stimulus varies periodically with wavelength and if the period of this modulation is small, then the response of the visual system will show little variation as the phase of the modulation is changed (Barlow, 1982).

In considering his main question – that of why our colour vision is three-dimensional – Barlow was led also to ask several secondary questions: ‘Why do the photopigments have such broad bandwidths?’, ‘Are broad bandwidths deleterious to hue discrimination?’ and ‘Why are the peak sensitivities of the pigments so asymmetrically placed in the spectrum?’ We hope that the present paper may say something in answer to these secondary questions. We first put forward a general view of the early stages of colour vision, the view that it consists of two subsystems, one recently overlaid on a much earlier one; and then we review some experimental work on wavelength discrimination, work that bears on the two subsystems of colour vision.

By the end of the nineteenth century the mood of sensory physiology was marvellously mechanistic. Sir Charles Bell and Johannes Müller set the scene in the 1830s with their evidence for a direct and automatic link between the stimulation of particular sensory nerves and the resulting perceptions; but even then there was a dissenting voice. In volume 3 of his Treatise on Physiological Optics Hermann von Helmholtz drew attention to many problems in a simplistic interpretation of the relation between sensory messages and perception. He pointed out that ‘without any change of the retinal images, the same observer may see in front of him various perceptual images in succession’. ‘We are not simply passive to the impressions that are urged on us, but we observe.’ Equally, the generalities of experience that we can express in single words correspond to an infinity of precise patterns of sensory stimulation: ‘the species “table” includes all individual tables and expresses their common peculiarities’. Helmholtz concluded that the processes underlying perception were essentially inductive, learned, but below the level of consciousness: ‘by their peculiar nature they may be classed as conclusions, inductive conclusions unconsciously formed’.

No wonder that Helmholtz's revolutionary views were anathema to many physiologists and philosophers of the time. They flew in the face of the prevailing notion of the innate basis of knowledge and experience, which linked such heavyweights as Ewalt Hering and Immanuel Kant. But Helmholtz's appeal to the inferential, learned nature of perception echoed the much earlier views of the empiricist philosophers, most especially George Berkeley, and presaged the ideas of the buoyant new field of cognitive psychology.

Introduction

One of Horace Barlow's major contributions to neurobiology was the discovery of directionally selective cells in the rabbit retina (Barlow & Levick, 1965) and hence of the neural mechanism for detecting one of the most fundamental and primitive of all visual stimuli, namely motion. It thus seems appropriate that my contribution to this dedicatory volume should be devoted to the same general topic of motion detection, although in the cortex, not the retina, and in the monkey, not the rabbit. My emphasis will be mainly anatomical. Although the occasion may have demanded a more physiological contribution, many of the problems raised by the anatomical facts are themselves physiological and ones in which, as I know, Horace Barlow is deeply interested and to which he continues to make contributions (Barlow, 1981). That I should have allowed myself some speculative asides in the following pages was motivated by the fact that Barlow was never concerned by exposing oneself to possible ridicule (Phillips, Zeki & Barlow, 1984) but welcomed instead informed speculations, particularly when they could be experimentally tested, as many of mine can. Some require simple experiments, others more complex ones. That even the simple ones have not been done, either by myself or others, is really due to nothing more than the exigencies of time, though in writing this article I have often wished, presumably like others, that I had the result of this experiment or that. I can only hope that the results of some of these experiments will be available in the coming years and will come to delight Barlow.

In a recent review entitled ‘Why can't the eye see better?’, Horace Barlow (1986) discussed how optical factors, photoreceptor characteristics and the dynamic range of retinal ganglion cells limit visual performance. Barlow's question, like all good riddles, demands a shift in perspective before it can be tackled. In this essay we pose a complementary puzzle, ‘is there more than meets the eye?’ Although there is no doubt that much remains to be discovered about the retina, we question whether the apparent simplicity of retinal function critically underestimates the sophistication of early stages of visual processing.

The complexity of visual coding by retinal ganglion cells is often presented in terms of those receptive field characteristics that may provide the neural substrate for psychophysical phenomena. Thus the centre–surround organization of concentric cells is related to the coding of chromatic or luminance contrast; the On and Off pathways underpin the efficient signalling of increased light and darkness; the spatial and temporal properties of X and Y ganglion cells seem matched to the requirements of pattern vision and motion detection. Although this approach has inherent appeal, the sophistication that psychophysicists and central physiologists demand of retinal function is, in fact, rather limited.

Hierarchical concepts of visual processing deny to retinal function those characteristics that are presently perceived to be intrinsic to cortical function. For example, the long-range horizontal interactions between cortical modules are thought to underlie such diverse processes as vernier acuity and figure–ground discrimination. In the retina, however, the many amacrine connections beyond the classic receptive field are credited with little more than producing the periphery effect.

Introduction

Late 19th century studies of the brain provided evidence that part of the cerebral cortex was made up of primary sensory receiving areas and primary motor areas. These comprised a relatively small portion of the total surface area of the cortex and with the exception of some specialized regions (such as Broca's area), the functional relevance of the other parts of the cortex remained a mystery. Vision was relegated to a small portion of the human cortex, occupying about 15 per cent of the total surface. Surrounding this primary area were secondary and tertiary zones, often referred to as ‘association cortex’.

Very recent advances in neuroanatomy and neurophysiology, however, have changed this picture dramatically. Thanks to the pioneering work of Allman & Kaas (1974), Zeki (1978), and Van Essen (1985), we now know that monkey visual cortex contains at least 19 separate maps of the visual field and according to a recent review by Maunsell & Newsome (1987) visual processing occupies about 60 per cent of the cortical surface!

This overwhelming dominance of vision in relation to other functions should serve as a reminder that, as generally practiced, the current subdisciplines of visual perception and psychophysics may be too narrow to capture the wealth of processing involved. Threshold psychophysics, especially, has been preoccupied with just the earliest aspects of vision. It has neglected the seemingly intractable questions such as the nature of visual experience, pattern recognition, visual memory, attention, etc.

Meanwhile the neurophysiologists have been making recordings from diverse regions of the visual cortex which could be closely related to these higher functions.

Introduction

It can be argued that stereopsis has evolved on at least three separate occasions; in mammals, in birds and in machines (see Bishop & Pettigrew, 1986; Pettigrew, 1986). While the evolution of machine algorithms for stereopsis is still occurring, such algorithms have reached a level of sophistication where comparison with animals can be made (see Poggio & Poggio, 1984). When the mammalian, avian and machine systems for stereopsis are compared, some fundamental similarities in organisation emerge despite the many structural differences. In particular, all three systems appear to delay image processing until after information from both eyes is brought together. The surprising fact that stereopsis can occur without prior monocular pattern recognition was already recognised from psychophysical studies on humans and was reinforced by detailed studies of the neural machinery underlying stereopsis in both mammals and birds, not to mention the more recent machine algorithms which also avoid complex form analysis prior to extraction of the stereoscopic information. This chapter deals with the possible reasons behind this common feature of successful stereoscopic systems. Two general lines of argument will be advanced;

1. i. If the primary role of stereopsis is not the judgement of depth but rather the detection of edges which are invisible to monocular inspection, it is necessary that this edge-detection step precede any attempts at form analysis. This is a teleological argument which can be supported by an examination of the evolutionary context within which some avian groups have acquired stereopsis while other close relatives have not.

2. […]

Introduction

Many visual decisions are fundamentally statistical in nature. This can be because of noisy physical processes such as quantum fluctuations (Rose, 1948; Barlow, 1962), or the indeterminacy of the environmental causes of the images on the eye's retinae. The problem with noisy imaging is that a single source can give rise to many images. The challenge is to integrate image information in such a way as to discount the noise. The problem with indeterminacy of the environmental causes of the image is that there are too many possible scene descriptions that could have given rise to the image data. If one can find a statistical description of a visual task, one can often devise an observer that handles both of these types of uncertainty optimally. This ideal observer is one that makes optimal use of the data available, given a statistical description of the task, and a criterion to satisfy (e.g. minimal classification error). The ideal observer comprises a quantitative computational theory for a given task, in the sense of Marr (1982), and as such is an abstract model of the information processing requirements of the task, independent of the particular algorithm or implementation. The statistical approach to understanding biological vision, thus, involves three parts: (1) formulate the statistically optimal observer for a visual task; (2) devise a biological or psychophysical model of performance; (3) compare experimental data to ideal and model. Typically, one compares the biological model with the data, leaving out an analysis of the statistical requirements imposed by the task as specified by an ideal observer.