The question

An organism confronts an environment which has a range of alternative possible states. The organism itself has a range of possible states, a range of possible behavioral or developmental choices. The alternative environmental states have consequences for the organism's chances of surviving and reproducing, and the right organic choice for one environmental state is not the right choice for another. The organism receives imperfect information about the actual state of the environment, as a consequence of correlations between environmental conditions which matter to it and environmental conditions which directly affect the periphery of its body. Under what conditions is it best for the organism to make use of this information, and adopt a flexible behavioral or developmental strategy, choosing its state in accordance with what it perceives, and under what conditions is it best for the organism to ignore the information, and always choose the same option, come what may?

This chapter and the next will discuss this problem with the aid of some simple mathematical tools. The aim of the present chapter is to describe abstractly some of the circumstances in which it is best to be a smart, flexible organism and some circumstances under which it is best to be unresponsive and rigid. This should tell us something about the value of cognition, as cognition is conceived here as a device making possible extensive flexibility and adaptibility to local conditions.

Another kind of complexity

In Chapters 7 and 8 we looked at the situation of an individual faced with a complex environment, and described some conditions under which it is best for the individual to meet this environmental complexity with a complex organic response. This is not the only type of organic system which can be complex or simple; another is the population. This chapter is about population-level responses to environmental complexity, and the relation between individual-level and population-level responses.

Complexity in the case of individuals was understood as heterogeneity. Complexity in populations will be understood the same way. A complex population is one which contains a diversity of types or forms. A simple population is homogeneous. The models of Chapters 7 and 8 examined a particular kind of individual-level complexity: the ability of an individual to do a variety of different things in different circumstances. In this chapter we will look at a particular case of population-level complexity: heterogeneity or “polymorphism” in the genetic make-up of the population. We will also investigate the relations between two different realizations of biological complexity, the relations between (i) simple populations of complex individuals, and (ii) complex populations of simple individuals.

The first part of this chapter is only indirectly relevant to the environmental complexity thesis. The aim is an understanding of a pattern of externalist explanation of complexity in biology, and certain work on genetic polymorphism will be discussed as a case study.

It is hard to think of any significant aspect of our lives that is not influenced by what we have learned in the past. The world looks and sounds the way it does because as infants we learned to partition it up in certain meaningful ways: we see familiar faces rather than meaningless blobs of colour and hear words rather than noise. Similarly, we behave in the ways we do because we have learned from past experience that our various actions have certain specific consequences. Like many topics of psychological inquiry, the importance of learning can perhaps best be realised by considering what life is like for people who have learning difficulties. Consider the case of Greg, a patient described by Sacks (1992), who became profoundly amnesic as a result of a benign brain tumour that was removed in 1976. Although his memory for events from his early life was almost completely normal, Greg remembered virtually nothing that had happened to him from 1970 onwards and appeared quite unable to learn anything new. He continued to believe, for instance, that Lyndon Johnson was the American President. In 1991 he was taken to a rock concert given by a group that he had been a great fan of in the 1960s, and despite sitting through the concert in rapture and recalling many of the songs, by the next morning he had no memory of the concert. More distressingly, when told of his father's death he was immeasurably sad but forgot the news within a few minutes. He was unable to learn that his father was no longer alive, and relived his grief anew every time he was told the news.

In the Introduction I argued that human associative learning can best be understood by considering three questions. What does the system do? How, in broad informational terms, does it do it? And how is this achieved at the mechanistic level? We have answered the first question by suggesting that to a first approximation the system does what a statistician would do: that is to say, it computes the degree of conditional contingency between events as defined by the metric ΔP. The second question is answered (again to a first approximation) by reference to the memorisation of instances, with stimuli being represented in a multidimensional psychological space and with interstimulus similarities being an exponential function of distance in the space. We now turn to our final question: what is the mechanism of learning? What sort of mechanism computes contingency, and represents associative knowledge in terms of memorised instances? In this chapter we will examine how contemporary associationist learning systems attempt to answer this final question.

I also pointed out in the Introduction that enthusiasm for associationist accounts of human learning waned somewhat in the 1960s and 1970s. Partly, this was due to impatience with the highly constrained tasks that researchers in the verbal learning tradition employed, but partly it was also due to the apparent inability of associationist theories to cope with more complex examples of human learning and cognition. For instance, a number of extremely sophisticated analyses had been undertaken of the difference between novices and experts performing various skills such as playing chess or remembering strings of digits (e.g. Chase and Ericsson, 1981; Chase and Simon, 1973).

If we take the commonsense view that the human associative learning system has evolved for adaptive purposes, then it is immediately clear that the major benefits learning affords an organism are the ability to make predictions about events in the environment and the ability to control them. If it is possible to predict that a certain event signals either impending danger such as a predator, or imminent reward such as access to food, then appropriate action can be taken to avoid the danger or extract maximum benefit from the reward. It has been common, particularly in discussions of animal conditioning, to interpret learning from the perspective of the benefit it brings the organism.

Of course, this is not to say that learning is always beneficial, and the high incidence of maladaptive behaviours, such as phobias in humans that can be traced to prior learning episodes, attests to this fact. Nevertheless, it seems plausible that such learned behaviours emerge from a system that fundamentally exists to exploit and benefit from regularities that exist in the world, whether they be signal–outcome or action–outcome regularities. For instance, when Pavlov's dogs learned to salivate to a bell that signalled food, it is likely that they benefited from the increased digestibility and hence nutritional benefit of the food. When a child learns that saying ‘juice’ reliably produces a rewarding drink, it has acquired the ability to control a small but important aspect of its environment.

In this chapter we will try to establish the degree to which human learning is appropriately adapted to the environment.

The picture of concept learning that emerges from the previous chapters is of a rather passive process in which instances are encoded in memory as a result of weight adjustments in an adaptive network system. This is passive in the sense that so long as the subject attends to the stimuli, the hypothesised processes operate automatically on the incoming information. But it has commonly been argued that in some circumstances a different, active process can operate whereby a person considers various hypotheses concerning relationships between events, modifies or rejects inadequate hypotheses, and in short tries to induce a rule describing the relationship between stimuli and outcomes. In this chapter we consider the evidence that the account of associative learning discussed in the previous chapters is incomplete and needs to be supplemented by an additional and possibly independent rule-learning mechanism.

Before considering the evidence and nature of this rule-learning process, it is necessary first to consider what exactly we mean by a ‘rule’. This concept has, to put it mildly, been a source of some debate and confusion amongst psychologists and philosophers. On the surface, the definition of a rule seems unproblematic: we simply say that a rule is a principle that specifies definitively whether an object or event is of a particular sort or not. For instance, if an object has four sides of equal length lying in a plane and with right-angles between them, then it is a square. Any object conforming to this principle is a square, and any object that violates the principle is not a square.

The place of learning theory in psychology has fluctuated dramatically over the last few decades. At the height of the behaviourist era in the 1940s and 1950s, many people would have identified learning as the single most important topic of investigation in psychology. By the 1970s, though, the picture was very different. A large number of influential psychologists had come to regard learning theory as a sterile field that made little contact with the realities of human cognition, and instead their interest switched towards such topics as knowledge representation and inference. Lately, things have come full circle, and the expanding new field of connectionism has restored learning theory to centre stage.

My aim in this book has been to have a fresh look at learning theory. I believe that the changing fortunes of the field have occurred because psychologists have often come to the topic of learning with certain deep conceptions of the mind already in place. But I suggest that rather than seeing learning as a topic to be annexed by and interpreted in terms of whatever the current fashionable theory happens to be, it is far more profitable for traffic to flow in the opposite direction. Let us commence by asking some general questions about learning, and see what sort of mind we end up with.

Someone coming to the field of learning afresh is likely to be disconcerted by the apparent incompatibility of a number of theoretical approaches and even terminologies.

In the last chapter we established that, to a first approximation, the human learning system behaves normatively. In attempting to answer the question ‘What is the system doing?’ (the first of our three questions), we have found that associative learning corresponds reasonably well to the prescriptions of contingency theories. In reaching this conclusion, we have remained agnostic about how the system actually works; all we have shown is that the behaviour it yields in associative learning tasks is roughly what a statistician utilising the notion of contingency would prescribe. In the present chapter we begin our consideration of how the system achieves this end. Here, we ask the second question, ‘What sort of information is acquired during the course of a learning experience?’. In the next chapter, we will ask exactly how at the mechanistic level this information is acquired.

We begin by considering the phenomenon of generalisation, which represents one of the principal challenges to any theory of learning. Having learned something about one stimulus, how will acquired knowledge determine responding to some further stimulus? Generalisation is of interest not just because it is something we would like our theories of learning to explain, but also because it provides data that may tell us about the way in which information is represented. Two quite different views of the form of information underlying associative learning have been embodied in prototype and instance theories, and for these theories generalisation is a central issue. They attempt to describe how learning takes place in situations where there is considerable stimulus variation from trial to trial, and where the ability to generalise to new stimuli perceptibly different from ones already encountered is essential.

Introduction

The relationships between steroid hormones and brain function have been envisioned mostly within the framework of endocrine mechanisms, as responses elicited by secretory products of steroidogenic endocrine glands, borne by the bloodstream, and exerting actions on the brain.

In fact, the brain is a target organ for steroid hormones. Intracellular receptors involved in the regulation of specific gene transcription have been identified in neuroendocrine structures, with each class of receptor having a unique distribution pattern in the complex anatomy of the brain (Fuxe et al. 1981; McEwen 1991a). Mechanisms involving nuclear receptors account for most steroid-induced feedback and many behavioral effects, for the regulation of the synthesis of several neurotransmitters, hormone-metabolizing enzymes, and hormone and neuromediator receptors, and for the organizational effects on neural circuitry that occur during development and persist into adulthood.

Local target tissue metabolism is an important factor in the mechanism of action of sex steroid hormones. Not only might such metabolism be involved in the regulation of intracellular hormone levels, but it might also provide an essential contribution to the cellular response. The brain is a site of extensive steroid metabolism. Aromatization and 5α-reduction represent major routes of androgen metabolism (Naftolin et al. 1975; Celotti et al. 1979; McLusky et al. 1984). The importance of these two pathways lies in the fact that they give rise to metabolites with considerable biological activity and thus are involved in the mechanism by which circulating androgens influence neuroendocrine function and behavior.

Introduction

It should not have been surprising that gonadal steroids exert powerful influences on cell–cell interactions in the magnocellular hypothalamoneurohypophyseal system (HNS) of the rat, but somehow it was. Since this system is responsible for the manufacture and release of oxytocin during parturition and lactation, times when the levels of circulating gonadal steroids show dramatic variations, some steroid involvement in the functioning of HNS could have been anticipated. Perhaps such anticipation was dulled by the observation that the dynamic interactions taking place among the cells of the HNS also occurred in response to manipulations of the animal's hydrational state, which have not been associated traditionally with variations in gonadal steroid output. However, gonadal steroids appear to exert some control over the cellular mechanisms that release both oxytocin and vasopressin in response to dehydration. Estrogens and androgens, under comparable conditions, often have opposite effects on the HNS. This chapter reviews the main structure–function relationships of the HNS, the dynamics of these relationships under physiological conditions of altered peptide hormone demand, and some of the roles possibly played in these functions by gonadal steroids in both males and females.

The magnocellular HNS

The magnocellular HNS is constituted chiefly by the supraoptic (SON) and paraventricular (PVN) nuclei, accessory nuclei in the anterior hypothalamus, and the neurohypophysis or neural lobe (NL) of the posterior pituitary to which the neurons of those hypothalamic nuclei send axonal projections (Fig. 18.1).

Sexual behavior of male rodents depends heavily on the actions of testosterone and its metabolites. One means by which testosterone may facilitate copulation is by altering the release and/or effectiveness of neurotransmitters. This chapter integrates information and speculation concerning the multiple roles of one such neurotransmitter, dopamine, in the regulation of male rat sexual behavior.

The three major dopamine systems that are important for male sexual behavior, including sexual motivation and genital reflexes, will be reviewed. Then, evidence suggesting that testosterone influences dopamine activity in the medial preoptic area (MPOA) and nucleus accumbens (NAc) will be discussed. Recent results suggesting that a novel messenger molecule, nitric oxide, might affect both dopamine release and sexual behavior will be presented, as will preliminary evidence suggesting that stimulation of the D1 dopamine receptor promotes copulation-induced expression of the immediate early gene c-fos in the MPOA. Finally, a model summarizing some of the events related to central dopamine release will be described.

The problem of neural coordination of behavior

The execution of a behavior as complex as copulation requires exquisite coordination of neural activity in numerous sites. Olfactory, visual, auditory, and somatosensory stimuli elicit a precisely timed and coordinated motor sequence that includes locomotor pursuit, mounting and pelvic thrusting, penile erection and insertion, and ultimately ejaculation and postejaculatory grooming and quiescence. Steroid hormones facilitate this process by biasing sensorimotor integration, so that a sexually relevant stimulus is more likely to elicit a sexual response. Most effects of steroid hormones on sexual behavior are relatively long term.

The discovery that the hypothalamus is responsible for controlling both reproductive hormones and behavior suggested various mechanisms by which hormonal and behavioral cycles are inexorably linked, and even co-regulated. While our knowledge about this process has grown dramatically, our understanding of the essential control circuits that operate during normal reproduction, or fail in abnormal functioning, is still limited. Various neurotransmitter candidates have been proposed as essential elements of the systems that regulate reproduction. However, few are involved in so many aspects of reproduction as are the opioid peptides, which play a critical or supporting role in (a) controlling hormonal cycling in females (Akabori and Barraclough 1986; Kalra 1985; Wiesner et al. 1984), (b) regulating reproductive behavior in males (Hughes et al. 1988; Matuszewich and Dornan 1992; Myers and Baum 1979) and females (Pfaus and Pfaff 1992; Sirinathsinghji 1986; Wiesner and Moss 1986a), and even (c) modulating mesolimbic dopamine release mediated by reinforcing sexually relevant olfactory stimuli (Mitchell and Gratton 1991).

Gonadal steroid regulation of hypothalamic opioids represents an important feedback system by which to control reproduction. Hypothalamic (opioid) circuits regulate hormonal releasing hormones that control pituitary secretion. This regulation in turn affects gonadal steroid hormones, which act centrally to alter opioid function and facilitate reproductive behavior. Since such feedback is vitally important for the regulation of hormonal and behavioral events during the estrous cycle, most of this discussion will be limited to opioid action in females. Many of the experiments that we will describe utilized models of hormone manipulation to investigate natural regulation of hypothalamic opioid systems in animals, primarily rodents.

The expression of sexually differentiated patterns of behavior is a characteristic of many vertebrate species and often correlates with sex differences in the relative abundance of neurons in brain regions thought to control such behaviors. In general, two fundamental processes determine the number of neurons that survive into adulthood. First, changes in the number of neuroblasts formed in the ventricular zone can occur in response to mechanisms that are intrinsic to a particular population of cells or that are controlled by extrinsic factors such as cell–cell interactions, neuronal growth factors, and circulating hormones. Second, similar extrinsic cellular and hormonal factors can determine the number of cells of a particular lineage that reach their permanent destination, establish appropriate connections, and achieve a regionally specific functional phenotype. Sex steroid hormones can affect both processes but appear to exert their most pronounced influences on neuronal development during a restricted perinatal critical period (Arnold and Gorski 1984; Breedlove 1986; Goy and McEwen 1980; Harris and Levine 1965; Rhees et al. 1990). Thus, treatment of female neonates with sex steroids during the first few postnatal days alters the number of neurons residing in certain nuclei, as well as the morphology, synaptology, and neurotransmitter expression of individual neurons (Arai et al. 1986; Arnold and Jordan 1988; De Vries 1990; Gorski 1985; Raisman and Field 1973; Simerly 1989, 1991).