This text is the result of two complementary experiences which I had in 1987 and 1988. The first was the opportunity, which I owe to Claude Godrèche, of delivering, in a pleasant seaside resort in Brittany, a series of lectures on the theory of neural networks. Claude encouraged me to write the proceedings in the form of a pedagogical book, a text which could be useful to the many people who are interested in the field. The second was a one-year sabbatical which I spent at the Hebrew University of Jerusalem on a research program on spin glasses and neural networks. The program was initiated by the Institute for Advanced Studies and organized by a team of distinguished physicists and biologists, namely Moshe Abeles, Hanoch Gutfreund, Haim Sompolinsky and Daniel Amit. Throughout the year, the Institute welcomed a number of researchers who shed different lights on a multi-faceted subject. The result is this introduction to the modeling of neural networks.

First of all, it is an introduction. Indeed the field evolves so fast that it is already impossible to have its various aspects encompassed within a single account.

Also it is an introduction, that is a peculiar perspective which rests on the fundamental hypothesis that the information processed by the nervous systems is encoded in the individual neuronal activities. This is the most widely admitted point of view. However, other assumptions have been suggested.

This chapter is not a course on neurobiology. As stated in the title, it is intended to gather a few facts relevant to neural modeling, for the benefit of those not acquainted with biology. The material which is displayed has been selected on the following accounts. First of all, it is made of neurobiological data that form the basic bricks of the model. Then it comprises a number of observations which have been subjects for theoretical investigations. Finally, it strives to settle the limits of the current status of research in this field by giving an insight on the huge complexity of central nervous systems.

Three approaches to the study of the functioning of central nervous systems

Let us assume that we have a very complicated machine of unknown origin and that our goal is to understand its functioning. Probably the first thing we do is to observe its structure. In general this analysis reveals a hierarchical organization comprising a number of levels of decreasing complexity: units belonging to a given rank are made of simpler units of lower rank and so on, till we arrive at the last level of the hierarchy, which is a collection of indivisible parts.

The next step is to bring to light what the units are made for, how their presence manifests itself in the machine and how their absence damages its properties. This study is first carried out on pieces of the lowest order, because the functions of these components are bound to be simpler than those of items of higher rank.

The scope of this book covers the use of synchrotron radiation in the X-ray analysis of single crystals of proteins, nucleic acids and viruses. The impact of this new X-ray source with its polychromatic nature and associated high intensity and fine collimation has brought important advances in the field of macromolecular crystallography. It has extended structure determinations to higher resolution, allowed use of smaller samples and larger, more complex, unit cells. Several new methods have come to the fore and some old methods have been revived. Firstly, the Laue method is being developed and used now for quantitative, time resolved analysis of structure. Secondly, variable wavelength methods are being developed and used for phase determination for metallo-proteins or derivatised proteins. Thirdly, the diffuse scattering is being measured more easily and procedures for analysing it are being developed in order to study molecular flexibility; hopefully its use will be increasingly widespread but at present it is the least developed of these three methods. The availability of the synchrotron is a very modern development but it has reopened fundamental questions of which crystallographic method to use. It is interesting to wonder what von Laue, W. H. and W. L. Bragg and the other early pioneers would have made of the synchrotron instead of starting with the X-ray emission tube.

The original X-ray diffraction experiment was based on an idea of von Laue and conducted by Friedrich and Knipping (Friedrich et al 1912). It earned von Laue the Nobel Prize for Physics in 1914. The basis of the idea was that if X-rays were electromagnetic waves then their wavelengths might be of the same order as the interatomic separation in crystals and diffraction would be observed. The original diffraction photograph was from a crystal of copper sulphate.

The essential feature of the Laue method, as it became called, is that the incident X-ray beam is polychromatic and the crystal sample is held stationary. All the X-rays emitted by the emission tube and passing through the tube exit window are allowed to impinge onto the sample; no special filtering or monochromatisation is employed. The Bremsstrahlung continuum and the characteristic emission lines constitutes the incident spectrum of X-rays. This beam hits the stationary crystal and the spots making up the diffraction pattern arise from the different wavelengths. A given reflecting plane in the crystal extracts from the beam the particular wavelength which allows constructive interference or reflection to occur. In contrast to the angular rocking width of a reflection in the monochromatic rotating crystal method, in the Laue method each reflection is stimulated by a small range of wavelengths whose mean wavelength lies somewhere in the broad range of incident wavelengths.

The pace of technological change in the field of macromolecular crystallography is quite breathtaking. The pursuits of particle physics have led to particle accelerators tailored for the production of synchrotron X-radiation of incredible intensity, geometric quality and wide tunable range.

The exploitation of this radiation, particularly the brilliance and use of short λ's, has made virus crystal data collection routine from difficult samples; although it is at present necessary to use hundreds of crystals in the gathering of just one data set. Maybe the use of ultra-short wavelength beams (≈0.33 Å) from a harmonic of an undulator could be harnessed to improve the lifetime of one such sample sufficiently to give a complete data set. Much larger macromolecular assemblies are currently under study, such as the ribosome, which possess little or no symmetry (unlike viruses) and are therefore more difficult to solve.

The revival of the Laue technique has been made possible by the polychromatic nature of the emitted spectrum and the associated high brightness. The study of time resolved phenomena in biological structures (as well as chemistry and solid state physics) with this method is a major research and development effort that is under way.

The interpretation of the diffuse scattering from macromolecular crystals will provide increasing information on the mobility of macromolecules.

Macromolecular crystallography is a very powerful method used to study complex biological systems. The structures of a wide variety of proteins, nucleic acids and their assemblies have been determined at atomic or near-atomic resolution. As a result, a detailed understanding has been gained of various living processes such as enzyme catalysis, the immune response, the encoding of hereditary information, viral infection and photosynthesis.

The first X-ray diffraction photograph ever taken was from copper sulphate by Friedrich and Knipping at von Laue's suggestion in 1912. In the following year W. L. Bragg deduced the crystal structure of sodium chloride from Laue photographs. A variety of relatively small molecular structures were then solved at an increasing rate.

The first X-ray diffraction pictures of a protein crystal were taken in 1934 by Bernal in Cambridge, but in those days the data quality was crude and the techniques for deriving a crystal structure of a macromolecule from the X-ray data were not sufficiently developed. The advent of the computer has been a critical development.

The first protein structures to be determined were myoglobin and haemoglobin in the late 1950s by Kendrew et al (1958) and Perutz et al (1960). From then on a steadily increasing number of protein structures have become known.

The understanding of biology has been transformed from being at a gross anatomical level to a molecular level. A major contribution to this has come from the application of physical techniques, especially X-ray diffraction, for the determination of structures which have provided, therefore, explanations for many key functions of organisms. The molecular basis of heredity followed from the discovery of the double-helix structure of deoxyribonucleic acid (DNA). The mechanism of action of many different protein molecules can be explained on the basis of their three-dimensional structures, for example, oxygen transport and storage, enzymes, membrane proteins and the immune response. Also the means by which viral infection takes place is currently being unravelled. The application of all this structural information has started with the engineering of new proteins with enhanced or modified functions and also the rational design of drugs. On the horizon is the detailed structure determination of the ribosome, which is the cell organelle involved in protein synthesis. A list of books dealing with the structure and function of macromolecules in detail is given in the bibliography, section 5.

The scale of atoms, molecules and macromolecules is illustrated in figure 3.1. The dimensions of bond lengths are of the order of 10-10m or 1 Å and determine, therefore, the resolving power and the wavelength required in the technique of X-ray diffraction applied to determining molecular structure.

DEFINITION OF REQUIREMENTS

The instrumentation requirements for macromolecular crystallography at the synchrotron are quite diverse and technically exacting. The diversity arises because of the different classes of experiment, namely:

1. (a) routine data collection usually at a fixed wavelength;

2. (b) variable single or multiple wavelength anomalous dispersion measurements;

3. (c) time resolved crystallography;

4. (d) small crystals.

There are certainly needs common to each category.

In order to use the synchrotron X-radiation effectively, the ‘white beam’ of photons diverging from the source must be collected by beam line optical element (s), and brought to a focus with a size approximately equal to the protein crystal size. The sample must be centred in the beam and oriented on a goniostat of some kind. The diffraction pattern has to be measured accurately and efficiently with a detector. In the case of monochromatic experiments the beam line optical scheme will include a monochromator of which there are several common types. There are some special needs for each experimental class.

‘Routine’ data collection often involves the measurement of relatively weak, high resolution diffraction data. Critical design goals here include a very high intensity at the specimen and a short wavelength (often ≤0.9 Å) monochromatised beam to reduce radiation damage, which affects these high angle data first.

Particle accelerators were originally developed for high energy physics research into the subatomic structure of matter. The SR, which was produced in circular electron accelerators (‘synchrotrons’) was a nuisance by-produce — an energy loss process. The early stages of the utilisation of SR were therefore parasitic on the high energy physics machines whose parameters were, of course, not optimised for SR. However, SR became well recognised in its own right as a major tool in research in biology, chemistry and physics. Particle accelerators began to be designed specifically for SR production with parameters optimised solely for this work, e.g. continuous beams with long lifetimes, stable source positions and magnetic insertion devices to produce radiation of specific properties; the Daresbury Synchrotron Radiation Source (SRS) which came on-line in 1981 was the first dedicated, high energy source. Table 4.1 gives a list of storage ring X-ray sources. All modern SR sources are storage rings rather than synchrotrons. The particles used may be electrons or positrons.

Studies of the properties of the radiation from accelerated charges extend over the last 100 years. Extensive theoretical work on the radiation effects in circular electron accelerators has been done by Schwinger (1949) and Sokolov and Ternov (1968) and the theory is reviewed in Jackson (1975).

Not all the diffracted photons from a crystal end up in the Bragg reflections from specified (hkl) planes. Indeed, for quite a large number of macromolecular crystals the non-Bragg diffraction or diffuse scattering is strong in intensity. The diffuse scattering is due to a breakdown in the periodicity of the crystal and carries information on the mobility and flexibility of the molecules in the crystal. There are text-books describing diffuse scattering from small molecule crystals such as Amorós and Amorós (1968) and Wooster (1962).

At the synchrotron the long exposure times used in the measurement of the high resolution Bragg data on film or IP also automatically give the diffuse scattering and reveal a diversity of diffuse background patterns from different crystals. These observations have stimulated considerable interest in trying to understand and interpret these features. Of great interest to the molecular biologist is the relationship between macromolecular structure and function. Recent years have shown that besides the static/time-averaged structural information, appreciation of the molecular flexibility and dynamics is essential. Usually this information has been derived from the crystallographic atomic thermal parameters and also from molecular dynamics simulations (see, e.g., McCammon (1984)) which yield individual atomic trajectories. A characteristic feature of macromolecular crystals compared to small molecule crystals, however, is that their diffraction patterns extend to quite limited resolution even employing SR.

This appendix is based, with permission, on my sections in the new International Tables for Crystallography, Volume C (editor A. J. C. Wilson, 1991).

There are text-books which concentrate on almost every diffraction geometry. References to these books are given in the respective sections in the following pages. However, in addition, there are several books which contain details of diffraction geometry. Blundell and Johnson (1976) described the use of the various diffraction geometries with the examples taken from protein crystallography. There is an extensive discussion and many practical details to be found in the text-books of Stout and Jensen (1968, 1989), Woolfson (1970), Glusker and Trueblood (1971, 1985), Vainshtein (1981) and McKie and McKie (1986), for example. A collection of early papers on the diffraction of X-rays by crystals involving, inter alia, experimental techniques and diffraction geometry, can be found in Bijvoet, Burgers and Hägg (1969, 1972). A collection of recent papers on primarily protein and virus crystal data collection via the rotation film method and diffractometry can be found in Wycoff, Hirs and Timasheff (1985); detailed references are also made to this volume later.

In this appendix which deals with monochromatic methods, the convention is adopted that the Ewald sphere takes a radius of unity and the magnitude of the reciprocal lattice vector is λ/d.