In this paper we describe an experiment in sentence processing which was intended to relate two properties of syntactic structures that have received much discussion in linguistics and psychology (see references cited in the next section). First, some syntactic structures, such as the passive construction, require more processing effort than corresponding structures which express the same grammatical relations. Passive sentences in particular have been the subject of much experimental work. Second, it is clear, as was observed by Jespersen (1924), that the difference between active and passive sentences has something to do with focus of attention on a particular constituent, the grammatical subject. And the consequences of the difference of focus of attention is in some way related to the context formed by the discourse in which the sentence occurs. In this experiment we wanted to study syntactic structures which might have similar properties to the passive/active construction, so as to define exactly what features of passive sentences are responsible for their observed greater processing demands and definition of focus of attention, or sentence topic. One of the bases of the experiment, underlying the hypotheses we wanted to test, is that processing load and definition of sentence topic are related in some way.

We combined sentences exemplifying five different syntactic constructions with context sentences having different relations to the target sentences, and measured reaction time for reading and understanding the second or target sentence. The results show that there is a fairly consistent relationship of processing load for the other constructions as well as passive, and that overall processing time is sensitive to both syntactic structure and contextual information.

Language is a system for encoding and transmitting ideas. A theory that seeks to explain linguistic phenomena in terms of this fact is a functional theory. One that does not misses the point. In particular, a theory that shows how the sentences of a language are all generable by rules of a particular formal system, however restricted that system may be, does not explain anything. It may be suggestive, to be sure, because it may point to the existence of an encoding device whose structure that formal system reflects. But, if it points to no such device, it simply constitutes a gratuitous and wholly unwelcome addition to the set of phenomena to be explained.

A formal system that is decorated with informal footnotes and amendments explains even less. If I ask why some phenomenon, say relativization from within the subject of a sentence, does not take place in English and am told that it is because it does not take place in any language, I go away justifiably more perplexed than I came. The theory that attempts to explain things in this way is not functional. It tells me only that the source of my perplexity is more widespread than I had thought. The putative explanation makes no reference to the only assertion that is sufficiently self-evident to provide a basis for linguistic theory, namely that language is a system for encoding and transmitting ideas.

Kimball's parsing principles (Kimball, 1973), Frazier and Fodor's Sausage Machine (Frazier and Fodor, 1978; Fodor and Frazier, 1980) and Wanner's augmented transition network (ATN) model (Wanner, 1980) have tried to explain why certain readings of structurally ambiguous sentences are preferred to others, in the absence of semantic information. The kinds of ambiguity under discussion are exemplified by the following two sentences.

1. Tom said that Bill had taken the cleaning out yesterday.

2. John bought the book for Susan.

For sentence (1), the reading ‘Yesterday Bill took the cleaning out’ is preferred to ‘Tom spoke yesterday about Bill taking the cleaning out.’ Kimball (1973) introduced the principle of Right Association (RA) to account for this kind of preference. The basic idea of the Right Association principle is that, in the absence of other information, phrases are attached to a partial analysis as far right as possible.

For sentence (2), the reading ‘The book was bought for Susan’ is preferred to ‘John bought a book that had been beforehand destined for Susan.’ To account for this preference, Frazier and Fodor (1978) introduced the principle of Minimal Attachment (MA), which may be summarized as stating that, in the absence of other information, phrases are attached so as to minimize the complexity of the analysis.

Much of the debate about the formulation and interaction of such principles is caused by their lack of precision and, at the same time, by their being too specific. I propose a simple, precise, and general framework in which improved versions of Right Association and Minimal Attachment can be formulated.

Speakers of certain bilingual communities systematically produce utterances in which they switch from one language to another possibly several times, in the course of an utterance, a phenomenon called code switching. Production and comprehension of utterances with intrasentential code switching is part of the linguistic competence of the speakers and hearers of these communities. Much of the work on code switching has been sociolinguistic or at the discourse level, but there have been few studies of code switching within the scope of a single sentence. And until recently, this phenomenon has not been studied in a formal or computational framework.

The discourse level of code switching is important; however, it is only at the intrasentential level that we are able to observe with some certainty the interaction between two grammatical systems. These interactions, to the extent they can be systematically characterized, provide a nice framework for investigating some processing issues both from the generation and the parsing points of view.

There are some important characteristics of intrasentential code switching which give hope for the kind of work described here. These are as follows. (1) The situation we are concerned with involves participants who are about equally fluent in both languages. (2) Participants have fairly consistent judgments about the “acceptability” of mixed sentences. (In fact it is amazing that participants have such acceptability judgments at all.) (3) Mixed utterances are spoken without hesitation, pauses, repetitions, corrections, etc., suggesting that intrasentential code switching is not some random interference of one system with the other.

Parsing in traditional grammar

Like so many aspects of modern intellectual frameworks, the idea of parsing has its roots in the Classical tradition; (grammatical) analysis is the Greek-derived term, parsing (from pars orationis ‘part of speech’) the Latin-derived one. In this tradition, which extends through medieval to modern times,

1. parsing is an operation that human beings perform,

2. on bits of natural language (usually sentences, and usually in written form),

3. resulting in a description of those bits, this description being itself a linguistic discourse (composed of sentences in some natural language, its ordinary vocabulary augmented by technical terms);

4. moreover, the ability to perform this operation is a skill,

5. acquired through specific training or explicit practice, and not possessed by everyone in a society or to equal degrees by those who do possess it,

6. and this skill is used with conscious awareness that it is being used.

Parsing, in the traditional sense, is what happens when a student takes the words of a Latin sentence one by one, assigns each to a part of speech, specifies its grammatical categories, and lists the grammatical relations between words (identifying subject and various types of object for a verb, specifying the word with which some other word agrees, and so on). Parsing has a very practical function:

LANGUAGES AND REWRITING SYSTEMS

Both natural and programming languages can be viewed as sets of sentences—that is, finite strings of elements of some basic vocabulary. The notion of a language introduced in this section is very general. It certainly includes both natural and programming languages and also all kinds of nonsense languages one might think of. Traditionally, formal language theory is concerned with the syntactic specification of a language rather than with any semantic issues. A syntactic specification of a language with finitely many sentences can be given, at least in principle, by listing the sentences. This is not possible for languages with infinitely many sentences. The main task of formal language theory is the study of finitary specifications of infinite languages.

The basic theory of computation, as well as of its various branches, such as cryptography, is inseparably connected with language theory. The input and output sets of a computational device can be viewed as languages, and—more profoundly—models of computation can be identified with classes of language specifications, in a sense to be made more precise. Thus, for instance, Turing machines can be identified with phrase-structure grammars and finite automata with regular grammars.

FINITE AUTOMATA

A finite automaton is a strictly finitary model of computation. Everything involved is of a fixed, finite size and cannot be extended during the course of computation. The other types of automata studied later have at least a potentially infinite memory. Differences between various types of automata are based mainly on how information can be accessed in the memory.

A finite automaton operates in discrete time, as do all essential models of computation. Thus, we may speak of the “next” time instant when specifying the functioning of a finite automaton.

The simplest case is the memoryless device, where, at each time instant, the output depends only on the current input. Such devices are models of combinational circuits.

In general, however, the output produced by a finite automaton depends on the current input as well as on earlier inputs. Thus, the automaton is capable (to a certain extent) of remembering its past inputs. More specifically, this means the following.

The automaton has a finite number of internal memory states. At each time instant i it is in one of these states, say qi. The state qi + 1 at the next time instant is determined by qi and by the input at given at time instant i. The output at time instant i is determined by the state qi (or by qi and ai, together).

A GENERAL MODEL OF COMPUTATION

As is true for all our models of computation, a Turing machine also operates in discrete time. At each moment of time it is in a specific internal (memory) state, the number of all possible states being finite. A read-write head scans letters written on a tape one at a time. A pair (q, a) determines a triple (q′, a′, m) where the q's are states, a's are letters, and m (“move”) assumes one of the three values l (left), r (right), or 0 (no move). This means that, after scanning the letter a in the state q, the machine goes to the state q′ writes a′ in place of a (possibly a′ = a, meaning that the tape is left unaltered), and moves the read-write head according to m.

If the read-write head is about to “fall off” the tape, that is, a left (resp. right) move is instructed when the machine is scanning the leftmost (resp. rightmost) square of the tape, then a new blank square is automatically added to the tape. This capability of indefinitely extending the external memory can be viewed as a built-in hardware feature of every Turing machine. The situation is depicted in Figure 4.1.

BACKGROUND AND CLASSICAL CRYPTOSYSTEMS

It might seem strange that a chapter on cryptography appears in a book dealing with the theory of computation, automata, and formal languages. However, in the last two chapters of this book we want to discuss some recent trends. Undoubtedly, cryptography now constitutes such a major field that it cannot be omitted, especially because its interconnections with some other areas discussed in this book are rather obvious. Basically, cryptography can be viewed as a part of formal language theory, although it must be admitted that the notions and results of traditional language theory have so far found only few applications in cryptography. Complexity theory, on the other hand, is quite essential in cryptography. For instance, a cryptosystem can be viewed as safe if the problem of cryptanalysis—that is, the problem of “breaking the code”—is intractable. In particular, the complexity of certain number-theoretic problems has turned out to be a very crucial issue in modern cryptography. And more generally, the seminal idea of modern cryptography, public key cryptosystems, would not have been possible without an understanding of the complexity of problems. On the other hand, cryptography has contributed many fruitful notions and ideas to the development of complexity theory.