Introduction

In the last decade substantial progress has been made in our understanding of restricted classes of Boolean circuits, in particular those restricted to have constant depth (Furst, Sipser, Saxe, Ajtai, Yao, Haiåstad, Razborov, Smolensky or to be monotone (Razborov, Andreev, Alon and Boppana, Tardos, Karchmer and Wigderson). The question arises, perhaps more urgently than before, as to what approaches could be pursued that might contribute to progress on the unrestricted model.

In this note we first argue that if P ≠ NP then any circuit-theoretic proof of this would have to be preceded by analogous results for the more constrained arithmetic model. This is because, as we shall observe, there are proven implications showing that if, for example, the Hamiltonian cycle problem (HC) requires exponential circuit size, then so does the analogous problem on arithmetic circuits. Since the set of valid algebraic identities in the latter model form a proper subset of those in the former, a lower bound proof for it should be strictly easier.

In spite of the above relationship the algebraic model is often regarded as an alternative, rather than a restriction of the Boolean model. One reason for this is that specific computations are usually understandable in one of these models, and not in both. In particular, the main power of the algebraic model derives from the possibility of cancellations, and it is usually difficult to express explicitly how these help in computing combinatorial problems.

Introduction

In recent years several methods have been developed for obtaining superpolynomial lower bounds on the monotone formula and circuit size of explicitly given Boolean functions. Among these are the method of approximations, the combinatorial analysis of a communication problem related to monotone depth and the use of matrices with very particular rank properties. Now it can be said almost surely that each of these methods would need considerable strengthening to yield nontrivial lower bounds for the size of circuits or formulae over a complete basis. So, it seems interesting to try to understand from the formal point of view what kind of machinery we lack.

The first step in that direction was undertaken by the author in. In that paper two possible formalizations of the method of approximations were considered. The restrictive version forbids the method to use extra variables. This version was proven to be practically useless for circuits over a complete basis. If extra variables are allowed (the second formalization) then the method becomes universal, i.e. for any Boolean function f there exists an approximating model giving a lower bound for the circuit size of f which is tight up to a polynomial. Then the burden of proving lower bounds for the circuit size shifts to estimating from below the minimal number of covering sets in a particular instance of “MINIMUM COVER”. One application of an analogous model appears in where the first nonlinear lower bound was proven for the complexity of MAJORITY with respect to switching-and-rectifiers networks.

Abstract

Let f be an arbitrary Boolean function depending on n variables and let A be a network computing them, i.e., A has n inputs and one output and for an arbitrary Boolean vector a of length n outputs f(a). Assume we have to compute simultaneously the values f(a1), …,f(ar) of f on r arbitrary Boolean vectors a1, …,ar. Then we can do it by r copies of A. But in most cases it can be done more efficiently (with a smaller complexity) by one network with nr inputs and r outputs (as already shown in Uhlig (1974)). In this paper we present a new and simple proof of this fact based on a new construction method. Furthermore, we show that the depth of our network is “almost” minimal.

Introduction

Let us consider (combinatorial) networks. Precise definitions are given in [Lu58, Lu65, Sa76, We87]. We assume that a complete set G of gates is given, i.e., every Boolean function can be computed (realized) by a network consisting of gates of G. For example, the set consisting of 2-input AND, 2-input OR and the NOT function is complete. A cost C(Gi) (a positive number) is associated with each of the gates Gi ∈ G. The complexity C(A) of a network A is the sum of the costs of its gates. The complexity C(f) of a Boolean function f is defined by C(f) = min C(A) where A ranges over all networks computing f.

By Bn we denote the set of Boolean functions {0, l}n → {0, 1}.

INTRODUCTION

Before turning to some selected issues in type theory, type-shifting and the semantics of noun phrases, which form the main topic of this chapter, I will briefly mention some other, very exciting topics of current research in formal semantics that are relevant to the theme of this book, but which do not receive the attention they deserve in what follows.

One is the centrality of context dependence, context change, and a more dynamic view of the basic semantic values. This view of meanings, in terms of functions from contexts to contexts, is important both for the way it helps us make better sense of such things as temporal anaphora, and also for the way it solves some of the crucial foundational questions about presuppositions and the principles governing presupposition projection. The work of Irene Heim [86] is central in this area.

Another crucial area that will not be covered is the recently emerging emphasis on the algebraic structure of certain model-theoretic domains, such as entities, events or eventualities. As well as the work of Godehard Link [137] and Jan-Tore Lønning [140] in this area, referred to by Fenstad [54], I would also mention the attempt by Emmon Bach [6] to generalize the issue into questions about natural language metaphysics. An example of such a question is whether languages that grammaticize various semantic distinctions (such as mass versus count nouns) differently have different ontologies, or whether Link's use of Boolean algebras, which may be either atomic or not necessarily atomic, provides a framework in which to differentiate those structures that are shared from additional ones that only some languages express through grammar.

INTRODUCTION

In the linguistic study of syntax, various formats have been developed for measuring the combinatorial power of natural languages. In particular, there is Chomsky's well-known hierarchy of grammars which can be employed to calibrate syntactic complexity. No similar tradition exists in linguistic semantics, however. Existing powerful formalisms for stating truth conditions, such as Montague Semantics, are usually presented as monoliths like Set Theory or Intensional Type Theory, without any obvious way of expressive fine-tuning. The purpose of this chapter is to show how semantics has its fine-structure too, when viewed from a proper logical angle.

The framework used to demonstrate this claim is that of Categorial Grammar (see Buszkowski et al. [24] or Oehrle et al. [173] for details). In this linguistic paradigm, syntactic sensitivity resides in the landscape of logical calculi of implication, which manipulate functional types as conditional propositions. The landscape starts from a classical Ajdukiewicz system with modus ponens only and then ascends to intuitionistic conditional logic, which allows also subproofs with conditionalisation. A well-known intermediate system is the Lambek Calculus (Moortgat [156], van Benthem [232]), which takes special care in handling occurrences of propositions or types. These calculi represent various ‘engines’ for driving categorial parsing, which can be studied as to their formal properties by employing the tools of logical proof theory.

RELATING THE DISCIPLINES

My role in the original workshop which gave rise to this book was to sum up what had been said during the week, trying to relate the different contributions one to another. In preparing for what threatened to be an arduous task, it occurred to me that a fundamental assumption underlying the organization of such a workshop needed to be taken out and examined overtly: calling a workshop ‘Computational Linguistics and Formal Semantics’ assumes that these two areas of academic endeavour have something to say to each other; may, even, be inextricably related.

Superficially, of course, this seems likely to be true: an investigation of language and its use could be seen as the core interest of both disciplines. But, on looking a little more closely, the intimate connection tends to evaporate. For most computational linguists, the task is to define and implement an adequate treatment of (some subset of) some natural language within the framework of a computer application. Even when they are primarily interested in demonstrating that a particular linguistic theory or a particular paradigm of computation is superior to its rivals, the argument will frequently be couched in application-oriented terms: such and such a theory is superior because it leads to clearer linguistic descriptions which are easier to modify and to debug, for example, or such and such a computational paradigm is superior because it allows linguistic programmers to compartmentalize their knowledge of a language, describing sub-parts independently without having to worry about all the possible intricate complexities of interactions between them.

INTRODUCTION

This paper is a tutorial on property-theoretic semantics: our aim is to provide an accessible account of property theory and its application to the formal semantics of natural language. We develop a simple version of property theory and provide the semantics for a fragment of English in the theory. We shall say more about the particular form of the theory in the next section but to begin with we outline the main reasons why we believe property-theoretic semantics to be worthy of attention.

INTENSIONALITY

The main motivation for the development of a theory of propositions and properties stems from the desire to develop an adequate account of intensionality in natural language. In this section we briefly review some possible approaches to intensionality in order to motivate the approach we shall eventually adopt.

INTENSIONALITY VIA POSSIBLE WORLDS

Traditional approaches to intensionality employ the notion of possible world. Propositions are taken to be sets of possible worlds and properties to be functions from individuals to propositions. The modal and doxastic operators are then unpacked as functions from propositions to propositions. For example, the modal operator of necessity is analysed as that function which maps a proposition P to that proposition which consists of all those worlds whose accessible worlds are elements of P. Different choices of the relation of accessibility between worlds yield different modal and doxastic notions. Kripke [127] introduced the possible world analysis of necessity and possibility while Hintikka [88] extended the analysis to deal with belief and knowledge.

The workshop that inspired this book was the high point of a collaborative research project entitled ‘Incorporation of Semantics into Computational Linguistics’, supported by the EEC through the COST13 programme. The workshop was also the first official academic event to be organized by IDSIA, a newly inaugurated institute of the Dalle Molle Foundation situated in Lugano, Switzerland devoted to artificial intelligence research.

A few words about the project may help to place the contents of the book in its proper context. The underlying theme was that although human language is studied from the standpoint of many different disciplines, there is rather less constructive interaction between the disciplines than might be hoped for, and that for such interaction to take place, a common framework of some kind must be established. Computational linguistics (CL) and artificial intelligence (AI) were singled out as particular instances of such disciplines: each has focused on rather different aspects of the relationship between language and computation.

Thus, what we now call CL grew out of attempts to apply the concepts of computer science to generative linguistics (e.g. Zwicky and his colleagues [255], Petrick [181]). Given these historical origins, CL unwittingly inherited some of the goals for linguistic theory that we normally associate with Chomskyan linguistics. One such goal, for example, is to finitely describe linguistic competence, that is, the knowledge that underlies the human capacity for language by means of a set of universal principles that characterize the class of possible languages, and a set of descriptions for particular languages.

INTRODUCTION

This paper is written from the point of view of one who works in artificial intelligence (AI): the attempt to reproduce interesting and distinctive aspects of human behaviour with a computer, which, in my own case, means an interest in human language use.

There may seem little of immediate relevance to cognition or epistemology in that activity. And yet it hardly needs demonstration that AI, as an aspiration and in practice, has always been of interest to philosophers, even to those who may not accept the view that AI is, essentially, the pursuit of metaphysical goals by non-traditional means.

As to cognition in particular, it is also a commonplace nowadays, and at the basis of cognitive science, that the structures underlying AI programs are a guide to psychologists in their empirical investigations of cognition. That does not mean that AI researchers are in the business of cognition, nor that there is any direct inference from how a machine does a task, say translating a sentence from English to Chinese, to how a human does it. It is, however, suggestive, and may be the best intellectual model we currently have of how the task is done. So far, so well known and much discussed in the various literatures that make up cognitive science.

INTRODUCTION

Large practical computational-linguistic applications, such as machine translation systems, require a large number of knowledge and processing modules to be put together in a single architecture and control environment. Comprehensive practical systems must have knowledge about speech situations, goal-directed communicative actions, rules of semantic and pragmatic inference over symbolic representations of discourse meanings and knowledge of syntactic and phonological/graphological properties of particular languages. Heuristic methods, extensive descriptive work on building world models, lexicons and grammars as well as a sound computational architecture are crucial to the success of this paradigm. In this paper we discuss some paradigmatic issues in building computer programs that understand and generate natural language. We then illustrate some of the foundations of our approach to practical computational linguistics by describing a language for representing text meaning and an approach to developing an ontological model of an intelligent agent. This approach has been tested in the dionysus project at Carnegie Mellon University which involved designing and implementing a natural language understanding and generation system.

SEMANTICS AND APPLICATIONS

Natural language processing projects at the Center for Machine Translation of Carnegie Mellon University are geared toward designing and building large computational applications involving most crucial strata of language phenomena (syntactic, semantic, pragmatic, rhetorical, etc.) as well as major types of processing (morphological and syntactic parsing, semantic interpretation, text planning and generation, etc.). Our central application is machine translation which is in a sense the paradigmatic application of computational linguistics.

INTRODUCTION

This paper stands in marked contrast to many of the others in this volume in that it is intended to be entirely tutorial in nature, presupposing little on the part of the reader but a user's knowledge of English, a modicum of good will, and the desire to learn something about the notion of unification as it has come to be used in theoretical and computational linguistics. Most of the ideas I shall be talking about were first introduced to the study of ordinary language by computational linguists and their adoption by a notable, if by no means representative, subset of theoretical linguists represents an important milestone in our field, for it is the first occasion on which the computationalists have had an important impact on linguistic theory. Before going further, there may be some value in pursuing briefly why this rapprochement between these two branches has taken so long to come about. After all, the flow of information in the other direction – from theoretician to computationalist – has continued steadily from the start.

PRODUCTIVITY

The aspects of ordinary language that have proved most fascinating to its students all have something to do with its productivity, that is, with the fact that there appears to be no limit to the different utterances that can be made and understood in any of the languages of the world. Certainly, speakers can make and understand utterances that they have never made or understood before, and it is presumably largely in this fact that the immense power and flexibility of human language resides.

INTRODUCTION

The integration of syntactic and semantic processing has prompted a number of different architectures for natural language systems, such as rule-by-rule interpretation [221], semantic grammars [22] and cascaded ATNs [253]. The relationship between syntax and semantics has also been of central concern in theoretical linguistics, particularly following Richard Montague's work. Also, with the recent rapprochement between theoretical and computational linguistics, Montague's interpretation scheme has made its way into natural language systems. Variations on Montague's interpretation scheme have been adopted and implemented in several syntactic theories with a significant following in computational linguistic circles. The first steps in this direction were taken by Hobbs and Rosenschein [89]. A parser for LFG was augmented with a Montagovian semantics by Halvorsen [76]. GPSG has been similarly augmented by Gawron et al. [65], and Schubert and Pelletier [206] followed with a compositional interpretation scheme using a first order logic rather than Montague's computationally intractable higher-order intensional logic.

In parallel with these developments in the syntax/semantics interface, unification-based mechanisms for linguistic description had significant impact both on syntactic theory and syntactic description. But the Montagovian view of compositionality in semantics and the architectures for configuring the syntax/semantics interface were slower to achieve a similar revaluation in view of unification and the new possibilities for composition which it brought.