The asymptotic theory of nonlinear regression models, in particular consistency results, heavily depends on uniform laws of large numbers. Understanding these laws requires knowledge of abstract probability theory. In this chapter we shall review the basic elements of this theory as needed in what follows, to make this book almost self-contained. For a more detailed treatment, see for example Billingsley (1979) and Parthasarathy (1977). However, we do assume the reader has a good knowledge of probability and statistics at an intermediate level, for example on the level of Hogg and Craig (1978). The material in this chapter is a revision and extension of section 2.1 in Bierens (1981).

Measure-theoretical foundation of probability theory

The basic concept of probability theory is the probability space. This is a triple {Ω,ℑ,P} consisting of:

— An abstract non-empty set Ω, called the sample space. We do not impose any conditions on this set.

— A non-empty collection ℑ of subsets of Ω, having the following two properties:

where Ec denotes the complement of the subset E with respect to Ω: Ec = Ω\E.

These two properties make ℑ, by definition, a Borel field of subsets of Ω. (Following Chung [1974], the term “Borel field” has the same meaning as the term “σ-Algebra” used by other authors.)

— A probability measure P on {Ω,ℑ}. This is a real-valued set function on ℑ such that: Example: Toss a fair coin.