This book developed out of some topics courses given at M.I.T. and my lectures at the St.-Flour probability summer school in 1982. The material of the book has been expanded and extended considerably since then. At the end of each chapter are some problems and notes on that chapter.

Starred sections are not cited later in the book except perhaps in other starred sections. The first edition had several double-starred sections in which facts were stated without proofs. This edition has no such sections.

The following, not proved in the first edition, now are: (i) for Donsker's theorem on the classical empirical process αn := √n(Fn − F), and the Komlós–Major–Tusnády strengthening to give a rate of convergence, the Bretagnolle–Massart proof with specified constants; (ii) Massart's form of the Dvoretzky–Kiefer–Wolfowitz inequality for αn with optimal constant; (iii) Talagrand's generic chaining approach to boundedness of Gaussian processes, which replaces the previous treatment of majorizing measures; (iv) characterization of uniform Glivenko–Cantelli classes of functions (from a paper by Dudley, Giné, and Zinn, but here with a self-contained proof); (v) Giné and Zinn's characterization of uniform Donsker classes of functions; (vi) its consequence that uniformly bounded, suitably measurable classes of functions satisfying Pollard's entropy condition are uniformly Donsker; and (vii) Bousquet, Koltchinskii, and Panchenko's theorem that a convex hull preserves the uniform Donsker property.

This chapter will treat some classes of sets satisfying a combinatorial condition. In Chapter 6 it will be shown that under a mild measurability condition to be treated in Chapter 5, these classes have the Donsker property, for all probability measures P on the sample space, and satisfy a law of large numbers (Glivenko–Cantelli property) uniformly in P. Moreover, for either of these limit-theorem properties of a class of sets (without assuming any measurability), the Vapnik–Červonenkis property is necessary (Section 6.4).

The name Červonenkis is sometimes transliterated into English as Chervonenkis. The present chapter will be self-contained, not depending on anything earlier in this book, except in some examples.

Vapnik–Červonenkis Classes of Sets

Let X be any set and C a collection of subsets of X. For A ⊂ X let CA:= C ⊓ A:= A ⊓ C:= {C ⋂ A: C ∈ C}. Let card(A):= |A| denote the cardinality (number of elements) of A and 2A:={B: B ⊂ A}. Let ΔC(A):=|CA|. If A ⊓ C = 2A, then C is said to shatter A. If A is finite, then C shatters A if and only if ΔC(A) = 2|A|.

Universal Lower Bounds

This chapter is primarily about asymptotic lower bounds for ∥Pn − P∥F on certain classes F of functions, as treated in Chapter 8, mainly classes of indicators of sets. Section 11.2 will give some upper bounds which indicate the sharpness of some of the lower bounds. Section 11.4 gives some relatively difficult lower bounds on classes such as the convex sets in ℝ3 and lower layers in ℝ2. In preparation for this, Section 11.3 treats Poissonization and random “stopping sets” analogous to stopping times. The present section gives lower bounds in some cases which hold not only with probability converging to 1, but for all possible Pn. Definitions are as in Sections 3.1 and 8.2, with P:= U(Id) = λd = Lebesgue measure on Id. Specifically, recall the classes G(α, K, d):= Gα,K, d of functions on the unit cube Id ⊂ ℝd with derivatives through αth order bounded by K, and the related families C(α, K, d) of sets (subgraphs of functions in G(α, K, d − 1)), both defined early in Section 8.2.

Theorem 11.1 (Bakhvalov) For P = U(Id), any d = 1, 2, … and α > 0, there is a γ = γ (d, α) > 0 such that for all n = 1, 2,…, and all possible values of Pn, we have ∥Pn − P ∥G(α1,d) ≥ γn−α/d.