This chapter covers the basics from real analysis to linear algebra and the theory of computation that is foundational for the rest of the book. A careful discussion of different models of computation is taken up, which discusses several issues that are often ignored in other presentations of optimization theory and algorithms.

A careful exposition of the conceptual underpinnings of algorithmic or computational optimization is presented. Computation in continuous optimization has its origins in the traditions of scientific computing and numerical analysis, whereas discrete optimization broadly views computation via the Turing machine model. The different views lead to some friction. In the continuous world, one often designs algorithms assuming one can perform exact operations with real numbers (consider, for example, Newton’s method), which is impossible in the Turing machine model. In the discrete world, the “input" to a Turing machine becomes a tricky question when dealing with general nonlinear functions and sets. The question of “complexity" of an optimization algorithm is also treated in somewhat different ways in the two communities. This chapter, combined with the careful discussion of computation models in Chapter 1, shows how all these issues can be handled in a unified, coherent way making no distinction whatsoever between "continuous" and "discrete" optimization.

This chapter deals with the important question of certifying optimality of a solution to a mixed-integer convex optimization problem. The classical duality theory for continuous optimization, including Lagrangian relaxations, KKT and general optimality conditions, and Slater type conditions for strong duality, is rigorously covered in complete detail. Recent work on duality for mixed-integer convex optimization is succinctly summarized.

In the early part of the 20th century, Hermann Minkowski developed a novel geometric approach to several questions in number theory. This approach developed into a field called the geometry of numbers and it had an influence on fields outside number theory as well, particularly functional analysis and the study of Banach spaces, and more recently on cryptography and discrete optimization. This chapter covers those aspects of the geometry of numbers that are most relevant for the second part of the book on optimization. Topics include the basic theory of lattices (including Minkowski’s convex body theorem), packing and covering radii, shortest and closest lattice vector problems (SVPs and CVPs), Dirichlet-Voronoi cells, Khinchine’s flatness theorem, and maximal lattice-free convex sets. Several topics like lattice basis reduction and SVP/CVP algorithms are presented without making a rationality assumption as is common in other expositions. This presents a slightly more general perspective on these topics that contains the rational setting as a special case.

This chapter introduces the concept of a convex function and develops the basic theory of convex functions. Standard continuity and differentiability properties are established. Fundamental notions like subgradients and subdifferentials are introduced and their properties are investigated in detail. Sublinear functions get particular focus, given their recent importance in optimization theory and practice. Some new results on sublinear functions that have never before appeared outside specialized research articles are presented with clean, textbook-style proofs. Elementary Brunn-Minkowski theory is covered, including important consequences like the concavity principle and the Rogers-Shepard inequality.

This chapter introduces the fundamental notion of a convex set. It establishes basic structural properties of convex sets, illustrated via examples throughout. The chapter gives equal emphasis on the analytic as well as discrete or combinatorial aspects of convexity. Topics include foundational results like the Separating and Supporting Hyperplane theorems, polarity, the combinatorial theorems of Caratheodory, Radon and Helly, and the basic theory of polyhedra and ellipsoids.

This chapter presents the theory of mixed-integer convex optimization, i.e., minimizing a convex function subject to convex constraints where some of the decision variables have to take integer values. State-of-the-art results on information and algorithmic complexity of mixed-integer convex optimization are established. The basics of continuous convex optimization are presented as the special case where no variable is integer constrained.

Information complexity of classical continuous optimization has been well understood since the 1970s. The information complexity in the presence of integer variables was not well developed until research work done in the past decade and is covered in complete detail here. On the algorithmic side, the best known upper bound of $2^{n\log(n)}$ on the complexity of deterministic algorithms for convex integer optimization is presented, which does not appear outside specialized, technical research articles. Moreover, a general mixed-integer complexity bound allowing for both integer and continuous variables is presented that does not explicitly appear anywhere in the literature. A complete theory of branch-and-cut methods is also covered.