The operation of mapping (naturally occurring) continuous time/analog signals into (electronics-friendly) discrete/digital signals is known as quantization, which is an important subject in signal processing in its own right. In information theory, the study of optimal quantization is called rate-distortion theory, introduced by Shannon in 1959. To start, in Chapter 24 we will take a closer look at quantization, followed by the information-theoretic formulation. A simple (and tight) converse bound is then given, with the matching achievability bound deferred to Chapter 25.

In Chapter 4 we collect some results on variational characterizations of information measures. It is a well-known method in analysis to study a functional by proving variational characterizations representing it as a supremum or infimum of some other, simpler (often linear) functionals. Such representations can be useful for multiple purposes:

• Convexity: the pointwise supremum of convex functions is convex.

• Regularity: the pointwise supremum of lower semicontinuous (lsc) functions is lsc.

• Bounds: the upper and lower bounds on the functional follow by choosing good solutions in the optimization problem.

We summarize the meaning of the Standard Model and present a list of profound openquestions within its framework and beyond.

Teleological conceptions of morality originated in ancient Greek philosophy. The major systems of ethics among the ancient Greeks, those of Plato and Aristotle, in particular, were teleological. So too were those of Epicurus and other thinkers who founded important schools of philosophy in the period that came after Plato and Aristotle. Deontological conceptions, by contrast, have a different origin. They derive from an ideal of universal divine law that Christianity drew from the Judaic materials from which it sprang. Christianity, to be sure, drew from the ancient Greeks as well. Its identification of universal divine laws with the laws of nature, for instance, comes from the Stoics, chiefly through Cicero (106–43 BCE). But the ideas in Christianity that yielded deontological conceptions are found in its understanding of divine laws as the laws of a supreme ruler that bind his subjects to obey him in the way that a covenant with him would bind them. These juristic ideas, which originated in Mosaic law, are the original frame for deontological conceptions. The principal text that inspired them is Paul’s statement in Romans: “When Gentiles who have not the law do by nature what the law requires, they are a law to themselves even though they do not have the law. They show that what the law requires is written on their hearts, while their conscience also bears witness and their conflicting thoughts accuse and perhaps excuse them.”

So far we have been focusing on the paradigm for one-way communication: data are mapped to codewords and transmitted, and later decoded based on the received noisy observations. Chapter 23 looks at the more practical setting (except for storage), where the communication frequently goes in both ways so that the receiver can provide certain feedback to the transmitter. As a motivating example, consider the communication channel of the downlink transmission from a satellite to earth. Downlink transmission is very expensive (power constraint at the satellite), but the uplink from earth to the satellite is cheap which makes virtually noiseless feedback readily available at the transmitter (satellite). In general, channel with noiseless feedback is interesting when such asymmetry exists between uplink and downlink. Even in less ideal settings, noisy or partial feedbacks are commonly available that can potentially improve the reliability or complexity of communication. In the first half of our discussion, we shall follow Shannon to show that even with noiseless feedback “nothing” can be gained in the conventional setup. In the process, we will also introduce the concept of Massey’s directed information. In the second half of the Chapter we examine situations where feedback is extremely helpful: low probability of error, variable transmission length and variable transmission power.

Scalar quantum electrodynamics is constructed by promoting a global U(1) symmetry to alocal one. We address electrically charged infraparticles, and the correspondingsuperselection sectors, in infinite volume and in finite volume with two kinds of boundaryConditions.

In Chapter 21 we will consider an interesting variation of the channel coding problem. Instead of constraining the blocklength (i.e., the number of channel uses), we will constrain the total cost incurred by the codewords. The motivation is the following. Consider a deep-space probe that has a k-bit message that needs to be delivered to Earth (or a satellite orbiting it). The duration of transmission is of little worry for the probe, but what is really limited is the amount of energy it has stored in its battery. In this chapter we will learn how to study this question abstractly and how this fundamental limit is related to communication over continuous-time channels.

Modeling relies on the presence of patterns in reference datasets, and some of those patterns might occur in text. For example, customer service operations at an application service provider might need to know if customer comments on social media indicate satisfaction or dissatisfaction with the company’s products. There could be many thousands of comments across many channels, so getting people to read all of them would be slow and costly. Operations would be glad of a predictive model that automatically discerns sentiment expressed in the comments.

By means of an ion crystal model, we illustrate the concepts of a particle in the senseof quantum mechanics and of quantum field theory. The latter describes reality in particlephysics, but in order to avoid confusion, we temporarily denote it as a “wavicle”.

In Chapter 25 we present the hard direction of the rate-distortion theorem: the random coding construction of a quantizer. This method is extended to the development of a covering lemma and soft-covering lemma, which lead to the sharp result of Cuff showing that the fundamental limit of channel simulation is given by Wyner’s common information. We also derive (a strengthened form of) Han and Verdú’s results on approximating output distributions in Kullback–Leibler.

This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. The book introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite blocklength approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning, and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC-Bayes and variational principle, Kolmogorov’s metric entropy, strong data-processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by additional stand-alone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.

Chiral symmetry of free fermions is studied in the continuum and on the lattice. In thelatter case, we review the fermion doubling problem and the Nielsen–Ninomiya theorem, thenwe construct Wilson fermions and finally several types of Ginsparg–Wilson fermions, whichare endowed with an exact, lattice modified chiral symmetry.