In this chapter we describe two ensemble learning techniques, known as bagging and boosting, which aggregate the decisions of a mixture of learners to enable enhanced classification performance. In particular, they help transform a collection of “weak” learners into a more robust learning machine.

The expectation-maximization (EM) and Baum–Welch algorithms are particularly useful for the processing of data arising from mixture models. Both techniques enable us to identify the parameters of the underlying components, for both cases when the observations are independent of each other or follow a first-order Markovian process. In this chapter, we consider another important example of a mixture model consisting of a collection of independent sources, a mixture matrix, and the observations. The objective is to undo the mixing and recover the original sources. The resulting technique is known as independent component analysis (ICA).

The Laplace method approximates the posterior distribution $f_{\mathit{z}|\mathit{y}}\left(z|y\right)$ through a Gaussian probability density function (pdf) that is not always accurate. The Markov chain Monte Carlo (MCMC) method, on the other hand, relies on sampling from auxiliary (proposal) distributions and provides a powerful way to approximate posterior distributions albeit through repeated simulations. In this chapter, we describe a third approach for approximating the posterior distribution, known as expectation propagation (EP). This method restricts the class of distributions from which the posterior is approximated to the Gaussian or exponential family and assumes a factored form for the posterior. The method can become analytically demanding, depending on the nature of the factors used for the posterior, because these factors can make the computation of certain moments unavailable in closed form. The EP method has been observed to lead to good performance in some applications such as the Bayesian logit classification problem, but this behavior is not universal and performance can degrade for other problems, especially when the posterior distribution admits a mixture model.

The $k$-nearest neighbor ($k$-NN) rule is appealing. However, each new feature $h\in {\mathbb{\text{R}}}^M$ requires searching over the entire training set of size $N$ to determine the neighborhood around $h$.

In this chapter, we describe three other data-based generative methods that approximate the solution to the optimal Bayes classifier (52.8) in the absence of knowledge of the conditional probabilities $\mathbb{P}\left(r=r|h=h\right)$. The methods estimate the prior probabilities $\mathbb{P}\left(r=r\right)$ for the classes and, in some cases, assume a Gaussian form for the reverse conditional distribution, $f_{h|r}\left(h|r\right)$. The training data is used to estimate the priors and the first-and second-order moments of $f_{h|r}\left(h|r\right)$.

The discussion in the last two chapters focused on directed graphical models or Bayesian networks, where a directed link from a variable ${\mathit{x}}_1$ toward another variable ${\mathit{x}}_2$ carries with it an implicit connotation of “causal effect” by ${\mathit{x}}_1$ on ${\mathit{x}}_2$. In many instances, this implication need not be appropriate or can even be limiting. For example, there are cases where conditional independence relations cannot be represented by a directed graph. One such example is provided in Prob. 43.1. In this chapter, we examine another form of graphical representations where the links are not required to be directed anymore, and the probability distributions are replaced by potential functions. These are strictly positive functions defined over sets of connected nodes; they broaden the level of representation by graphical models. The potential functions carry with them a connotation of “similarity” or “affinity” among the variables, but can also be rolled back to represent probability distributions. Over undirected graphs, edges linking nodes will continue to reflect pairwise relationship between the variables but will lead to a fundamental factorization result in terms of the product of clique potential functions. We will show that these functions play a prominent role in the development of message-passing algorithms for the solution of inference problems.

The inference of a random variable $\mathit{x}$ from observations $\{{\mathit{y}}_1,{\mathit{y}}_2,\dots ,{\mathit{y}}_N\}$ requires that we evaluate the posterior distribution $f_{\mathit{x}\mid {\mathit{y}}_{1:N}}(x\mid y_1,\dots ,y_N)$ as happens, for example, in inference formulations based on mean-square-error (MSE), maximum a-posteriori (MAP), or probability of error metrics. In previous chapters, we described several techniques to facilitate the computation or approximation of such posterior distributions using Monte Carlo or variational inference methods. We will encounter other types of approximations in later chapters. For example, in the context of naïve Bayes classifiers in Chapter 55, we will assume that, conditioned on the latent variable $\mathit{x}$, the observations are independent of each other in order to write

We continue our treatment of Markov decision processes (MDPs) and focus in this chapter on methods for determining optimal actions or policies. We derive two popular methods known as value iteration and policy iteration, and establish their convergence properties. We also examine the Bellman optimality principle in the context of value and policy learning. In a later section, we extend the discussion to the more challenging case of partially observable MDPs (POMDPs), where the successive states of the MDP are unobservable to the agent, and the agent is only able to sense measurements emitted randomly by the MDP from the various states. We will define POMDPs and explain that they can be reformulated as belief‐MDPs with continuous (rather than discrete) states. This fact complicates the solution of the value iteration. Nevertheless, we will show that the successive value iterates share a useful property, namely, that they are piecewise linear and convex. This property can be exploited by computational methods to reduce the complexity of solving the value iteration for POMDPs.

In this chapter, we describe a popular discriminative approach for classification problems, known as logistic regression. Assuming binary classification with labels $\text{γ}\in \left\{\pm 1\right\}$ and features $h\in {\mathbb{\text{R}}}^M$, we explained earlier in expression (28.85) that the optimal Bayes classifier for predicting $\text{γ}$ is given by

We mentioned earlier in Section 52.3 that the nearest-neighbor (NN) rule for classification and clustering treats equally all attributes within each feature vector, $h_n\in {\mathbb{\text{R}}}^M$. If, for example, some attributes are more relevant to the classification task than other attributes, then this aspect is ignored by the NN classifier because all entries of the feature vector will contribute similarly to the calculation of Euclidean distances and the determination of neighborhoods.

As readers of Michael Silverstein’s works know, his published papers are always carefully annotated with the places they have been delivered as “talks,” the persons who invited him to speak or write, and the thanks to those who commented and contributed in different ways to the finished product. Tragically, he cannot append that form of acknowledgment to this book. Michael Silverstein died in July 2020, after a year of illness and in the midst of editing and polishing the manuscript of this book. To follow his usual practice of providing a natural history of this text seems a fitting form of acknowledgment to those devoted colleagues who participated in bringing this book to publication.