This chapter covers digital information sources in some depth. It provides intuition on the information content of a digital source and introduces the notion of redundancy. As a simple but important example, discrete memoryless sources are described. The concept of entropy is defined as a measure of the information content of a digital information source. The properties of entropy are studied, and the source-coding theorem for a discrete memoryless source is given. In the second part of the chapter, practical data compression algorithms are studied. Specifically, Huffman coding, which is an optimal data-compression algorithm when the source statistics are known, and Lempel–Ziv (LZ) and Lempel–Ziv–Welch (LZW) coding schemes, which are universal compression algorithms (not requiring the source statistics), are detailed.

The basics of digital modulation over additive white Gaussian noise (AWGN) channels are studied. To facilitate a formal study, the concepts of signal space and signal constellations are introduced. The Gram–Schmidt orthonormalization procedure, a systematic method to obtain an orthogonal and normalized basis for a given set of signals, is described. Binary antipodal signaling is studied in detail; the MAP and ML receivers are derived, and the average probability of error is computed. The concepts are then generalized to the case of M-ary signaling, and the union bound is introduced as a performance analysis tool. Correlation-type and matched filter-type receivers are described. The properties of the matched filter are summarized. Different signal constellations are compared in terms of their error rate performance through a simplified (asymptotic) analysis. As specific examples, the details of two important digital modulation schemes, pulse amplitude modulation and orthogonal signaling, are given. Finally, timing recovery techniques are briefly studied.

Frequency-shift keying (FSK) is described as an alternative way of transmitting digital information. Specifically, orthogonal FSK with both coherent and non-coherent detection is studied. Minimum-shift keying is introduced as a special case of FSK, preserving phase continuity at the symbol boundaries. In addition, orthogonal frequency-division multiplexing (OFDM) is covered in some depth. It is shown that OFDM can be efficiently implemented using fast Fourier transform (FFT) and its inverse. The use of a cyclic prefix to avoid intersymbol interference over dispersive channels is also shown.

The fundamental limits of communication over a noisy channel, in particular, over an AWGN channel, are described, and channel coding is introduced as a way of approaching the ultimate information-theoretic limits of reliable communication. Linear block codes and convolutional codes are studied in some depth. Encoding and decoding algorithms, as well as basic performance analysis results, are developed. The Viterbi algorithm is introduced for both hard-decision decoding and soft-decision decoding of convolutional codes.

This chapter first provides an overview of a general communication system and then shifts the focus to a digital communication system. It describes elements of a digital communication system and explains the functionalities of source coding, channel coding, and digital modulation blocks for communicating over a noisy channel. It also highlights the differences between analog and digital communication systems.