Introduction

Here we introduce the discrete wavelet transform (DWT), which is the basic tool needed forstudying time series via wavelets and plays a role analogous to that of the discrete Fouriertransform in spectral analysis. We assume only that the reader is familiar with the basic ideas fromlinear filtering theory and linear algebra presented in Chapters 2 and 3. Our exposition buildsslowly upon these ideas and hence is more detailed than necessary for readers with strongbackgrounds in these areas. We encourage such readers just to use the Key Facts and Definitions ineach section or to skip directly to Section 4.12 – this has a concise self-containeddevelopment of the DWT. For complementary introductions to the DWT, see Strang (1989, 1993), Riouland Vetterli (1991), Press et al. (1992) and Mulcahy (1996).

The remainder of this chapter is organized as follows. Section 4.1 gives a qualitativedescription of the DWT using primarily the Haar and D(4) wavelets as examples. The formalmathematical development of the DWT begins in Section 4.2, which defines the wavelet filter anddiscusses some basic conditions that a filter must satisfy to qualify as a wavelet filter. Section4.3 presents the scaling filter, which is constructed in a simple manner from the wavelet filter.The wavelet and scaling filters are used in parallel to define the pyramid algorithm for computing(and precisely defining) the DWT – various aspects of this algorithm are presented inSections 4.4, 4.5 and 4.6.

The last decade has seen an explosion of interest in wavelets, a subject area that has coalescedfrom roots in mathematics, physics, electrical engineering and other disciplines. As a result,wavelet methodology has had a significant impact in areas as diverse as differential equations,image processing and statistics. This book is an introduction to wavelets and their application inthe analysis of discrete time series typical of those acquired in the physical sciences. While wepresent a thorough introduction to the basic theory behind the discrete wavelet transform (DWT), ourgoal is to bridge the gap between theory and practice by

1. • emphasizing what the DWT actually means in practical terms;

2. • showing how the DWT can be used to create informative descriptive statistics fortime series analysts;

3. • discussing how stochastic models can be used to assess the statisticalproperties of quantities computed from the DWT; and

4. • presenting substantive examples of wavelet analysis of time seriesrepresentative of those encountered in the physical sciences.

To date, most books on wavelets describe them in terms of continuous functions and oftenintroduce the reader to a plethora of different types of wavelets. We concentrate on developingwavelet methods in discrete time via standard filtering and matrix transformation ideas.

Introduction

The continuous time wavelet transform is becoming a well-established tool for multiple scale representation of a continuous time ‘signal,’ which by definition is a finite energy function denned over the entire real axis. This transform essentially correlates a signal with ‘stretched’ versions of a wavelet function (in essence a continuous time band-pass filter) and yields a multiresolution representation of the signal. In this chapter we summarize the important ideas and results for the multiresolution view of the continuous time wavelet transform. Our primary intent is to demonstrate the close relationship between continuous time wavelet analysis and the discrete time wavelet analysis presented in Chapter 4. To make this connection, we adopt a formalism that allows us to bridge the gap between the inner product convention used in mathematical discussions on wavelets and the filtering convention favored by engineers. For simplicity we deal only with signals, scaling functions and wavelet functions that are all taken to be real-valued. Only the case of dyadic wavelet analysis (where the scaling factor in the dilation of the basis function takes the value of two) is considered here.

Introduction

As we saw in Chapters 4 and 5, one important use for the discrete wavelet transform (DWT) and its variant, the maximal overlap DWT (MODWT), is to decompose the sample variance of a time series on a scale-by-scale basis. In this chapter we explore wavelet-based analysis of variance (ANOVA) in more depth by defining a theoretical quantity known as the wavelet variance (sometimes called the wavelet spectrum). This theoretical variance can be readily estimated based upon the DWT or MODWT and has been successfully used in a number of applications; see, for example, Gamage (1990), Bradshaw and Spies (1992), Flandrin (1992), Gao and Li (1993), Hudgins et al. (1993), Kumar and Foufoula-Georgiou (1993, 1997), Tewfik et al. (1993), Wornell (1993), Scargle (1997), Torrence and Compo (1998) and Carmona et al. (1998). The definition for the wavelet variance and rationales for considering it are given in Section 8.1, after which we discuss a few of its basic properties in Section 8.2. We consider in Section 8.3 how to estimate the wavelet variance given a time series that can be regarded as a realization of a portion of length N of a stochastic process with stationary backward differences. We investigate the large sample statistical properties of wavelet variance estimators and discuss methods for determining an approximate confidence interval for the true wavelet variance based upon the estimated wavelet variance (Section 8.4).

Introduction

In Chapter 4 we discussed the discrete wavelet transform (DWT), which essentially decomposes atime series X into coefficients that can be associated with different scales and times. We can thusregard the DWT of X as a ‘time/scale’ decomposition. The wavelet coefficients for agiven scale Tj ≡ 2J−1 tell ushow localized weighted averages of X vary from one averaging period to the next. The scaleTj gives us the effective width in time (i.e., degree of localization)of the weighted averages. Because the DWT can be formulated in terms of filters, we can relate thenotion of scale to certain bands of frequencies. The equivalent filter that yields the waveletcoefficients for scale Tj is approximately a band-pass filter with apass-band given by [l/2j+1, 1/2j].For a sample size N = 2J, the N - 1wavelet coefficients constitute - when taken together - an octave band decomposition of thefrequency interval [1/2J+1, 1/2], while the single scalingcoefficient is associated with the interval [0, 1/2J+1]. Taken asa whole, the DWT coefficients thus decompose the frequency interval [0, 1/2] into adjacentindividual intervals.

In this chapter we consider the discrete wavelet packet transform (DWPT), whichcan be regarded as any one of a collection of orthonormal transforms, each of which can be readilycomputed using a very simple modification of the pyramid algorithm for the DWT.

Unpredictability and non-determinism are all around us. The future behaviour of any system—from an elementary particle to a complex organism—may follow a number of possible paths. Some of these paths may be more likely than others, but none is absolutely certain. Such unpredictable behaviour, and the phenomena that cause it, are usually described as ‘random’. Whether randomness is in the nature of reality, or is the result of imperfect knowledge, is a philosophical question which need not concern us here. More important is to learn how to deal with randomness, how to quantify it and take it into account, so as to be able to plan and make rational choices in the face of uncertainty.

The theory of probabilities was developed with this object in view. Its domain of applications, which was originally confined mainly to various games of chance, now extends over most scientific and engineering disciplines.

This chapter is intended as a self-contained introduction; it describes all the concepts and results of probability theory that will be used in the rest of the book. Examples and exercises are included. However, it is impossible to provide a thorough coverage of a major branch of mathematics in one chapter. The reader is therefore assumed to have encountered at least some of this material already.

The designers and users of complex systems have an interest in knowing how those systems behave under different conditions. This is true in all engineering domains, from transport and manufacturing to computing and communications. It is necessary to have a clear understanding, both qualitative and quantitative, of the factors that influence the performance and reliability of a system. Such understanding may be obtained by constructing and analysing mathematical models. The purpose of this book is to provide the necessary background, methods and techniques.

A model is inevitably an approximation of reality: a number of simplifying assumptions are usually made. However, that need not diminish the value of the insights that can be gained. A mathematical model can capture all the essential features of a system, display underlying trends and provide quantitative relations between input parameters and performance characteristics. Moreover, analysis is cheap, whereas experimentation is expensive. A few simple calculations carried out on the back of an envelope can often yield as much information as hours of observations or simulations.

The systems in which we are interested are subjected to demands of random character. The processes that take place in response to those demands are also random. Accordingly, the modelling tools that are needed to study such systems come from the domains of probability theory, stochastic processes and queueing theory.

Some of the most important applications of probabilistic modelling techniques are in the area of distributed systems. The term ‘distributed’ means, in this context, that various tasks that are somehow related can be carried out by different servers which may or may not be in different geographical locations. Such a broad definition covers a great variety of applications, in the areas of manufacturing, transport, computing and communications. To study the behaviour of a distributed system, one normally needs a model involving a number of service centres, with jobs arriving and circulating among them according to some random or deterministic routeing pattern. This leads in a natural way to the concept of a network of queues.

A queueing network can be thought of as a connected directed graph whose nodes represent service centres. The arcs between those nodes indicate one-step moves that jobs may make from service centre to service centre (the existence of an arc from nodei to nodej does not necessarily imply one from j to i). Each node has its own queue, served according to some scheduling strategy. Jobs may be of different types and may follow different routes through the network. An arc without origin leading into a node (or one without destination leading out of a node) indicates that jobs arrive into that node from outside (or depart from it and leave the network). Figure 4.1 shows a five-node network, with external arrivals into nodes 1 and 2, and external departures from nodes 1 and 5. At this level of abstraction, only the connectivity of the nodes is specified; nothing is said about their internal structure, nor about the demands that jobs place on them.

At a certain level of abstraction, computing and communication systems as well as banking, manufacturing and transport systems, can be described in terms of ‘jobs’ and ‘servers’, i.e. requests for service and devices that provide service. The jobs may be computing tasks, input/output commands, telephone calls, data packets. The servers may be processors, storage devices, communication channels, software modules. A model aimed at evaluating and predicting the performance of such a system has to capture the following essential aspects of its behaviour:

1. (a) The pattern of demand, i.e. the the manner in which jobs arrive into the system and the nature of services that they require.

2. (b) The competition for service, i.e. the effect of admission, queueing and routing policies on performance.

3. This chapter is devoted to (a). It introduces tools and results that are used when modelling the arrivals and services of jobs.

Renewal processes

Consider a phenomenon which takes place first at time 0 and thereafter keeps occurring, at random intervals, ad infinitum. Denote the consecutive instants of occurrence by Tn (n = 0,1,…; T0 0), and let Sn Tn — Tn-1 (n = 1,2,…) be the intervals between them. Assume that the random variables Sn are independent and identically distributed.