Introductory comments

Entropy is a concept that is basic to the second law of thermodynamics. It has been used extensively in deriving and studying probability distributions corresponding to maximum entropy. We know that it represents, in some way, disorder, and that the entropy of the universe increases. Eddington called it ‘time's arrow’. Its roots are in the erratic development of the laws of thermodynamics over the last century and a half. Bridgman (1941) says of these laws that ‘they smell more of their human origin’ (as compared to the other laws of physics). This origin encompasses such activities as the observation by Mayer of the colour of blood of his patients in the tropics. During bleeding of these patients in Java, he noticed that the red colour of their venous blood was much brighter than that of his patients in Germany. This brightness indicated that less oxygen had been used in the processing of body fuel than in Germany. The human body, treated as a heat engine, needed less energy in the tropics since less energy was taken out by the surroundings. Mayer concluded that work and heat are equivalent. The second law is illuminated by the irreversible generation of heat during the boring of cannons using teams of horses by Count Rumford.

Introduction

Uncertainty is an essential and inescapable part of life. During the course of our lives, we inevitably make a long series of decisions under uncertainty – and suffer the consequences. Whether it be a question of deciding to wear a coat in the morning, or of deciding which soil strength to use in the design of a foundation, a common factor in these decisions is uncertainty. One may view humankind as being perched rather precariously on the surface of our planet, at the mercy of an uncertain environment, storms, movements of the earth's crust, the action of the oceans and seas. Risks arise from our activities. In loss of life, the use of tobacco and automobiles pose serious problems of choice and regulation. Engineers and economists must deal with uncertainties: the future wind loading on a structure; the proportion of commuters who will use a future transportation system; noise in a transmission line; the rate of inflation next year; the number of passengers in the future on an airline; the elastic modulus of a concrete member; the fatigue life of a piece of aluminium; the cost of materials in ten years; and so on.

Probability distributions

We are now in the following situation. We have a probability mass of unity, and we wish to distribute this over the possible outcomes of events and quantities of interest. We also wish to manipulate the results to assist in decision-making. We have so far considered mainly random events, which may take the value 0 or 1. A common idealized example that we have given is that of red and pink balls in an urn, which we have labelled 1 and 0, respectively. Thus, the statement E = ‘the ball drawn from the urn is red’ becomes E = 0 or E = 1, depending on whether we draw a pink or red ball. The event E either happens or it does not, and there are only two possible outcomes. Random quantities can have more than two outcomes. For the case of drawings from an urn, the random quantity could be the number of red balls in ten (or any other number) of drawings, for example from Raiffa's urns of Figure 2.10. A further example is the throwing of a die which can result in six possible outcomes; the drawing of a card from a deck can result in 52 possible outcomes, and so on.

Probabilistic reasoning is a vital part of engineering design and analysis. Inevitably it is related to decision-making – that important task of the engineer. There is a body of knowledge profound and beautiful in structure that relates probability to decision-making. This connection is emphasized throughout the book as it is the main reason for engineers to study probability. The decisions to be considered are varied in nature and are not amenable to standard formulae and recipes. We must take responsibility for our decisions and not take refuge in formulae. Engineers should eschew standard methods such as hypothesis testing and think more deeply on the nature of the problem at hand. The book is aimed at conveying this line of thinking. The search for a probabilistic ‘security blanket’ appears as futile. The only real standard is the subjective definition of probability as a ‘fair bet’ tied to the person doing the analysis and to the woman or man in the street. This is our ‘rule for life’, our beacon. The relative weights in the fair bet are our odds on and against the event under consideration.

It is natural to change one's mind in the face of new information. In probabilistic inference this is done using Bayes' theorem. The use of Bayesian methods is presented in a rigorous manner. There are approximations to this line of thinking including the ‘classical’ methods of inference. It has been considered important to view these and others through a Bayesian lens.

Introduction to probability

Probability and utility are the primary and central concepts in decision-making. We now discuss the basis for assigning probabilities. The elicitation of a probability aims at a quantitative statement of opinion about the uncertainty associated with the event under consideration. The theory is mathematical, so that ground rules have to be developed. Rather than stating a set of axioms, which are dull and condescend, we approach the subject from the standpoint of the potential behaviour of a reasonable person. Therefore, we need a person, you perhaps, who wishes to express their probabilistic opinion. We need also to consider an event about which we are uncertain. The definition of possible events must be clear and unambiguous.

It is a general rule that the more we study the circumstances surrounding the event or quantity, the better will be our probabilistic reasoning. The circumstances might be physical, as in the case where we are considering a problem involving a physical process. For example, we might be considering the extreme wave height during a year at an offshore location. Or the circumstances might be psychological, for example, when we are considering the probability of acceptance of a proposal to carry out a study in a competitive bid, or the probability of a person choosing to buy a particular product. The circumstances might contain several human elements in the case of the extreme incoming traffic in a data network.

Uncertainty accompanies our lives. Coherent modelling of uncertainties for decision-making is essential in engineering and related disciplines. The important tools have been outlined. It is neither possible nor desirable to attempt to give a recipe that can be used for specific problems. The fun of engineering is to use the tools so as to create a methodology that can be used for a particular problem. Probability is seen as the measure of uncertainty. Decisions are based on an analysis of uncertainties using probability and of desires and aversion using utility. These factors guide our thinking in approaching problems.

In estimating probabilities, the beacon that guides us is the definition of probability as a fair bet. Frequencies can be used only to assist in evaluating probabilities but do not constitute a definition of probability. It is important to address all uncertainties without taking a problem out of reasonable practical proportion. If done in a blind manner without judgement, this can lead to gross overestimation of the uncertainty regarding our quantity of interest. Our estimates of mean values, including variances and other moments, should accord with our judgement, and we should beware of compounding uncertainties that might result in unrealistic engineering judgements.

Flowing from the definition of probability as a fair bet, Bayesian thought should guide our modelling.

Conflict

Up to now, we have considered decision-making in cases where we are the ‘protagonist’, and where the ‘antagonist’ is relatively benign, or more precisely, indifferent to our desires. In this case, we characterize the antagonist as ‘nature’ in which we have we considered nature to consist of those aspects such as weather that do not react to our everyday behaviour. We can then regard the activity of engineering design as a ‘game against nature’. In the example of design of an offshore structure to resist wave loading, we might choose the design wave height and ‘nature’ chooses the actual wave heights which the structure has to resist.

But nature includes creatures other than ourselves! In activities in which other human beings – or indeed, animals – are involved, we cannot rely on any kind of indifference. If we set up a business, our competitors are out to do better than us, and this implies trying to take business from us. Strategies will be worked out to achieve this objective. We can no longer rely on probabilistic estimates of our opponent's strategies based on a study of ‘impartial’ nature, as we might do in studying a record of wave heights.

Introduction

We have described various probability distributions, on the assumption that the parameters are known. We have dealt with classical assignments of probability, in which events were judged as being equally likely, and their close relation, the maximum-entropy distributions of Chapter 6. We have considered the Central Limit Theorem – the model of sums – and often this can be used to justify the use of the normal distribution. We might have an excellent candidate distribution for a particular application. The next step in our analysis is to estimate the model parameters, for example the mean µ and variance σ2 for a normal distribution. We often wish to use measured data to estimate these parameters. The subject of the present chapter, ‘inference’, refers to the use of data to estimate the parameter(s) of a probability distribution. This is key to the estimation of the probability of a future event or quantity of a similar kind. The fitting of distributions to data is outlined in Chapter 11.

References of interest for this chapter are de Finetti (1937, 1974), Raiffa and Schlaifer (1961), Lindley (1965, Part 2), Heath and Sudderth (1976), O'Hagan (1994) and Bernardo and Smith (1994).

What is risk? Its analysis as decision theory

It is commonplace to make statements such as ‘life involves risk’, or ‘one cannot exist without facing risk’, or ‘even if one stays in bed, there is still some risk – for example, a meteorite could fall on one, or one could fall out and sustain a fatal injury’, and so on. The Oxford dictionary defines risk as a ‘hazard, chance of … bad consequences, loss, …’; Webster's definition is similar: ‘the chance of injury, damage or loss’ or ‘a dangerous chance’. Two aspects thrust themselves forward: first, chance, and second, the unwanted consequences involved in risk. We have taken pains to emphasize that decision-making involves two aspects: probabilities (chance), and utilities (consequences). Protection of human life, property, and the environment are fundamental objectives in engineering projects. This involves the reduction of risk to an acceptable level. The word ‘safety’ conveys this overall objective.

In Chapter 4, aversion to risk was analysed in some detail. The fundamental idea was incorporated in the utility function. We prompt recollection of the concept by using an anecdote based on a suggestion of a colleague. A person is abducted by a doctor who is also a fanatical decision-theorist.