In this book we have investigated to what extent risk assessments and in particular quantitative risk assessments, meet the scientific quality requirements of reliability and validity. While reliability is concerned with the consistency of the “measuring instrument” (analysts, experts, methods, procedures), validity is concerned with the success at “measuring” what one set out to “measure” in the analysis. For each of these main criteria a set of sub-criteria are defined.

To be able to perform these investigations we have to study the scientific building blocks of the assessments (related to probability, risk, uncertainty, models, etc.), but also the role of the assessments in the decision-making process. What type of decision support should the assessments provide? Are the objectives (expectations) accurate risk estimates and/or uncertainty characterisations? The scientific quality of the assessments obviously needs to be seen in relation to the objectives. Also the requirements of reliability and validity depend on these objectives. Using these criteria, we evaluate the quality of the assessments for different objectives of the assessments. Three case studies are used to illustrate the analysis. The first of these examples is related to the analysis of accident data, the second relates to the siting of an LNG plant and the third discusses the design of a safety system.

The investigation shows that the reliability and validity criteria are not in general satisfied. Under certain conditions the criteria are met, and to a varying degree depending on the risk perspective and aim of the risk assessment.