In this chapter we discuss a few cases of scientific misconduct that turned out easy to spot, given some basic knowledge of statistics. We learn that it is always important to begin with a close look at the data that you are supposed to analyze. What is the source of the data, how were they collected, and who collected them and for what purpose? Next, we discuss various specific cases where the misconduct was obvious. We see that it is not difficult to create tables with fake regression outcomes, and that it is also not difficult to generate artificial data that match with those tables. Sometimes results are too good to be true. Patterns in outcomes can be unbelievable. We also see that it is not difficult to make the data fit better to a model. These are of course all unethical approaches and should not be replicated, but it is good to know that these things can happen and how.

The first chapter contains an overview of what is accepted as good practice. We review several general ethical guidelines. These can be used to appreciate good research and to indicate where and how research does not adhere to them. Good practice is “what we all say we (should) adhere to.” In the second part of this chapter, the focus is more on specific ethical guidelines for statistical analysis. Of course, there is overlap with the more general guidelines, but there are also a few specifically relevant to statistics: Examples are misinterpreting p values and malpractice such as p hacking and harking.

In practice it often happens that forecasts from econometric models are manually adjusted. There can be good reasons for this. Foreseeable structural changes can be incorporated. Recent changes in data, in measurement or in the relevance of variables, can be addressed. A main issue with manual adjustment is that the end user of a forecast needs to know why someone modified a forecast and, next, how that forecast was changed. This should therefore be documented. We discuss an example to show that one may also need to know specific details of econometric models, here growth curves, to understand that even a seemingly harmless adjustment by a priori fixing the point of inflection leads to any result that you would like. In this chapter we discuss why people manually adjust forecasts. We discuss the optimal situation when it comes to adjustment and the experience with manual adjustment so far. A plea is made to consider model-based adjustment of model forecasts, thus allowing for a clear understanding of how and why adjustment was made.

In this chapter we move towards more subtle aspects of econometric analysis, where it is not immediately obvious from the numbers or the graphs that something is wrong. We see that so-called influential observations may not be visible from graphs but become apparent after creating a model. This is one of the key takeaways from this chapter – that we do not throw away data prior to econometric analysis. We should incorporate all observations in our models and, based on specific diagnostic measures, decide which observations are harmful.

Econometricians develop and use methods and techniques to model economic behavior, create forecasts, to do policy evaluation, and to develop scenarios. Often, this ends up in advice. This advice can relate to a prediction for the future or for another sector or country, it can be a judgment on whether a policy measure was successful or not, or suggest a possible range of futures. Econometricians (must) make choices that can often only be understood by fellow econometricians. A key claim in this book is that it is important to be clear on those choices. This introductory chapter briefly describes the contents of all following chapters.

This chapter deals with features of data that suggest a certain model or method, but where this suggestion is erroneous. We highlight a few cases in which an econometrician could be directed in the wrong direction, and at the same time we show how this can be prevented from happening. These situations happen in cases where there is no strong prior information on how the model should be specified. The data are then used to guide model construction. This guidance can be in an inappropriate direction. We review a few empirical cases where some data features obscure a potentially proper view of the data and may suggest inappropriate models. We discuss spurious cycles and the impact of additive outliers on detecting ARCH and nonlinearity. We also focus on a time series that may exhibit recessions and expansions, allowing you to (wrongly) interpret the recession observations as outliers. Finally, we deal with structural breaks and trends and unit roots, and see how data with these features can look alike.

This last chapter summarizes most of the material in this book in a range of concluding statements. It provides a summary of the lessons learned. These lessons can be viewed as guidelines for research practice.

We first discuss a phenomenon called data mining. This can involve multiple tests on which variables or correlations are relevant. If used improperly, data mining may associate with scientific misconduct. Next, we discuss one way to arrive at a single final model, involving stepwise methods. We see that various stepwise methods lead to different final models. Next, we see that various configurations in test situations, here illustrated for testing for cointegration, lead to different outcomes. It may be possible to see which configurations make most sense and can be used for empirical analysis. However, we suggest that it is better to keep various models and somehow combine inferences. This is illustrated by an analysis of the losses in airline revenues in the United States owing to 9/11. We see that out of four different models, three estimate a similar loss, while the fourth model suggests only 10 percent of that figure. We argue that it is better to maintain various models, that is, models that stand various diagnostic tests, for inference and for forecasting, and to combine what can be learned from them.

In practice it may happen that a first-try econometric model is not appropriate because it violates one or more of the key assumptions that are needed to obtain valid results. In case there is something wrong with the variables, such as measurement error or strong collinearity, we may better modify the estimation method or change the model. In the present chapter we deal with endogeneity, which can, for example, be caused by measurement error, and which implies that one or more regressors are correlated with the unknown error term. This is of course not immediately visible because the errors are not known beforehand and are estimated jointly with the unknown parameters. Endogeneity can thus happen when a regressor is measured with error, and, as we see, when the data are aggregated at too low a frequency. Another issue is called multicollinearity, in which it is difficult to disentangle (the statistical significance of) the separate effects. This certainly holds for levels and squares of the same variable. Finally, we deal with the interpretation of model outcomes.

This chapter opens with some quotes and insights on megaprojects. We turn to the construction and the use of prediction intervals in a time series context. We see that depending on the choice of the number of unit roots (stochastic trends) or the sample size (when does the sample start?), we can compute a wide range of prediction intervals. Next, we see that those trends, and breaks in levels and breaks in trend, can yield a wide variety of forecasts. Again, we reiterate that maintaining a variety of models and outcomes is useful, and that an equal-weighted combination of results can be most appropriate. Indeed, any specific choice leads to a different outcome. Finally, we discuss for a simple first-order autoregression how you can see what the limits to predictability are. We see that these limits are closer than we may think at the onset.

This chapter uses a range of quotes and findings from the internet and the literature. The key premises of this chapter, which is illustrated with examples, are as follows. First, Big Data requires the use of algorithms. Second, algorithms can create misleading information. Third, algorithms can lead to destructive outcomes. But we should not forget that humans program algorithms. With Big Data come algorithms to run many and involved computations. We cannot oversee all these data ourselves, so we need the help of algorithms to make computations for us. We might label these algorithms as Artificial Intelligence, but this might suggest that they can do things on their own. They can run massive computations, but they need to be fed with data. And this feeding is usually done by us, by humans, and we also choose the algorithms to be used.

In practice we do not always have clear guidance from economic theory about specifying an econometric model. At one extreme, it may be said that we should “let the data speak.” It is good to know that when they “speak” that what they say makes sense. We must be aware of a particularly important phenomenon in empirical econometrics: the spurious relationship. If you encounter a spurious relationship but do not recognize it as such, you may inadequately consider such a relationship for hypothesis testing or for the creation of forecasts. A spurious relationship appears when the model is not well specified. In this chapter, we see from a case study that people can draw strong but inappropriate conclusions if the econometric model is not well specified. We see that if you a priori hypothesize a structural break at a particular moment in time, and based on that very assumption analyze the data, then it is easy to draw inaccurate conclusions. As with influential observations, the lesson here is that one should first create an econometric model, and, given that model, investigate whether there could have been a structural break.

This chapter deals with missing data and a few approaches to managing such. There are several reasons why data can be missing. For example, people can throw away older data, which can sometimes be sensible. It may also be the case that you want to analyze a phenomenon that occurs at an hourly level but only have data at the daily level; thus, the hourly data are missing. It may also be that a survey is simply too long, so people get tired and do not answer all questions. In this chapter we review various situations where data are missing and how we can recognize them. Sometimes we know how to manage the situation of missing data. Often there is no need to panic and modifications of models and/or estimation methods can be used. We encounter a case in which data can be made missing on purpose, by selective sampling, to subsequently facilitate empirical analysis. Such analysis explicitly takes account of the missingness, and the impact of missing data can become minor.