The purpose and philosophy of significance testing

Possibly no other subject within the broad area of statistics has given rise to so much misunderstanding as ‘significance testing’. This may be in part because non-statistical meanings of the words tend to get in the way, but also because statisticians have not been able to reach a consensus on how significance tests should be used or interpreted. Since significance tests were developed, effectively in the first half of the twentieth century, there has been a lively debate over the philosophy of significance tests and what they mean. Statisticians working within a variety of other disciplines have differed in their views. The controversies have sometimes been almost theological in their approaches and in their semantic subtlety, and occasionally in their acrimoniousness (see Morrison and Henkel [43]).

Not much has changed in this respect in the last fifty years. What has changed is that significance testing has become more accessible to the general researcher. No longer is a good grounding in statistics necessary to perform a ‘significance test’: anyone with access to one of the more popular statistical software packages can carry out a significance test procedure at the click of a mouse, without having to know anything about it at all.