p<.05, what does it mean according to ASA?

[gayle_yamazaki]

There have been quite a few articles examining the shortcomings of research across many disciplines. One of the elements that have been sited as problematic is the understanding and interpretation of the meaning of meeting the test of statistical significance. As a result, the journal for the American Statistical Association, The American Statistician, recently posted an article on the context, process, and purpose of the using the p-value test by Ronald Wasserstein and Nicole Lazar.

The article opens with a discussion about why the p-value is still used and the problems of using statistical analyses that are poorly understood and how the article came to be. As a result, they recently published their statement in which they clarify "several widely agreed upon principles underlying the proper use and interpretation of the p-value."

I strongly encourage reading the actual article/statement. It is relatively short and easy to read. It can be found at this link for a free download.

Here is a list of the six basic principles regarding the use and interpretation of the p-value according to the ASA.

1) P-values can indicate how incompatible the data are with a specified statistical model.

2) P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.

3) Scientific conclusion and business or policy decisions should not be based only on whether a p-value passes a specific threshold.

4) Proper inferences requires full reporting and transparency.

5) A p-value, or statistical significance, does not measure the size of the effect or the importance of a result.

6) By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

The next time you read a scientific paper, these are interesting points to consider when evaluating what they conclude about the statistical analysis performed and the author's interpretation of the findings.

In case you're wondering, the ASA does provide some alternative methods an investigator can use. Since journals and our peers are not likely to abandon the p-value any time soon, you may want to consider adding them as part of your analysis and interpretive processes.