What Is a P-Value?

Hypothesis tests or test of significance involve the calculation of a number known as a p-value. This number is very important to the conclusion of our test. P-values are related to the test statistic and give us a measurement of evidence against the null hypothesis.

Null and Alternative Hypotheses

Tests of statistical significance all begin with a null and an alternative hypothesis. The null hypothesis is the statement of no effect or a statement of commonly accepted state of affairs. The alternative hypothesis is what we are attempting to prove. The working assumption in a hypothesis test is that the null hypothesis is true.

Test Statistic

We will assume that the conditions are met for the particular test that we are working with. A simple random sample gives us sample data. From this data we can calculate a test statistic. Test statistics vary greatly depending upon what parameters our hypothesis test concerns. Some common test statistics include:

• z - statistic for hypothesis tests concerning the population mean, when we know the population standard deviation.
• t - statistic for hypothesis tests concerning the population mean, when we do not know the population standard deviation.
• t - statistic for hypothesis tests concerning the difference of two independent population mean, when we do not know the standard deviation of either of the two populations.
• z - statistic for hypothesis tests concerning a population proportion.
• Chi-square - statistic for hypothesis tests concerning the difference between an expected and actual count for categorical data.

Calculation of P-Values

Test statistics are helpful, but it can be more helpful to assign a p-value to these statistics. A p-value is the probability that, if the null hypothesis were true, we would observe a statistic at least as extreme as the one observed. To calculate a p-value we use the appropriate software or statistical table that corresponds with our test statistic.

For example, we would use a standard normal distribution when calculating a z test statistic. Values of z with large absolute values (such as those over 2.5) are not very common and would give a small p-value. Values of z that are closer to zero are more common, and would give much larger p-values.

Interpretation of the P-Value

As we have noted, a p-value is a probability. This means that it is a real number from 0 and 1. While a test statistic is one way to measure how extreme a statistic is for a particular sample, p-values are another way of measuring this.

When we obtain a statistical given sample, the question that we should always is, “Is this sample the way it is by chance alone with a true null hypothesis, or is the null hypothesis false?” If our p-value is small, then this could mean one of two things:

1. The null hypothesis is true, but we were just very lucky in obtaining our observed sample.
2. Our sample is the way it is due to the fact that the null hypothesis is false.

In general, the smaller the p-value, the more evidence that we have against our null hypothesis.

How Small Is Small Enough?

How small of a p-value do we need in order to reject the null hypothesis? The answer to this is, “It depends.” A common rule of thumb is that the p-value must be less than or equal to 0.05, but there is nothing universal about this value.

Typically, before we conduct a hypothesis test, we choose a threshold value. If we have any p-value that is less than or equal to this threshold, then we reject the null hypothesis. Otherwise we fail to reject the null hypothesis. This threshold is called the level of significance of our hypothesis test, and is denoted by the Greek letter alpha. There is no value of alpha that always defines statistical significance.