The Link Between Hypothesis Testing and Proof by Contradiction

Hypothesis testing has some similarities with one of the simplest methods of proof from mathematical logic: Proof by contradiction. Hypothesis testing can be thought of as being the probabilistic counterpart of proof by contradiction.

Let us refresh our memories first about this method. How does proof by contradiction work?

Let’s say we want to prove a statement, call it A. When we use proof by contradiction, we start by assuming the opposite of A (denoted ~A or not A) as being true, and then we try to reach to a contradiction (a statement that is always false). If we reach a contradiction, then this means that our assumption that ~A is true, should be false, and therefore A should be true.

For example, let’s prove by contradiction that there is no smallest positive rational number. Now A is the statement that we want to prove: “there is no smallest rational number greater than 0”. We start by assuming the opposite of A being true, ~A: “there exists a rational number greater than 0 that is the smallest, call it r”. But if r is a rational number > 0, this means that r/2 is also a rational number > 0. Moreover, r/2 is smaller than r. But this contradicts our assumption, ~A, that r is the smallest rational number > 0. This means that ~A is false, and therefore, A is true.

A downside of this method, and of pure mathematical proofs in general, is that it cannot be directly applied in many real-world situations. Let’s say you want to show that a certain type of medical treatment is effective. Can you use a pure mathematical proof to show that? No. There are too many unknowns and too much uncertainty in such situations to be able to show things in a purely mathematical way; that is to prove with no doubt that a certain thing is true.

But you can show that a certain thing has some probability of being true. That’s what statistics do, and there is this method in statistics called hypothesis testing that, in my opinion, borrowed the idea of a proof by contradiction and added a probabilistic dimension to it.

The idea of hypothesis testing is similar to that of proof by contradiction, but this time ~A is called the “null hypothesis” and denoted by H₀, and A is called the “alternative hypothesis” and denoted by H₁. Reaching a contradiction (that is, obtaining a result that cannot be true) is now reaching a result that can be true with low probability. How low this probability should be? An often-used value is 0.05, but it can also be something else, preferably smaller, depending on how confident you want to be of your result; this threshold value is called a significance level and is denoted by α. So, if the probability of your observed data given the null hypothesis is ≤ α, then you consider this probability being too low for H₀ to be true; so, you will reject H₀ and therefore accept H₁. In the case of hypothesis testing, instead of proving H₁ (that is, with 100% confidence; this is the case for pure mathematical proofs), you are 1-α confident that H₁ is true.

Note that we can accept only H₁, not H₀. Just because we may fail to reject H₀ sometimes, this does not mean that we accept it. As in the case of proof by contradiction, just because we may not have a good idea of how to reach a contradiction once we assumed ~A to be true, this does not necessarily mean that ~A is true.

Concluding that H₀ is true based on the initial assumption that H₀ is true would be a paradox, therefore we can only reject H₀.

This is my opinion regarding these 2 methods that may seem very distinct at a glance, as they are applied in vastly different situations, but in fact, they are both based on similar logical ideas.

I hope you found this story interesting and thanks for reading!

This article is also posted on my own website here. Feel free to have a look!