Math for the People

Let's begin with some probability. One way to represent probability is with a number between 1 and 0. A quick way to remember how to calculate some probabilities is

Consider the probability of rolling a number greater than 4 on a six sided die. There are 2 numbers greater than 4 (specifically, 5 and 6), and 6 total possibilities, so the probability of rolling greater than 4 is \(\frac{2}{6}\) or approximately 0.33333.

If events \(A\) and \(B\) do not influence the probability of each other happening (they are independent events), you can calculate their joint probability by just multiplying their probabilities together. Consider the act of rolling two dice. An example of two independent events are \(A\text{:}\) rolling a 6, and then \(B\text{:}\) rolling an odd number. These events are independent because rolling a 6 with die 1 does not change the probability of rolling an odd number with die 2. Observe that \(P(A)=\frac{1}{6}\) and that \(P(B)=\frac{3}{6}\) since there are three odd numbers on a die. We can then use the formula

and plug in the values we just determined to get

Consider the space of all possible outcomes of rolling two dice, all 36 outcomes. These individual outcome probabilities are recorded in the following table:

The probability of rolling a six for one of the rolls is represented in the bottom row. The probability of rolling a six and rolling either a 1,3, or 5 are represented by the corresponding columns in the last row, If you add them up, you get a 3/36 probability.

Now consider the game of craps. One desirable outcome of the game is to roll two dice that sum to 7 or 11. An example of two events are \(A\text{:}\) rolling a number less than 4, and \(B\text{:}\) rolling two dice that sum to 7. Let's say we wanted to know the probability of rolling two dice that sum to 7, given that the first roll is less than 4. In math notation, we write this probability as \(P(B \vert A)\text{.}\)

You may remember from a high school algebra course that

and then you can use this formula to determine the probability that the two dice sum to 7 given that the first roll is less than 4.

If we take this formula and multiply both sides by \(P(A)\text{,}\) we see that

But suppose we swap events \(A\) and \(B\text{.}\) Then notice that

If we solve this for \(P(A,B)\) we get

You can also calculate this probability in terms of the probability of event \(B\text{.}\)

But notice that the probability of \(A\) and \(B\) is the same as the probability of \(B\) and \(A\text{,}\) so we know \(P(A,B)=P(B,A)\text{.}\) Plugging in the formulas we just determined for each of these, we see that

Solving for \(P(A \vert B)\text{,}\) we obtain Bayes' Theorem:

You might ask, why would you do all of this manipulation? This theorem is now ubiquitous in science, machine learning, and artificial intelligence. As we will see in later sections, Bayes' theorem allows you to make predictions and inferences with what seems like incomplete data.