Independence does not imply conditional independence

Given a probability measure $P$ on a sample space $S$ and an event $F$ , we could have a set function denoted $P(.|F)$ which is defined for each event (measurable set) $A$ as $P(A|F) = \frac{P(A)}{P(F)}$ .

It is easy to see that $P(.|F)$ is itself a probability measure. That is, it satisfies the three axioms of probability namely:

For all events $A$ , $0 \le P(A|F) \le 1$
$P(S|F) = 1$
If $E_1, E_2, E_3,\cdots$ are mutually exclusive events, then $P\left(\left(\bigcup\limits_{n} E_n\right)|F\right) = \sum\limits_n P(E_n|F)$ .

If we denote $P(A|F)$ by $Q(A)$ , $Q$ defines a new probability measure on $S$ . Thus, $Q$ satisfies all properties of a probability measure.

For example, we have:

$Q\left(A\cup B\right) = Q(A) + Q(B) - Q\left(A\cap B\right)$ , or equivalently

$P\left(A\cup B|F\right) = P(A|F) + P(B|F) - P\left(AB|F\right)$

(Note that in this article we use $AB$ to denote $A\cap B$ )

Likewise,
$P\left(A^c|F\right) = 1 - P(A|F)$

Now for the conditional probability measure $Q$ , we could in turn define conditional probability of events the usual way: $Q(A|B) = \frac{Q(AB)}{Q(B)}$ . By this definition,

$Q(A|B) = \frac{Q(AB)}{Q(B)} = \frac{P(AB|F)}{P(B|F)} = \frac{P(ABF)}{P(BF)} = P(A|BF)$

Thus, the conditional probability of $A$ given $B$ , given $F$ is probability of $A$ given $B\cap F$ (informally, $P(A|B|F) = P(A|BF)$ ).

We also have equivalent of Bayes’ formula with conditional probability. If $A_1, A_2, \cdots, A_n$ form a partition of $S$ , then:

$P(A_i|BF) = \frac{P(B|A_iF)P(A_i|F)}{\sum\limits_{j=1}^n P(B|A_jF)P(A_j|F)}$

We now get to the concept of conditional independence of events $A$ and $B$ given event $F$ . We define two events $A$ and $B$ to be conditionally independent given (that) $F$ (has occurred) if and only if $P(A|BF) = P(A|F)$ . (Again, this can be remembered as either $P(A|B|F) = P(A|F)$ (informally) or $Q(A|B) = Q(A)$ , just like the independence of events).

Since $P(A|BF) = \frac{P(AB|F)}{P(B|F)}$ , we have $A$ and $B$ are conditionally independent given $F$ if and only if $P(AB|F) = P(A|F)P(B|F)$ .

Finally, we observe that neither independence implies conditional independence nor conditional independence implies independence. To justify the former consider the example: $S = \{1,2,3,4,5,6,7,8\}, A = \{1,3,5,7\}, B = \{7,8\}, F=\{6,7\}$ Then, with uniform probability distribution, we see that $A$ and $B$ are independent, but not conditionally independent given $F$ . To justify the latter, consider the example: $S = \{1,2,3,4,5,6,7,8\}, A = \{1,3,5,6,7,8\}, B=\{3,4\}, F = \{1,2,3,4\}$ . Then, again with uniform probability distribution, we see that $A$ and $B$ are conditionally independent given $F$ , but not just independent.