Independence

Ref:

1. NOTES ON PROBABILITY by Greg Lawler
2. Probability_Theory by Kyle Siegrist
3. The Book of Statistical Proofs

Independence of Events

Let $A_1,\cdots ,A_n$ be $n$ arbitrary statements about random variables $X_1,X_2,\cdots ,X_n$ with possible values ${\mathcal{X}}_1,{\mathcal{X}}_2,\cdots ,{\mathcal{X}}_n$ ,

$A_1,\cdots ,A_n$ is called statistically independent, if $\text{for all}{x}_i\in {\mathcal{X}}_i,i=1,\dots ,n$: $$p(\bigcap _{i=1}^n{A}_i)=\prod _{i=1}^{n}p(X_i=x_i)$$

Note: $$p(\bigcap _{i=1}^n{A}_i)=p(A_1,\cdots ,A_n)=p(X_1=x_1,\cdots ,X_n=x_n)$$

where $p(X_1=x_1,\cdots ,X_n=x_n)$ are the joint probabilities of $X_1=x_1,\cdots ,X_n=x_n$ and $p(X_i=x_i)$ are the marginal probabilities of $A_i:X_i=x_i$.

Independence of Random Variables

Two random variables $X$ and $Y$ are independent <==> the elements of the π-system generated by them are independent; that is: $$\text{P}(X\le x,Y\le y)=\text{P}(X\le x)\text{P}(Y\le y)$$

Conditional Independence

As noted at the beginning of our discussion, independence of events or random variables depends on the underlying probability measure.

Thus, suppose that $B$ is an event with positive probability. A collection of events or a collection of random variables is conditionally independent given $B$ if the collection is independent relative to the conditional probability measure $A\to p(A|B)$.

For example, a collection of events $p(\bigcap _{i=1}^n{A}_i)$ is conditionally independent given $B$ if for every $i$: $$p(\bigcap _{i=1}^n{A}_i|B)=\prod _{i=1}^{n}p(A_i|B)$$ Note: $$\begin{gathered} p(\bigcap _{i=1}^n{A}_i|B)=p(X_1=x_1,\dots ,X_n=x_n|Y=y), \\ \prod _{i=1}^{n}p(A_i|B)=\prod _{i=1}^{n}p(X_i=x_i|Y=y),\text{for all}{x}_i\in {\mathcal{X}}_i\text{and all}y\in \mathcal{Y} \end{gathered}$$

where $X_1,X_2,\cdots ,X_n$ are discrete random variables with possible values ${\mathcal{X}}_1,{\mathcal{X}}_2,\cdots ,{\mathcal{X}}_n$; $Y$ is discrete random variable with possible values $\mathcal{Y}$. $$p(X_1=x_1,\dots ,X_n=x_n|Y=y)=\prod _{i=1}^{n}p(X_i=x_i|Y=y)$$