Sources:

1. Jeseph K. Blitzstein & Jessica Hwang. (2019). Conditional propability. Introduction to Probability (2nd ed., pp. 369-375). CRC Press.

Change of Random Variables

Notation

Symbol	Type	Description
\(X\)	Random variable	Original continuous random variable
\(f_X(x)\)	Function	Probability density function (PDF) of \(X\)
\(Y\)	Random variable	Transformed continuous random variable
\(f_Y(y)\)	Function	Probability density function (PDF) of \(Y\)
\(g(x)\)	Function	Transform function, assumed to be differentiable and strictly monotonic
\(g^{-1}(y)\)	Function	Inverse of the transformation function \(g(x)\)
\(\mathbf{X}\)	Vector	Original random vector \((X_1, X_2, \ldots, X_n)\)
\(f_{\mathbf{X}}(\mathbf{x})\)	Function	Joint PDF of \(\mathbf{X}\)
\(\mathbf{Y}\)	Vector	Transformed random vector \((Y_1, Y_2, \ldots, Y_n)\)
\(f_{\mathbf{Y}}(\mathbf{y})\)	Function	Joint PDF of \(\mathbf{Y}\)
\(\frac{\partial \mathbf{x}}{\partial \mathbf{y}}\)	Matrix	Jacobian matrix of partial derivatives of \(g^{-1}\)
\(\det\left( \frac{\partial \mathbf{x}}{\partial \mathbf{y}} \right)\)	Scalar	Determinant of the Jacobian matrix
\(A_0, B_0\)	Sets	Open subsets of \(\mathbb{R}^n\), representing domain and range of \(g\), respectively

Abbreviations

Abbreviation	Description
PDF	Probability Density Function
CDF	Cumulative Distribution Function
r.v.	Random variable
Jacobian	Matrix of partial derivatives

Change of variables in one dimension

Theorem: Change of variables in one dimension Let \(X\) be a continuous r.v. with PDF \(f_X\), and let \(Y=g(X)\), where \(g\) is differentiable and strictly increasing (or strictly decreasing). Then the PDF of \(Y\) is given by \[ f_Y(y)=f_X(x)\left|\frac{d x}{d y}\right|, \]

where \(x=g^{-1}(y)\). The support of \(Y\) is all \(g(x)\) with \(x\) in the support of \(X\).

Proof:

Let \(g\) be strictly increasing. The CDF of \(Y\) is \[ F_Y(y)=P(Y \leq y)=P(g(X) \leq y)=P\left(X \leq g^{-1}(y)\right)=F_X\left(g^{-1}(y)\right)=F_X(x), \] so by the chain rule, the PDF of \(Y\) is \[ f_Y(y)= \frac{\partial F_Y(y)} {\partial y} = \frac{\partial F_X(x)} {\partial y} = \frac{\partial F_X(x)} {\partial x} \frac{\partial x} {\partial y} = f_X(x) \frac{d x}{d y} \] The proof for \(g\) strictly decreasing is analogous. In that case the PDF ends up as \(-f_X(x) \frac{d x}{d y}\), which is nonnegative since \(\frac{d x}{d y}<0\) if \(g\) is strictly decreasing. Using \(\left|\frac{d x}{d y}\right|\), as in the statement of the theorem, covers both cases.

Change of variables in multiple dimensions

Theorem: Let \(\mathbf{X}=\left(X_1, \ldots, X_n\right)\) be a continuous random vector with joint PDF \(f_{\mathbf{X}}(\mathbf{x})\). Let \(g: A_0 \rightarrow B_0\) be an invertible function, where \(A_0\) and \(B_0\) are open subsets of \(\mathbb{R}^n\). Suppose \(A_0\) contains the support of \(\mathbf{X}\), and \(B_0\) is the range of \(g\).

Define \(\mathbf{Y}=g(\mathbf{X})\), with \(\mathbf{y}=g(\mathbf{x})\). Since \(g\) is invertible, we have:

\[ \mathbf{X}=g^{-1}(\mathbf{Y}) \quad \text { and } \quad \mathbf{x}=g^{-1}(\mathbf{y}) \]

Assuming all partial derivatives \(\frac{\partial x_i}{\partial y_j}\) exist and are continuous, the Jacobian matrix of the transformation is:

\[ \frac{\partial \mathbf{x}}{\partial \mathbf{y}}=\left[\begin{array}{cccc} \frac{\partial x_1}{\partial y_1} & \frac{\partial x_1}{\partial y_2} & \cdots & \frac{\partial x_1}{\partial y_n} \\ \vdots & & & \vdots \\ \frac{\partial x_n}{\partial y_1} & \frac{\partial x_n}{\partial y_2} & \cdots & \frac{\partial x_n}{\partial y_n} \end{array}\right] \]

If \(\operatorname{det}\left(\frac{\partial \mathrm{x}}{\partial \mathrm{y}}\right) \neq 0\) everywhere, the joint PDF of \(\mathbf{Y}\) is:

\[ f_{\mathbf{Y}}(\mathbf{y})=f_{\mathbf{X}}\left(g^{-1}(\mathbf{y})\right) \cdot\left|\operatorname{det}\left(\frac{\partial \mathbf{x}}{\partial \mathbf{y}}\right)\right| . \]

This holds for all \(\mathbf{y} \in B_0\).