Chapter 19 : Change of variable formula

Let $\mathbf{X}=(X_{1},\ldots ,X_{m})$ be a random vector with density $f(t_{1},\ldots ,t_{m})$. Let $T:\mathbb{R}^{m}\rightarrow \mathbb{R}^{m}$ be a one-one function which is continuously differentiable (many exceptions can be made as remarked later).

Let $\mathbf{Y}=T(\mathbf{X})$. In co-ordinates we may write $\mathbf{Y}=(Y_{1},\ldots ,Y_{m})$ and $Y_{1}=T_{1}(X_{1},\ldots ,X_{m})$\dots $Y_{m}=T_{m}(X_{1},\ldots ,X_{m})$ where $T_{i}:\mathbb{R}^{m}\rightarrow \mathbb{R}$ are the components of $T$.

Question : What is the joint density of $Y_{1},\ldots ,Y_{m}$?

The change of variable formula : In the setting described above, the joint density of $Y_{1},\ldots ,Y_{m}$ is given by $$ g(\mathbf{y})=f\left(T^{-1}\mathbf{y}\right)\ \pmb{\big|} \ J[T^{-1}](\mathbf{y})\ \pmb{\big|} \ $$ where $J[T^{-1}](\mathbf{y})$ is the Jacobian determinant of the function $T^{-1}$ at the point $\mathbf{y}=(y_{1},\ldots ,y_{m})$.

Justification : We shall not prove this formula, but give a imprecise but convincing justification that can be made into a proof. There are two factors on the right. The first one, $f(T^{-1}\mathbf{y})$ is easy to understand - if $\mathbf{Y}$ is to be close to $\mathbf{y}$, then $\mathbf{X}$ must be close to $T^{-1}\mathbf{y}$. The second factor involving the Jacobian determinant comes from the volume change. Let us explain with analogy with mass density which is a more familiar quantity.

Consider a solid cube with non-uniform density. If you rotate it, the density at any point now is the same as the original density, but at a different point (the one which came to the current position). Instead of rotating, suppose we uniformly expand the cube so that the center stays where it is and the side of the cube becomes twice what it is. What happens to the density at the center? It goes down by a factor of $8$. This is simply because of volume change - the same mass spreads over a larger volume. More generally, we can have non-uniform expansion, we may cool some parts of the cube, heat some parts and to varying degrees. What happens to the density? At each point, the density changes by a factor given by the Jacobian determinant.

Now for a slightly more mathematical justification. We use the language for two variables ($m=2$) but the same reasoning works for any $m$. Fix twp point $\mathbf{x}=(x_{1},x_{2})$ and $\mathbf{y}=(y_{1},y_{2})$ such that $\mathbf{y}=T(\mathbf{x})$ (and hence $\mathbf{x}=T^{-1}(\mathbf{y})$). The density of $\mathbf{Y}$ at $\mathbf{y}$ is given by $$\begin{align*} g(\mathbf{y})&\approx \frac{1}{\mbox{area}(\mathcal N)}\mathbf{P}\{\mathbf{Y}\in \mathcal N\} \end{align*}$$ where $\mathcal N$ is a small neighbourhood of the point $\mathbf{y}$ (for example a disk of small radius $\delta$ centered at $\mathbf{y}$). By the one-one nature of $T$ and the relationship $\mathbf{Y}=T(\mathbf{X})$, we see that $$\mathbf{P}\{\mathbf{Y}\in \mathcal N\}=\mathbf{P}\{\mathbf{X}\in T^{-1}(\mathcal N)\}$$ where $T^{-1}(\mathcal N)$ is the image of $\mathcal N$ after mapping by $T^{-1}$. Now, $T^{-1}(\mathcal N)$ is a small neighbourhood of $\mathbf{x}$ (if $\mathcal N$ is a disk, then $T^{-1}(\mathcal N)$ would be an approximate ellipse) and hence, by the same interpretation of density we see that $$\begin{align*} \mathbf{P}\{\mathbf{X}\in T^{-1}(\mathcal N)\}&\approx \mbox{area}(T^{-1}(\mathcal N))f(\mathbf{x}) \end{align*}$$ Putting the three displayed equations together, we arrive at the formula $$ g(\mathbf{y})\approx f(\mathbf{x})\frac{\mbox{area}(T^{-1}(\mathcal N))}{\mbox{area}(\mathcal N)} $$ Thus the problem boils down to how areas change under transformations. A linear map $S(\mathbf{y})=A\mathbf{y}$ where $A$ is a $2\times 2$ matrix changes area of any region by a factor of $|\det(A)|$, i.e., $\mbox{area}(S(\mathcal R))=|\det(A)|\mbox{area}(\mathcal R)$.

The differentiability of $T$ means that in a small neighbourhood of $\mathbf{y}$, the mapping $T^{-1}$ looks like a linear map, $T^{-1}(\mathbf{y}+\mathbf h)\approx \mathbf{x}+DT^{-1}(\mathbf{y}){\bf h}$. Therefore, the areas of small neighbourhoods of $\mathbf{y}$ change by a factor equal to $|\det(DT^{-1}(\mathbf{y}))|$ which is the Jacobian determinant. In other words, $\mbox{area}(T^{-1}(\mathcal N))\approx |JT^{-1}(\mathbf{y})|\mbox{area}(\mathcal N)$. Consequently $g(\mathbf{y})=f(T^{-1}\mathbf{y})|JT^{-1}(\mathbf{y})|$.

Enlarging the applicability of the change of variable formula : The change of variable formula is applicable in greater generality than we stated above.

1. Firstly, $T$ does not have to be defined on all of $\mathbb{R}^{m}$. It is sufficient if it is defined on the range of $\mathbf{X}$ (i.e., if $f(t_{1},\ldots ,t_{m})=0$ for $(t_{1},\ldots ,t_{m})=\mathbb{R}^{m}\setminus A$, then it is enough if $T$ is defined on $A$.
2. Even within the range of $\mathbf{X}$, we can allow $T$ to be undefined, but $\mathbf{X}$ must have zero probability to fall in the set where it is undefined. For example, it can happen at finitely many points, or on a line (if $m\ge 2$) or on a plane (if $m\ge 3$) etc.
3. Similarly, the differentiability of $T$ is required only on a subset outside of which $\mathbf{X}$ has probability $0$ of falling.
4. One-one property of $T$ is important, but there are special cases which can be dealt with by a slight modification. For example, if $T(x)=x^{2}$ or $T(x_{1},x_{2})=(x_{1}^{2},x_{2}^{2})$ where we can split the space into parts on each of which $T$ is one-one.