This week we will focus on transformations of random variables.
We started today with the following warm-up problem:
If \(U\) is a \(\operatorname{Unif}(0,1)\) random variable, then \(50+50U\) will be a \(\text{Unif}(50,100)\) random variable. What about other transformations of random variables? For situations like that, we have the following change of variables theorem.
If \(X\) is a continuous random variable with density \(f_X\) and \(Y = g(X)\) where \(g\) is a strictly increasing (or decreasing) differentiable function, then \[f_Y(y) = f_X(x) \left| \frac{dx}{dy} \right|.\] The support of \(Y\) is the interval from \(g(x_\text{min})\) to \(g(x_\text{max})\) where \((x_\text{min},x_\text{max})\) is the support of \(X\).
According to the Change of Variables Theorem, what is the density function for \(50+50U\)?
Suppose \(Y = -\ln(X)\) where \(X \sim \operatorname{Unif}(0,1)\). What is the probability distribution of \(Y\)?
What is the density function for a linear tranformation \(a+bX\) of an \(\operatorname{Exp}(\lambda)\) r.v. \(X\)? What is the support of the density function?
So why does this work?
Let’s assume \(g\) is a strictly increasing function. (Showing that the theorem is true when \(g\) is strictly decreasing is Homework 2, Problem 4). Strictly increasing means that \(a < b\) iff \(g(a) < g(b)\). To find the density function for \(Y\), first we’ll write down the CDF for \(Y\): \[\begin{align*}F_Y(y) &= P(Y \le y) \hspace{0.85in} \text{(definition of CDF)} \\ &= P(g(X) \le y) ~~~~ ~~~~ ~~~ \text{(substitution)} \\ &= P(X \le g^{-1}(y)) \hspace{0.4in} \text{(since }g\text{ is strictly increasing)} \\ &= F_X(g^{-1}(y)). \end{align*}\] To get the PDF from the CDF, take the derivative of both sides with respect to \(y\). Then by the chain rule, this is: \[f_Y(y) = f_X(g^{-1}(y)) \frac{dx}{dy}.\] Since \(g\) is increasing, so is \(g^{-1}\) and therefore \(dx/dy\) is nonnegative. \(\square\)
Today we covered multivariate transformations of random variables. But first, we looked at an example where the Change of Basis theorem didn’t apply directly, but we were able to use the idea of the proof to make things work.
Let \(Y = X^2\) where \(X \sim \operatorname{Norm}(0,1)\).
Why doesn’t the change of basis theorem apply to \(Y\) and \(X\)?
Write the CDF for \(Y\) (\(F_Y(y) = P(Y \le y)\)) in terms of the CDF for \(X\).
Differentiate both sides with respect to \(y\) to find the density function \(f_Y\).
Once we worked out that example, we moved on to the main result of today:
Suppose \(X = (X_1, X_2, \ldots, X_n)\) is a continuous random vector with joint density \(f_X(x)\) and \(Y = g(X)\) where \(g: \mathbb{R}^n \rightarrow \mathbb{R}^n\) is function that satisfies the following assumptions:
Then \[f_Y(y) = f_X(x) \left| \frac{\partial x}{\partial y} \right|\] where \(\displaystyle \left| \frac{\partial x}{\partial y} \right|\) is the absolute value of the determinant of the Jacobian matrix.
Suppose that \(U \sim \text{Unif}(0,2\pi)\) and \(T \sim \text{Exp}(1)\) are independent random variables. Let \[\begin{align*} X &= \sqrt{2T} \cos U \\ Y &= \sqrt{2T} \sin U. \\ \end{align*}\]
Show that \(T = \frac{1}{2}(X^2+Y^2)\).
Find a formula for \(U\) (Hint: what is \(Y/X\)?).
Find the Jacobian matrix \[\frac{\partial (t, u)}{\partial (x, y)} = \begin{pmatrix} \partial t/\partial x & \partial t/\partial y \\ \partial u/\partial x & \partial u/\partial y \end{pmatrix},\] and show that the determinant is always 1.
Use the Multivariate Change of Variables Theorem to show that the joint density for \((X,Y)\) is \[f_{(X,Y)}(x,y) = \frac{1}{2\pi} e^{-(x^2+y^2)/2}.\]
Observe that \(X\) and \(Y\) are independent \(\text{Norm}(0,1)\) random variables since \(f_{(X,Y)}(x,y)\) factors into \[\left(\frac{1}{\sqrt{2\pi}} e^{-x^2/2} \right)\left(\frac{1}{\sqrt{2\pi}} e^{-y^2/2} \right).\]