Week 3 Lecture Notes

This week we will focus on transformations of random variables.

Wednesday, January 29

We started today with the following warm-up problem:

Most programming languages have a built in random number generator function that will generate (psuedo)random numbers between 0 and 1. How could you use a random number generator like that to generate a number between 50 and 100?

If $U$ is a $\operatorname{Unif}(0,1)$ random variable, then $50+50U$ will be a $\text{Unif}(50,100)$ random variable. What about other transformations of random variables? For situations like that, we have the following change of variables theorem.

Theorem (Change of Variables)

If $X$ is a continuous random variable with density $f_X$ and $Y = g(X)$ where $g$ is a strictly increasing (or decreasing) differentiable function, then $f_Y(y) = f_X(x) \left| \frac{dx}{dy} \right|.$ The support of $Y$ is the interval from $g(x_\text{min})$ to $g(x_\text{max})$ where $(x_\text{min},x_\text{max})$ is the support of $X$ .

According to the Change of Variables Theorem, what is the density function for $50+50U$ ?
Suppose $Y = -\ln(X)$ where $X \sim \operatorname{Unif}(0,1)$ . What is the probability distribution of $Y$ ?
What is the density function for a linear tranformation $a+bX$ of an $\operatorname{Exp}(\lambda)$ r.v. $X$ ? What is the support of the density function?

So why does this work?

Proof of Change of Variables Theorem

Let’s assume $g$ is a strictly increasing function. (Showing that the theorem is true when $g$ is strictly decreasing is Homework 2, Problem 4). Strictly increasing means that $a < b$ iff $g(a) < g(b)$ . To find the density function for $Y$ , first we’ll write down the CDF for $Y$ : $$\begin{align*}F_Y(y) &= P(Y \le y) \hspace{0.85in} \text{(definition of CDF)} \\ &= P(g(X) \le y) ~~~~ ~~~~ ~~~ \text{(substitution)} \\ &= P(X \le g^{-1}(y)) \hspace{0.4in} \text{(since }g\text{ is strictly increasing)} \\ &= F_X(g^{-1}(y)). \end{align*}$$ To get the PDF from the CDF, take the derivative of both sides with respect to $y$ . Then by the chain rule, this is: $f_Y(y) = f_X(g^{-1}(y)) \frac{dx}{dy}.$ Since $g$ is increasing, so is $g^{-1}$ and therefore $dx/dy$ is nonnegative. $\square$

Study Question

We saw in class today that if you randomly generate a number $X$ uniformly between 0 and 1, then $Y = -\ln(X)$ will have an $\text{Exp}(1)$ distribution. What if you wanted a random number with an $\text{Exp}(\lambda)$ distribution where $\lambda \ne 1$ ? How could you generate a random number with that distribution?

Friday, January 31

Today we covered multivariate transformations of random variables. But first, we looked at an example where the Change of Basis theorem didn’t apply directly, but we were able to use the idea of the proof to make things work.

Let $Y = X^2$ where $X \sim \operatorname{Norm}(0,1)$ .

Why doesn’t the change of basis theorem apply to $Y$ and $X$ ?
Write the CDF for $Y$ ( $F_Y(y) = P(Y \le y)$ ) in terms of the CDF for $X$ .
Differentiate both sides with respect to $y$ to find the density function $f_Y$ .

Once we worked out that example, we moved on to the main result of today:

Theorem (Multivariate Change of Variables)

Suppose $X = (X_1, X_2, \ldots, X_n)$ is a continuous random vector with joint density $f_X(x)$ and $Y = g(X)$ where $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is function that satisfies the following assumptions:

$g$ is invertible.
All partial derivative $\partial x_i/\partial y_j$ of $g^{-1}$ exist and are continuous, and
The Jacobian matrix $\frac{\partial x}{\partial y} = \begin{pmatrix} \partial x_1/\partial y_1 & \ldots & \partial x_1/\partial y_n \\ \vdots & & \vdots \\ \partial x_n/\partial y_1 & \ldots & \partial x_n/\partial y_n \end{pmatrix}$ has nonzero determinant.

Then $f_Y(y) = f_X(x) \left| \frac{\partial x}{\partial y} \right|$ where $\displaystyle \left| \frac{\partial x}{\partial y} \right|$ is the absolute value of the determinant of the Jacobian matrix.

Example: Box-Muller Transformation

Suppose that $U \sim \text{Unif}(0,2\pi)$ and $T \sim \text{Exp}(1)$ are independent random variables. Let $\begin{align*} X &= \sqrt{2T} \cos U \\ Y &= \sqrt{2T} \sin U. \\ \end{align*}$

Show that $T = \frac{1}{2}(X^2+Y^2)$ .
Find a formula for $U$ (Hint: what is $Y/X$ ?).
Find the Jacobian matrix $\frac{\partial (t, u)}{\partial (x, y)} = \begin{pmatrix} \partial t/\partial x & \partial t/\partial y \\ \partial u/\partial x & \partial u/\partial y \end{pmatrix},$ and show that the determinant is always 1.
Use the Multivariate Change of Variables Theorem to show that the joint density for $(X,Y)$ is $f_{(X,Y)}(x,y) = \frac{1}{2\pi} e^{-(x^2+y^2)/2}.$
Observe that $X$ and $Y$ are independent $\text{Norm}(0,1)$ random variables since $f_{(X,Y)}(x,y)$ factors into $\left(\frac{1}{\sqrt{2\pi}} e^{-x^2/2} \right)\left(\frac{1}{\sqrt{2\pi}} e^{-y^2/2} \right).$