Let $(\Omega,p)$ be a probability space and $X:\Omega\rightarrow \mathbb{R}$ be a random variable. We define two objects associated to $X$.
Probability mass function (pmf). The range of $X$ is a countable subset of $\mathbb{R}$, denote it by $\mbox{Range}(X)=\{t_{1},t_{2},\ldots\}$. Then, define $f_{X}:\mathbb{R}\rightarrow [0,1]$ as the function
$$
f_{X}(t)=\begin{cases}
\mathbf{P}\{\omega {\; : \;} X(\omega)=t\} & \mbox{ if }t\in \mbox{Range}(X). \\
0 & \mbox{ if }t\not\in \mbox{Range}(X).
\end{cases}
$$
One obvious property is that $\sum_{t\in \mathbb{R}}f_{X}(t)=1$. Conversely, any non-negative function $f$ that is non-zero on a countable set $S$ and such that $\sum_{t\in \mathbb{R}} f(t)=1$ is a pmf of some random variable.
Cumulative distribution function (CDF). Define $F_{X}:\mathbb{R}\rightarrow [0,1]$ by
$$F_{X}(t)=\mathbf{P}\{\omega{\; : \;} X(\omega)\le t\}.$$
Example 74
Let $\Omega=\{(i,j){\; : \;} 1\le i,j\le 6\}$ with $p_{(i,j)}=\frac{1}{36}$ for all $(i,j)\in \Omega$. Let $X:\Omega \rightarrow \mathbb{R}$ be the random variable defined by $X(i,j)=i+j$. Then, $\mbox{Range}(X)=\{2,3,\ldots ,12\}$. The pmf and CDF of $X$ are given by
$$
f_{X}(k) = \begin{cases}
1/36 & \mbox{ if }k=2. \\
2/36 & \mbox{ if }k=3. \\
3/36 & \mbox{ if }k=4. \\
4/36 & \mbox{ if }k=5. \\
5/36 & \mbox{ if }k=6. \\
6/36 & \mbox{ if }k=7. \\
5/36 & \mbox{ if }k=8. \\
4/36 & \mbox{ if }k=9. \\
3/36 & \mbox{ if }k=10. \\
2/36 & \mbox{ if }k=11. \\
1/36 & \mbox{ if }k=12. \\
\end{cases}
\qquad
F_{X}(t) = \begin{cases}
0 & \mbox{ if }t < 2. \\
1/36 & \mbox{ if }t\in[2,3). \\
3/36 & \mbox{ if }t\in[3,4). \\
6/36 & \mbox{ if }t\in[4,5). \\
10/36 & \mbox{ if }t\in[5,6). \\
15/36 & \mbox{ if }t\in[6,7). \\
21/36 & \mbox{ if }t\in[7,8). \\
26/36 & \mbox{ if }t\in[8,9). \\
30/36 & \mbox{ if }t\in[9,10). \\
33/36 & \mbox{ if }t\in[10,11). \\
35/36 & \mbox{ if }t\in[11,12). \\
1 & \mbox{ if }t\ge 12. \\
\end{cases}
$$
A picture of the pmf and CDF for a Binomial distribution are shown in Figure reffig:pdfandcdfofbinomial.
Basic properties of a CDF : The following observations are easy to make.
- $F$ is an increasing function on $\mathbb{R}$.
- $\lim\limits_{t\rightarrow +\infty}F(t)=1$ and $\lim\limits_{t\rightarrow -\infty}F(t)=0$.
- $F$ is right continuous, that is, $\lim\limits_{h\searrow 0}F(t+h)=F(t)$ for all $t\in \mathbb{R}$.
- $F$ increases only in jumps. This means that if $F$ has no jump discontinuities (an increasing function has no other kind of discontinuity anyway) in an interval $[a,b]$, then $F(a)=F(b)$.
Since $F(t)$ is the probability of a certain event, these statements can be proved using the basic rules of probability that we saw earlier.
Let $t < s$. Define two events, $A=\{\omega {\; : \;} X(\omega)\le t\}$ and $B=\{\omega{\; : \;} X(\omega)\le s\}$. Clearly $A\subseteq B$ and hence $F(t)=\mathbf{P}(A)\le \mathbf{P}(B)=F(s)$. This proves the first property.
To prove the second property, let $A_{n}=\{\omega {\; : \;} X(\omega)\le n\}$ for $n\ge 1$. Then, $A_{n}$ are increasing in $n$ and $\bigcup_{n=1}^{\infty}A_{n}=\Omega$. Hence, $F(n)=\mathbf{P}(A_{n})\rightarrow \mathbf{P}(\Omega)=1$ as $n\rightarrow \infty$. Since $F$ is increasing, it follows that $\lim_{t\rightarrow +\infty}F(t)=1$. Similarly one can prove that $\lim_{t\rightarrow -\infty}F(t)=0$.
Right continuity of $F$ is also proved the same way, by considering the events $B_{n}=\{\omega {\; : \;} X(\omega)\le t+\frac{1}{n}\}$. We omit details.
The point is that probabilistic questions about $X$ can be answered by knowing its CDF $F_{X}$. Therefore, in a sense, the probability space becomes irrelevant. For example, the expected value of a random variable can be computed using its CDF only. Hence, we shall often make statements like ''$X$ is a random variable with pmf $f$'' or ''$X$ is a random variable with CDF $F$'', without bothering to indicate the probability space.
Some distributions (i.e., CDF or the associated pmf) occur frequently enough to merit a name.
Example 76
Let $f$ and $F$ be the pmf, CDF pair
$$
f(t)=\begin{cases}p & \mbox{ if }t=1, \ q & \mbox{ if }t=0, \end{cases} \qquad F_{X}(t)=\begin{cases} 1 &\mbox{ if } t\ge 1, \ q & \mbox{ if }t\in [0,1), \ 0 & \mbox{ if }t < 0. \end{cases}
$$
A random variable $X$ having this pmf (or equivalently the CDF) is said to have Bernoulli distribution with parameter $p$ and write $X\sim \mbox{Ber}(p)$. For example, if $\Omega=\{1,2,\ldots ,10\}$ with $p_{i}=1/10$, and $X(\omega)={\mathbf 1}_{\omega\le 3}$, then $X\sim \mbox{Ber}(0.3)$. Any random variable taking only the values $0$ and $1$, has Bernoulli distribution.
Example 77
Fix $n\ge 1$ and $p\in [0,1]$. The pmf defined by $f(k)=\binom{n}{k}p^{k}q^{n-k}$ for $0\le k\le n$ is called the
Binomial distribution with parameters $n$ and $p$ and is denoted Bin($n,p$). The CDF is as usual defined by $F(t)=\sum_{u:\mathbf{u}\le t}f(u)$, but it does not have any particularly nice expression.
For example, if $\Omega=\{0,1\}^{n}$ with $p_{\underline{\omega}}=p^{\sum_{i}\omega_{i} }q^{n-\sum_{i}\omega_{i} }$, and $X(\underline{\omega})=\omega_{1}+\ldots +\omega_{n}$, then $X\sim \mbox{Bin}(n,p)$. In words, the number of heads in $n$ tosses of a $p$-coin has $\mbox{Bin}(n,p)$ distribution.
Example 78
Fix $p\in (0,1]$ and let $f(k)=q^{k-1}p$ for $k\in \mathbb{N}_{+}$. This is called the Geometric distribution with parameter $p$ and is denoted Geo($p$). The CDF is
$$
F(t) = \begin{cases}
0 & \mbox{ if } t < 1, \\
1-q^{k} & \mbox{ if } k\le t < k+1, \mbox{ for some }k\ge 1.
\end{cases}
$$
For example, the number of tosses of a $p$-coin till the first head turns up, is a random variable with $\mbox{Geo}(p)$ distribution.
Example 79
Fix $\lambda > 0$ and define the pmf $f(k)=e^{-\lambda}\frac{\lambda^{k} }{k!}$. This is called the
Poisson distribution with parameter $\lambda$ and is denoted Pois($\lambda$).
In the problem of a psychic (randomly) guessing the cards in a deck, we have seen that the number of matches (correct guesses) had an approximately Pois($1$) distribution.
Example 80
Fix positive integers $b,w$ and $m\le b+w$. Define the pmf $f(k)=\frac{\binom{b}{k}\binom{w}{m-k} }{\binom{b+w}{m} }$ where the binomial coefficient $\binom{x}{y}$ is interpreted to be zero if $y > x$ (thus $f(k) > 0$ only for $\max\{m-w,0\}\le k\le b$). This is called the
Hypergeometric distribution with parameters $b,w,m$ and we shall denote it by Hypergeo($b,w,m$).
Consider a population with $b$ men and $w$ women. The number of men in a random sample (without replacement) of size $m$, is a random variable with the Hypergeo($b,w,m$) distribution.
Computing expectations from the pmf Let $X$ be a random variable on $(\Omega,p)$ with pmf $f$. Then we claim that
$$
\mathbf{E}[X] = \sum_{t\in \mathbb{R}}tf(t).
$$
Indeed, let $\mbox{Range}(X)=\{x_{1},x_{2},\ldots\}$. Let $A_{k}=\{\omega{\; : \;} X(\omega)=x_{k}\}$. By definition of pmf we have $\mathbf{P}(A_{k})=f(x_{k})$. Further, $A_{k}$ are pairwise disjoint and exhaustive. Hence
$$
\mathbf{E}[X] = \sum_{\omega\in \Omega}X(\omega)p_{\omega} = \sum_{k}\sum_{\omega\in A_{k} }X(\omega)p_{\omega} = \sum_{k}x_{k}\mathbf{P}(A_{k})=\sum_{k}x_{k}f(x_{k}).
$$
Similarly, $\mathbf{E}[X^{2}]=\sum_{k}x_{k}^{2}f(x_{k})$. More generally, if $h:\mathbb{R}\rightarrow \mathbb{R}$ is any function, then the random variable $h(X)$ has expectation $\mathbf{E}[h(X)]=\sum_{k}h(x_{k})f(x_{k})$. Although this sounds trivial, there is a very useful point here. To calculate $\mathbf{E}[X^{2}]$ we do not have to compute the pmf of $X^{2}$ first, which can be done but would be more complicated. Instead, in the above formulas, $\mathbf{E}[h(X)]$ has been computed directly in terms of the pmf of $X$.
Exercise 81
Find $\mathbf{E}[X]$ and $\mathbf{E}[X^{2}]$ in each case.
- $X\sim \mbox{Bin}(n,p)$.
- $X\sim \mbox{Geo}(p)$.
- $X\sim \mbox{Pois}(\lambda)$.
- $X\sim \mbox{Hypergeo}(b,w,m)$.