2.4 Families of distributions
Above, we introduced the basic concepts of random variables and probability distributions. Now, let’s explore some common distributions.
2.4.1 Discrete probability distributions
Bernoulli distribution
Definition:
A Bernoulli trial is a random experiment with exactly two possible outcomes: “success” and “failure.” These outcomes are typically denoted as 1 for success and 0 for failure. The probability of success is often denoted by \(p\), and the probability of failure is then given by \(q = 1 - p\).
The Bernoulli distribution is a discrete probability distribution that describes the probability of getting exactly successes in a Bernoulli trials with the probability of success \(p\in(0,1)\).
Denote \(X \sim \text{Ber}(p)\) if a random variable \(X\) follows Bernoulli distribution. The probability mass function of the Bernoulli distribution is given by: \[ \begin{align*} p_X(x) &= \begin{cases} p, & x = 1 \\ 1-p, & x = 0 \\ 0, & \mbox{otherwise} \end{cases} \\ &= p^x(1-p)^{1-x}, \quad x=0,1 \end{align*} \] and the range \(R_X=\{0,1\}\).
Proposition:
\(\mathbb{E}(X)=p, \quad \text{Var}(X)=p(1-p)\)
Binomial distribution
The Binomial distribution describes the probability of getting exactly \(k\) successes in \(n\) Bernoulli trials with the probability of success \(p\in(0,1)\).
Denote \(X \sim \text{Bin}(n,p)\) if a random variable \(X\) follows Binomial distribution. The probability mass function is given by: \[ \begin{align*} p_X(x) = {n \choose x} p^x(1-p)^{n-x}, \quad x=0,1,\cdots,n. \end{align*} \] and the range \(R_X=\{0,1,\cdots,n\}\).
Proposition:
- \(\mathbb{E}(X)=np, \quad \text{Var}(X)=np(1-p)\)
- \(\text{Ber}(p) \sim \text{Bin}(1,p)\)
- (additive property) Let \(X \sim \text{Bin}(n_1,p), Y \sim Bin(n_2,p)\) and \(X \perp \!\!\! \perp Y\). Then \(X+Y \sim \text{Bin}(n_1+n_2,p)\)
Poisson distribution
Before we introduce the Poisson distribution, we need to know what is Poisson process.
Definition:
A Poisson process is a stochastic process that models a sequence of events occurring randomly over time or space. It is characterized by the following properties:
- The probability that exactly 1 event occurs in a given interval of length \(h\) is equal to \(\lambda h + o(h)\).
- The probability that 2 or more events occur in an interval of length \(h\) is equal to \(o(h)\).
- For any integers \(n\), \(j_1, j_2, \cdots, j_n\) and any set of \(n\) non-overlapping intervals, if we define \(E_i\) to be the event that exactly \(j_i\) of the events under consideration occur in the \(i\)-th of these intervals, then events \(E_1,E_2,\cdots,E_n\) are independent.
Little \(o\) notation: \(o(h)\) stands for any function \(f(h)\) for which \(\displaystyle \lim_{h \to 0} \frac{f(h)}{h} = 0\).
\(p_X(k) = \frac{\lambda^{k} e^{-\lambda}}{k!}, \quad k \in \mathbb{N}, \lambda \in (0,\infty)\)