04 Important Discrete Distributions

In the realm of probability, certain discrete probability distributions emerge repeatedly due to their fundamental nature and wide applicability. These distributions serve as building blocks for modeling a vast array of random phenomena. We now delve into some of the most important discrete distributions, exploring their properties, characteristics, and applications.

Bernoulli Distribution (2.5.1)

The Bernoulli distribution is the simplest non-trivial discrete distribution. It models a single trial with two possible outcomes, often labeled “success” and “failure”. Let’s denote these outcomes as 1 (success) and 0 (failure). A random variable $X$ following a Bernoulli distribution takes the value 1 with probability $p$ (the success probability) and the value 0 with probability $1-p$ (the failure probability).

Formally, a random variable $X$ with range $W_X=\{0,1\}$ and density function

is said to be Bernoulli-distributed. We denote this as $X\sim \text{Bernoulli}(p)$.

Properties of Bernoulli Distribution:

• Expectation: The expected value of a Bernoulli random variable is simply its success probability $p$:

  $$E[X]=p$$

• Variance: The variance of a Bernoulli random variable is given by:

  $$Var[X]=p(1-p)$$

The Bernoulli distribution is the foundation for many other discrete distributions, as it models the most basic random trial. It is used extensively in modeling binary events, such as coin flips, success/failure of experiments, or the state of a binary variable.

Binomial Distribution (2.5.2)

The binomial distribution arises when we repeat a Bernoulli trial multiple times and count the number of successes. Consider performing $n$ independent Bernoulli trials, each with success probability $p$. Let $X$ be the random variable representing the total number of successes in these $n$ trials. Then $X$ follows a binomial distribution.

Formally, a random variable $X$ with range $W_X=\{0,1,\dots ,n\}$ and density function

is said to be binomially distributed. We denote this as $X\sim \text{Bin}(n,p)$.

Derivation of Binomial Distribution:

The probability of getting exactly $x$ successes in $n$ trials involves two components:

1. Number of ways to choose x successes: There are $\left(\genfrac{}{}{0ex}{}{n}{x}\right)$ ways to choose which $x$ trials out of $n$ will be successes.

2. Probability of a specific sequence with x successes: The probability of any specific sequence with $x$ successes and $n-x$ failures is $p^x(1-p{)}^{n-x}$, due to the independence of trials.

Combining these components yields the binomial density function.

Properties of Binomial Distribution:

• Expectation: The expected number of successes in $n$ Bernoulli trials is:

  $$E[X]=np$$

• Variance: The variance of a binomial random variable is:

  $$Var[X]=np(1-p)$$

The binomial distribution is widely used in modeling counts of successes in a fixed number of independent trials, such as the number of heads in a fixed number of coin tosses, the number of defective items in a sample, or the number of clicks on an online advertisement in a given number of impressions.

Geometric Distribution (2.5.3)

The geometric distribution models the number of trials needed to achieve the first success in a sequence of independent Bernoulli trials. Consider repeatedly performing Bernoulli trials with success probability $p$ until the first success occurs. Let $X$ be the random variable representing the number of trials needed to get the first success. Then $X$ follows a geometric distribution.

Formally, a random variable $X$ with range $W_X=\mathbb{N}=\{1,2,3,\dots \}$ and density function

is said to be geometrically distributed. We denote this as $X\sim \text{Geo}(p)$.

Derivation of Geometric Distribution:

For $X=i$ (first success on the $i$-th trial), we must have $i-1$ failures followed by one success. The probability of this sequence is $(1-p{)}^{i-1}\cdot p$, due to independence.

Properties of Geometric Distribution:

• Expectation: The expected number of trials to get the first success is:

  $$E[X]=\frac{1}{p}$$

• Variance: The variance of a geometric random variable is:

  $$Var[X]=\frac{1-p}{p^2}$$

• Memorylessness: The geometric distribution possesses a unique property called memorylessness. It states that the probability of waiting for $t$ more trials to get the first success, given that we have already had $s$ failures, is the same as the probability of waiting $t$ trials from the beginning. Formally, for all $s,t\in \mathbb{N}$:

  $$Pr[X\ge s+t\mid X>s]=Pr[X\ge t]$$

The geometric distribution is used to model waiting times until the first occurrence of an event, such as the number of trials until the first success in a process, the waiting time for a customer to be served, or the number of attempts until a task is completed.

Poisson Distribution (2.5.4)

The Poisson distribution models the number of events occurring in a fixed interval of time or space, given that these events occur independently and at a constant average rate. It is particularly useful for rare events occurring in a large population or over a long period.

Formally, a random variable $X$ with range $W_X={\mathbb{N}}_0=\{0,1,2,\dots \}$ and density function

where $\lambda >0$ is a parameter representing the average rate of events, is said to be Poisson-distributed. We denote this as $X\sim \text{Po}(\lambda )$.

Properties of Poisson Distribution:

• Expectation: The expected number of events in the interval is:

  $$E[X]=\lambda$$

• Variance: The variance of a Poisson random variable is also equal to its parameter $\lambda$:

  $$Var[X]=\lambda$$

Poisson Approximation to Binomial: The Poisson distribution can be seen as a limiting case of the binomial distribution when the number of trials $n$ is large, the success probability $p$ is small, and the product $np=\lambda$ remains constant. In such cases, the binomial distribution $\text{Bin}(n,p)$ can be approximated by the Poisson distribution $\text{Po}(\lambda )$.

The Poisson distribution is widely used in modeling rare events, such as the number of phone calls arriving at a call center in an hour, the number of radioactive decays in a given time interval, the number of typos on a page, or the number of accidents at an intersection in a year.

These important discrete distributions—Bernoulli, binomial, geometric, and Poisson—provide a powerful toolkit for modeling and analyzing a wide range of random phenomena. Understanding their properties and applications is crucial for probabilistic reasoning and algorithm design.

Prev: 03 Random Variables, Expectation, Variance | Next: 05 Multiple Random Variables