Discrete Distributions

Binomial

The binomial distribution with parameters [math]n[/math] and [math]p[/math] is the discrete probability distribution of the number of successes in a sequence of [math]n[/math] independent yes/no experiments, each of which yields success with probability [math]p[/math]. A success/failure experiment is also called a Bernoulli experiment or Bernoulli trial; when [math]n = 1[/math], the binomial distribution is a Bernoulli distribution. The binomial distribution is the basis for the popular binomial test of statistical significance.

The binomial distribution is frequently used to model the number of successes in a sample of size [math]n[/math] drawn with replacement from a population of size [math]N[/math]. If the sampling is carried out without replacement, the draws are not independent and so the resulting distribution is a hypergeometric distribution, not a binomial one. However, for [math]N[/math] much larger than [math]n[/math], the binomial distribution is a good approximation, and widely used.

Specification

Probability mass function

In general, if the random variable [math]X[/math] follows the binomial distribution with parameters [math]n[/math] ∈ ℕ and [math]p[/math] ∈ [0,1], we write [math]X \sim B(n,p)[/math]. The probability of getting exactly [math]k[/math] successes in [math]n[/math] trials is given by the probability mass function:

[[math]] f(k;n,p) = \operatorname{P}(X = k) = \binom n k p^k(1-p)^{n-k}[[/math]]

for [math]k = 0, \ldots, n[/math] where

[[math]]\binom n k =\frac{n!}{k!(n-k)!}[[/math]]

is the binomial coefficient, hence the name of the distribution. The formula can be understood as follows: we want exactly [math]k[/math] successes ([math]p^k[/math]) and [math]n-k[/math] failures ([math](1-p)^{-(n-k)}[/math]). However, the [math]k[/math] successes can occur anywhere among the [math]n[/math] trials, and there are [math]{n\choose k}[/math] different ways of distributing [math]k[/math] successes in a sequence of [math]n[/math] trials.

In creating reference tables for binomial distribution probability, usually the table is filled in up to [math]n/2[/math] values. This is because for [math]k \gt n/2[/math], the probability can be calculated by its complement as

[[math]]f(k,n,p)=f(n-k,n,1-p). [[/math]]

The probability mass function satisfies the following recurrence relation, for every [math]n,p[/math]:

[[math]]\left\{\begin{array}{l} p (n-k) f(k,n,p) = (k+1) (1-p) f(k+1,n,p), \\[10pt] f(0,n,p)=(1-p)^n \end{array}\right\}[[/math]]

Looking at the expression [math]f(k,n,p)[/math] as a function of [math]k[/math], there is a [math]k[/math] value that maximizes it. This [math]k[/math] value can be found by calculating

[[math]] \frac{f(k+1,n,p)}{f(k,n,p)}=\frac{(n-k)p}{(k+1)(1-p)} [[/math]]

and comparing it to 1. There is always an integer [math]M[/math] that satisfies

[[math]](n+1)p-1 \leq M \lt (n+1)p.[[/math]]

[math]f(k,n,p)[/math] is monotone increasing for [math]k \lt M[/math] and monotone decreasing for [math]k \gt M[/math], with the exception of the case where [math](n+1)p[/math] is an integer. In this case, there are two values for which [math]f[/math] is maximal: [math](n+1)p[/math] and [math](n+1)p-1[/math]. [math]M[/math] is the most probable (most likely) outcome of the Bernoulli trials and is called the mode. Note that the probability of it occurring can be fairly small.

Cumulative distribution function

The cumulative distribution function can be expressed as:

[[math]]F(k;n,p) = \Pr(X \le k) = \sum_{i=0}^{\lfloor k \rfloor} {n\choose i}p^i(1-p)^{n-i}[[/math]]

where [math]\scriptstyle \lfloor k\rfloor\,[/math] is the "floor" under [math]k[/math], i.e. the greatest integer less than or equal to [math]k[/math].

Mean and Variance

If [math]X \sim B(n,p)[/math], that is, [math]X[/math] is a binomially distributed random variable, [math]n[/math] being the total number of experiments and [math]p[/math] the probability of each experiment yielding a successful result, then the expected value of [math]X[/math] is [math]np[/math] and the variance is [math]npq[/math]. This follows directly from the fact that [math]X[/math] is equal in distribution to the sum of [math]n[/math] independent Bernouilli random variables each having success probability [math]p[/math] (see below).

Mode

Usually the mode of a binomial [math]B(n,p)[/math] distribution is equal to [math]\lfloor (n+1)p\rfloor[/math], where [math]\lfloor\cdot\rfloor[/math] is the floor function. However, when[math](n+1)p[/math] is an integer and [math]p[/math] is neither 0 nor 1, then the distribution has two modes: [math](n+1)p[/math](n + 1)p and [math](n+1)p -1 [/math]. When [math]p[/math] is equal to 0 or 1, the mode will be 0 and [math]n[/math] correspondingly. These cases can be summarized as follows:

[[math]] \begin{cases} \lfloor (n+1)\,p\rfloor & \text{if }(n+1)p\text{ is 0 or a noninteger}, \\ (n+1)\,p\ \text{ and }\ (n+1)\,p - 1 &\text{if }(n+1)p\in\{1,\dots,n\}, \\ n & \text{if }(n+1)p = n + 1. \end{cases}[[/math]]

Median

In general, there is no single formula to find the median for a binomial distribution, and it may even be non-unique. However several special results have been established:

• If [math]np[/math] is an integer, then the mean, median, and mode coincide and equal [math]np[/math].
• Any median [math]m[/math] must lie within the interval ⌊[math]np[/math]⌋ ≤ [math]m[/math] ≤ ⌈[math]np[/math]⌉.
• A median [math]m[/math] cannot lie too far away from the mean: [math]m-np \leq \textrm{min}\{\ln(2),\textrm{max}\{p,1-p\}\}[/math].
• The median is unique and equal to [math]m=[/math]round([math]np[/math]) in cases when either [math]p\leq 1-\ln(2)[/math] or [math]p\geq \ln(2)[/math] or [math]|m-np| \leq \textrm{min}\{p, 1-p\}[/math] (except for the case when [math]p = 1/2 [/math] and [math]n[/math] is odd).
• When [math]p=1/2[/math] and [math]n[/math] is odd, any number [math]m[/math] in the interval [math][(n-1)/2,(n+1)/2][/math] is a median of the binomial distribution. If [math]p = 1/2[/math] and [math]n[/math] is even, then [math]m = n/2[/math] is the unique median.

Related distributions

Sums of binomials

If [math]X \sim B(n,p)[/math] and [math]Y \sim B(m, p) [/math] are independent binomial variables with the same probability [math]p[/math], then [math]X+Y [/math] is again a binomial variable: its distribution is [math]Z=X+Y \sim B(n+m, p)[/math].

Bernoulli distribution

The Bernoulli distribution is a special case of the binomial distribution, where [math] n = 1[/math]. Symbolically, [math]X \sim B(1,p)[/math] has the same meaning as [math]X \sim B(p) [/math]. Conversely, any binomial distribution, [math]B(n,p)[/math], is the distribution of the sum of [math]n[/math] Bernoulli trials, [math]B(p)[/math], each with the same probability [math]p[/math].

Normal approximation

If [math]n[/math] is large enough, then the skew of the distribution is not too great. In this case a reasonable approximation to [math]B(n,p)[/math] is given by the normal distribution [math] \mathcal{N}(np,\,np(1-p))[/math], and this basic approximation can be improved in a simple way by using a suitable continuity correction. The basic approximation generally improves as [math]n[/math] increases (at least 20) and is better when [math]p[/math] is not near to 0 or 1. Various heuristics may be used to decide whether [math]n[/math] is large enough, and [math]p[/math] is far enough from the extremes of zero or one:

• One rule is that both [math]x = np [/math] and [math]n-p[/math] must be greater than 5. However, the specific number varies from source to source, and depends on how good an approximation one wants; some sources give 10 which gives virtually the same results as the following rule for large [math]n[/math] until [math]n[/math] is very large.
• A second rule is that for [math]n\gt5[/math] the normal approximation is adequate if

[[math]]\left | \left (\frac{1}{\sqrt{n}} \right ) \left (\sqrt{\frac{1-p}{p}}-\sqrt{\frac{p}{1-p}} \right ) \right |\lt0.3[[/math]]

• Another commonly used rule holds that the normal approximation is appropriate only if everything within 3 standard deviations of its mean is within the range of possible values, that is if

[[math]]\mu \pm 3 \sigma = np \pm 3 \sqrt{np(1-p)} \in [0,n].[[/math]]

The following is an example of applying a continuity correction. Suppose one wishes to calculate [math]\operatorname{P}(X \leq 8) [/math] for a binomial random variable [math]X[/math]. If [math]Y[/math] has a distribution given by the normal approximation, then [math]\operatorname{P}(X \leq 8 )[/math] is approximated by [math]\operatorname{P}(Y \leq 8.5 ) [/math]. The addition of 0.5 is the continuity correction; the uncorrected normal approximation gives considerably less accurate results.

This approximation, known as de Moivre–Laplace theorem, is a huge time-saver when undertaking calculations by hand (exact calculations with large [math]n[/math] are very onerous); historically, it was the first use of the normal distribution, introduced in Abraham de Moivre's book The Doctrine of Chances in 1738. Nowadays, it can be seen as a consequence of the central limit theorem since [math]B(n,p)[/math] is a sum of [math]n[/math] independent, identically distributed Bernoulli variables with parameter [math]p[/math]. This fact is the basis of a hypothesis test, a "proportion z-test", for the value of [math]p[/math] using [math]x/n[/math], the sample proportion and estimator of [math]p[/math], in a common test statistic.

For example, suppose one randomly samples [math]n[/math] people out of a large population and ask them whether they agree with a certain statement. The proportion of people who agree will of course depend on the sample. If groups of [math]n[/math] people were sampled repeatedly and truly randomly, the proportions would follow an approximate normal distribution with mean equal to the true proportion [math]p[/math] of agreement in the population and with standard deviation [math]\sigma = \sqrt{\frac{p(1-p)}{n}}[/math]

Geometric

The geometric distribution is the probability distribution of the number of failures before the first success supported on the set { 0, 1, 2, 3, ... }, i.e, if [math]p[/math] denotes the probability of success on each trial then

[[math]]\operatorname{P}(Y=k) = (1 - p)^k\,p\,.[[/math]]

Moments

The expected value of the geometrically distributed random variable [math]Y[/math] and its variance is

[[math]] \operatorname{E}(Y) = \frac{p}{1-p}, \qquad\operatorname{Var}(Y) = \frac{p}{(1-p)^2}. [[/math]]

Related distributions

• The geometric distribution [math]Y[/math] is a special case of the negative binomial distribution, with [math]r = 1 [/math]. More generally, if [math]Y_1,\ldots,Y_r[/math] are independent geometrically distributed variables with parameter [math]p[/math], then the sum

[[math]]Z = \sum_{m=1}^r Y_m[[/math]]

follows a negative binomial distribution with parameters [math]r[/math] and [math]p[/math].

• If [math]Y_1,\ldots,Y_r[/math] are independent geometrically distributed variables (with possibly different success parameters [math]p_m[/math]), then their minimum

[[math]]W = \min_{m \in 1, \dots, r} Y_m\,[[/math]]

is also geometrically distributed, with parameter [math]p = 1-\prod_m(1-p_{m}).[/math]

• Suppose [math]0 \lt r \lt 1 [/math], and for [math] k = 1,2,3,\ldots [/math] the random variable [math]X_k[/math] has a Poisson distribution with expected value [math]r^k[/math]. Then

[[math]]\sum_{k=1}^\infty k\,X_k[[/math]]

has a geometric distribution taking values in the set {0, 1, 2, ...}, with expected value [math]r/(1-r)[/math].

• The exponential distribution is the continuous analogue of the geometric distribution. If [math]X[/math] is an exponentially distributed random variable with parameter [math]\lambda[/math], then

[[math]]Y = \lfloor X \rfloor,[[/math]]

where [math]\lfloor \quad \rfloor[/math] is the floor (or greatest integer) function, is a geometrically distributed random variable with parameter [math]p = 1- e^{-\lambda} [/math] (thus [math]\lambda = - \ln(1-p) [/math]) and taking values in the set {0, 1, 2, ...}.

Poisson Distribution

The Poisson distribution , named after French mathematician Siméon Denis Poisson, is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time and/or space if these events occur with a known average rate and independently of the time since the last event. The Poisson distribution can also be used for the number of events in other specified intervals such as distance, area or volume. Within the context of insurance, the Poisson distribution can be used to model the number (frequency) of claims during a given time period.

Definition

A discrete random variable [math]X[/math] is said to have a Poisson distribution with parameter [math]\lambda \gt 0[/math], if, for [math]k = 0, 1, \ldots [/math], the probability mass function of [math]X[/math] is given by

[[math]]\!p_k= \operatorname{P}(X = k)= \frac{\lambda^k e^{-\lambda}}{k!}.[[/math]]

The probability mass function satisfies the following recurrence relation:

[[math]]\left\{\begin{array}{l} (k+1) p_{k+1}-\lambda p_{k}=0, \\ p_{0}=e^{-\lambda} \end{array}\right\}. [[/math]]

Mean

• The expected value and variance of a Poisson-distributed random variable are both equal to [math]\lambda[/math].
• The coefficient of variation is [math]\textstyle \lambda^{-1/2}[/math], while the index of dispersion is 1.
• The mean absolute deviation about the mean is

[[math]]\operatorname{E}|X-\lambda|= 2\exp(-\lambda) \frac{\lambda^{\lfloor\lambda\rfloor + 1}}{ \lfloor\lambda\rfloor!} .[[/math]]

• The mode of a Poisson-distributed random variable with non-integer [math]\lambda[/math] is equal to [math]\scriptstyle\lfloor \lambda \rfloor[/math], which is the largest integer less than or equal to [math]\lambda[/math]. This is also written as floor([math]λ[/math]). When [math]λ[/math] is a positive integer, the modes are [math]\lambda[/math] and [math]\lambda-1[/math].

Median

Bounds for the median ([math]ν[/math]) of the distribution are known and are sharp:

[[math]] \lambda - \ln 2 \le \nu \lt \lambda + \frac{1}{3}. [[/math]]

Other properties

• If [math]X_i \sim \operatorname{Pois}(\lambda_i)[/math] are independent and [math]\lambda=\sum_{i=1}^n \lambda_i[/math], then [math]Y = \left( \sum_{i=1}^n X_i \right) \sim \operatorname{Pois}(\lambda)[/math]. A converse is Raikov's theorem, which says that if the sum of two independent random variables is Poisson-distributed, then so is each of those two independent random variables.
• The Poisson distributions are infinitely divisible probability distributions.

• Bounds for the tail probabilities of a Poisson random variable [math] X \sim \operatorname{Pois}(\lambda)[/math] can be derived using a Chernoff bound argument:

[[math]] \begin{cases} \operatorname{P}(X \geq x) \leq \frac{e^{-\lambda} (e \lambda)^x}{x^x} & \text{ for } x \gt \lambda \\ \operatorname{P}(X \leq x) \leq \frac{e^{-\lambda} (e \lambda)^x}{x^x} & \text{ for } x \lt \lambda \,\, . \end{cases} [[/math]]

Negative Binomial

The negative binomial distribution is a discrete probability distribution of the number of failures in a sequence of independent and identically distributed Bernoulli trials before a specified number of successes (denoted [math]r[/math]) occurs. More precisely, suppose there is a sequence of independent Bernoulli trials. Thus, each trial has two potential outcomes called “success” and “failure”. In each trial the probability of success is [math]p[/math] and of failure is [math]1-p[/math]. We are observing this sequence until a predefined number [math]r[/math] of successes has occurred.

Probability Mass Function

The probability mass function of the negative binomial distribution is

[[math]] f(k; r, q) \equiv \operatorname{P}(N = k) = \binom{k+r-1}{k} (1-p)^kp^r \quad\text{for }k = 0, 1, 2, \dotsc [[/math]]

The binomial coefficient can be written in the following manner, explaining the name “negative binomial”:

[[math]] \begin{align*} \frac{(k+r-1)\dotsm(r)}{k!} &= (-1)^k \frac{(-r)(-r-1)(-r-2)\dotsm(-r-k+1)}{k!} \\ \label{*} &= (-1)^k\binom{-r}{k}. \end{align*} [[/math]]

To understand the above definition of the probability mass function, note that the probability for every specific sequence of [math]k[/math] failures and [math]r[/math] successes is [math]p^r(1-p)^k[/math], because the outcomes of the [math]k[/math] trials are supposed to happen independently. Since the [math]r[/math]^th success comes last, it remains to choose the [math]k[/math] trials with failures out of the remaining [math]r-1[/math] trials. The above binomial coefficient gives precisely the number of all these sequences of length [math]k-1[/math].

Extension to real-valued r

It is possible to extend the definition of the negative binomial distribution to the case of a positive real parameter [math]r[/math]. Although it is impossible to visualize a non-integer number of “successes”, we can still formally define the distribution through its probability mass function.

In the spirit of being consistent with the parametrizations found in , we consider the alternative parametrization defined implicitly by setting [math]p = 1(1+\beta)[/math].

As before, we say that [math]N[/math] has a negative binomial (or Pólya) distribution if it has a probability mass function:

[[math]] f(k; r, \beta) \equiv \operatorname{P}(N = k) = \binom{k+r-1}{k} \frac{\beta^k}{(1 + \beta)^{r + k}} \quad\text{for }k = 0, 1, 2, \dotsc [[/math]]

Here [math]r[/math] is a real, positive number. The binomial coefficient is then defined by the multiplicative formula and can also be rewritten using the gamma function:

[[math]] \binom{k+r-1}{k} = \frac{(k+r-1)(k+r-2)\dotsm(r)}{k!} = \frac{\Gamma(k+r)}{k!\,\Gamma(r)}. [[/math]]

To show that the probability mass function adds up to one, we have, by the binomial series

[[math]] (1 + \beta)^{-r} = (1 - (1-p))^{-r} =\sum_{k=0}^\infty(-1)^k\binom{-r}{k}(1-p)^k = (1 + \beta)^r \,\sum_{k=0}^\infty \operatorname{P}(N = k). [[/math]]

Finally, the following recurrence relation holds:

[[math]]\begin{array}{l} (k+1) \operatorname{P} (k+1)- (1-p) \operatorname{P} (k) (k+r)=0, \\ \operatorname{P} (0) = p^r. \end{array} [[/math]]

Wikipedia References

• Wikipedia contributors. "Binomial distribution". Wikipedia. Wikipedia. Retrieved 28 January 2022.
• Wikipedia contributors. "Geometric distribution". Wikipedia. Wikipedia. Retrieved 28 January 2022.
• Wikipedia contributors. "Poisson distribution". Wikipedia. Wikipedia. Retrieved 28 January 2022.
• Wikipedia contributors. "Negative binomial distribution". Wikipedia. Wikipedia. Retrieved 17 February 2022.