Discrete counterparts to exponential and Erlang random variables

An exponential random variable is a continuous random variable with parameter $\lambda > 0$ that has the probability density function:

$f(x) = \begin{cases} \lambda e^{-\lambda x} & \mbox{ if }x \ge 0 \\ 0 & \mbox{ if }x < 0\end{cases}$

An Erlang random variable is a continuous random variable with parameters $k, \lambda, \lambda > 0$ that has the probability density function:

$f(x) = \begin{cases} \frac{\lambda e^{-\lambda x}(\lambda x)^{k-1}}{(k-1)!} & \mbox{ if }x \ge 0 \\ 0 & \mbox{ if }x < 0\end{cases}$

An exponential random variable is typically used to model the time it takes for an event to occur. And, an Erlang random variable is typically (??) used to model the time it takes for $k$ events to occur.

The modelling of time by these distributions is appropriate if it is correct to think of time as composed of tiny intervals where the intervals are independent of each other and where each interval can have at most one occurrence – that is, two possible outcomes in each interval: occurrence (success) or no occurrence (failure).

When we have independent time intervals (sub-experiments) being run one after another each with a certain probability of occurrence (success), the number of time intervals (sub-experiments) to be run until an occurrence happens is modeled by the (discrete) geometric random variable, and the number of time intervals (sub-experiments) to be run until $k$ occurrences happen is modeled by the negative binomial random variable.

Thus, the exponential and Erlang random variables seem to be modelling such scenarios in the continuous world that when “discretized” are the ones modeled exactly by geometric and negative binomial random variables. This seems to imply that geometric and negative binomial random variables are the discrete counterparts to exponential and Erlang random variables.

The relation between geometric and exponential random variables and the relation between negative binomial and Erlang random variables are similar to the relation between binomial and Poisson random variables. Although, in the case of binomial and Poisson random variables, both are discrete.

In this article, we derive the probability density functions of the exponential and Erlang random variables using the probability mass functions of the geometric and negative binomial random variables.

Firstly, we work on the relation between geometric and exponential random variables.

As stated earlier, a geometric random variable applies to the scenarios where there is a sequence of trials conducted with each trial having a success probability of $p$ . In these scenarios, the geometric random variable counts the number of trials needed until a success occurs. The probability mass function of this geometric random variable $G$ is given by:

$P\{G=n\} = (1-p)^{n-1}p$

Now, if we consider time to be divided into a sequence of small independent intervals of length $h$ and $p$ is the probability that occurrence happens in a time interval and if $E$ is the continuous random variable that gives the time $t$ it takes until an occurrence happens, then:

$\begin{aligned}hf_E(t) & \approx P\{t\le E \le t+h\} \mbox{, where }f_E\mbox{ is the pdf of } E\\ & = P\{nh \le E \le (n+1)h\} \\ & = P\{G = n\} \\ & = (1-p)^{n-1}p \\ & \approx (1-p)^np\mbox{, assuming }p\mbox{ is small and }n\mbox{ is big} \\ & = (1-\lambda h)^{t/h}\lambda h\mbox{, using }p = \lambda h\mbox{ for some constant }\lambda \\ & \approx e^{-\lambda t}\lambda h\mbox{, for small } h \end{aligned}$

Thus, $f_E(t) = \lambda e^{-\lambda t}$ , which is the probability density function of exponential random variable. Note that, in the above, $\lambda=\frac{p}{h}$ may be thought average number of occurrences per unit time, which is the same idea as when we relate a binomial random variable to a Poisson random variable. Thus, exponential random variables model exactly those scenarios in continuous world whose counterparts in the discrete world are modeled by geometric random variables.

Let us now work on the relation between the negative binomial and Erlang random variables. We proceed in a similar fashion as when working with geometric and exponential random variables.

Like the geometric random variable, a negative binomial random variable also applies to scenarios where there is a sequence of trials conducted with each trial having a success probability of $p$ . In these scenarios, the negative binomial random variable counts the number of trials needed until $k$ successes occurs. The probability mass function of this negative binomial random variable $B$ is given by:

$P\{B=n\} = {n-1 \choose k-1}p^{k}(1-p)^{n-k}$

If we consider time to be divided into a sequence of small independent intervals of length $h$ and $p$ is the probability that occurrence happens in a time interval and if $L$ is the continuous random variable that gives the time $t$ it takes until $k$ occurrences happens, then:

$\begin{aligned}hf_L(t) & \approx P\{t\le L \le t+h\} \mbox{, where }f_L\mbox{ is the pdf of } L\\ & = P\{nh \le L \le (n+1)h\} \\ & = P\{B = n+1\} \\ & = {n \choose k-1}p^k(1-p)^{n-k+1} \\ & = \frac{n!}{(k-1)!(n-k+1)!}p^k(1-p)^{n-k+1} \\ & = \frac{n(n-1)(n-2)\cdots(n-k+2)}{(k-1)!}p^k(1-p)^{n-k+1} \\ & \approx \frac{n^{k-1}}{(k-1)!}p^k(1-p)^n\mbox{, assuming }p\mbox{ is small and }n\mbox{ is big} \\ & = \frac{\left(\frac{t}{h}\right)^{k-1}}{(k-1)!}(\lambda h)^k(1-\lambda h)^{t/h}\mbox{, using }p = \lambda h\mbox{ for some constant }\lambda \\ & \approx \frac{\lambda e^{-\lambda t}(\lambda t)^{k-1}}{(k-1)!} h\mbox{, for small } h \end{aligned}$

Thus, $f_L(t) = \frac{\lambda e^{-\lambda t}(\lambda t)^{k-1}}{(k-1)!} h$ , which is the probability density function of Erlang random variable. Thus, Erlang random variables model exactly those scenarios in continuous world whose counterparts in the discrete world are modeled by negative binomial random variables.