Negative exponential distribution

A continuous random variable $X$ is said to follow the negative exponential distribution (also called the exponential distribution) with parameter $\lambda > 0$ if its probability density function is

$$f(x)= \begin{cases} \lambda e^{-\lambda x}, & x \ge 0,\\[4pt] 0, & x<0. \end{cases}$$

It is the distribution of the waiting time between two successive events in a Poisson process. Its cumulative distribution function is

$$F(x)=P(X\le x)=1-e^{-\lambda x},\qquad x\ge 0.$$

Mean

$$E(X)=\int_0^\infty x\,f(x)\,dx=\int_0^\infty x\,\lambda e^{-\lambda x}\,dx.$$

Put $t=\lambda x$, so that $x=\dfrac{t}{\lambda}$ and $dx=\dfrac{dt}{\lambda}$:

$$E(X)=\int_0^\infty \frac{t}{\lambda}\,\lambda e^{-t}\,\frac{dt}{\lambda} =\frac{1}{\lambda}\int_0^\infty t\,e^{-t}\,dt =\frac{\Gamma(2)}{\lambda} =\frac{1}{\lambda}.$$

Hence

$$\boxed{E(X)=\frac{1}{\lambda}}.$$

Variance

First find the second moment:

$$E(X^2)=\int_0^\infty x^2\lambda e^{-\lambda x}\,dx.$$

Again putting $t=\lambda x$,

$$E(X^2)=\frac{1}{\lambda^2}\int_0^\infty t^2 e^{-t}\,dt =\frac{\Gamma(3)}{\lambda^2} =\frac{2}{\lambda^2}.$$

Therefore,

$$\operatorname{Var}(X)=E(X^2)-[E(X)]^2 =\frac{2}{\lambda^2}-\frac{1}{\lambda^2} =\frac{1}{\lambda^2}.$$ $$\boxed{\operatorname{Var}(X)=\frac{1}{\lambda^2}}$$

So the mean and variance of the negative exponential distribution are $\dfrac{1}{\lambda}$ and $\dfrac{1}{\lambda^2}$ respectively.

Negative exponential distribution — mean and variance

Probability density function. A continuous random variable $X$ is said to have the negative exponential distribution with parameter $\lambda > 0$ if its pdf is

$f(x) = \begin{cases} \lambda e^{-\lambda x}, & x \ge 0,\\[4pt] 0, & x < 0. \end{cases}$

Mean.

$E(X) = \int_0^{\infty} x\, f(x)\,dx = \int_0^{\infty} x\,\lambda e^{-\lambda x}\,dx$

Integrating by parts with $u = x,\; dv = \lambda e^{-\lambda x}dx$, so that $du = dx$ and $v = -e^{-\lambda x}$:

$E(X) = \Big[-x e^{-\lambda x}\Big]_0^{\infty} + \int_0^{\infty} e^{-\lambda x}\,dx$

The first term vanishes at both limits (at $\infty$ because $e^{-\lambda x} \to 0$ faster than $x \to \infty$), hence

$E(X) = \left[\frac{e^{-\lambda x}}{-\lambda}\right]_0^{\infty} = \frac{1}{\lambda}$

$\boxed{\text{Mean} = \frac{1}{\lambda}}$

Second moment.

$E(X^2) = \int_0^{\infty} x^2 \lambda e^{-\lambda x}\,dx$

Integrating by parts with $u = x^2,\; dv = \lambda e^{-\lambda x}dx$:

$E(X^2) = \Big[-x^2 e^{-\lambda x}\Big]_0^{\infty} + \int_0^{\infty} 2x\, e^{-\lambda x}\,dx = \frac{2}{\lambda}\int_0^{\infty} x\,\lambda e^{-\lambda x}\,dx = \frac{2}{\lambda}\cdot\frac{1}{\lambda} = \frac{2}{\lambda^2}$

Variance.

$\operatorname{Var}(X) = E(X^2) - [E(X)]^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}$

$\boxed{\operatorname{Var}(X) = \frac{1}{\lambda^2}}$

Result. For the negative exponential distribution with parameter $\lambda$,

$\text{Mean} = \frac{1}{\lambda}, \qquad \text{Variance} = \frac{1}{\lambda^2}, \qquad \text{S.D.} = \frac{1}{\lambda}.$

Note that the mean and the standard deviation are equal, each being $1/\lambda$.

For Beta distribution of the first kind with parameters (m) and (n), the probability density function is

$f(x)=\frac{1}{B(m,n)}x^{m-1}(1-x)^{n-1},\qquad 0<x<1.$

Given:

$m=3,\qquad n=4.$

Mean

$\mu=\frac{m}{m+n}=\frac{3}{3+4}=\frac{3}{7}.$

Mode

For (m>1,\ n>1),

$$\text{Mode}=\frac{m-1}{m+n-2} =\frac{3-1}{3+4-2} =\frac{2}{5}.$$

Variance

$\sigma^2=\frac{mn}{(m+n)^2(m+n+1)}.$

Substituting (m=3,\ n=4),

$$\sigma^2=\frac{3\times 4}{(3+4)^2(3+4+1)} =\frac{12}{7^2\times 8} =\frac{12}{392} =\frac{3}{98}.$$

Hence,

$\boxed{\text{Mean}=\frac{3}{7},\qquad \text{Mode}=\frac{2}{5},\qquad \text{Variance}=\frac{3}{98}}.$

Let (X\sim \text{Cauchy}(0,1)), so its pdf is

[ f_X(x)=\frac{1}{\pi(1+x^2)},\qquad -\infty<x<\infty. ]

Define

[ Y=X^2. ]

Since (Y\ge 0), for (y>0) the transformation (y=x^2) has two inverse branches:

[ x_1=\sqrt y,\qquad x_2=-\sqrt y. ]

The Jacobian is

[ \left|\frac{dx}{dy}\right|=\frac{1}{2\sqrt y}. ]

Hence the pdf of (Y=X^2) is

[ f_Y(y)=f_X(\sqrt y)\left|\frac{dx_1}{dy}\right| +f_X(-\sqrt y)\left|\frac{dx_2}{dy}\right|. ]

Substituting,

[ f_Y(y)=\frac{1}{\pi(1+y)}\cdot \frac{1}{2\sqrt y} +\frac{1}{\pi(1+y)}\cdot \frac{1}{2\sqrt y}. ]

Therefore,

[ \boxed{ f_Y(y)=\frac{1}{\pi\sqrt y(1+y)},\qquad y>0 } ]

and

[ f_Y(y)=0,\qquad y\le 0. ]

This is the probability density function of the F-distribution with (1) and (1) degrees of freedom, denoted

[ \boxed{Y=X^2\sim F(1,1)}. ]

Equivalently, it is also a Beta prime distribution with parameters (\frac12,\frac12).

The joint probability density function is

$f(x,y)=4xy,\qquad 0<x<1,\ 0<y<1.$

i) Joint distribution function (F(x,y))

By definition,

$F(x,y)=P(X\le x,\ Y\le y)=\int_{-\infty}^{x}\int_{-\infty}^{y} f(u,v)\,dv\,du.$

For (0<x<1,\ 0<y<1),

$$F(x,y)=\int_0^x\int_0^y 4uv\,dv\,du$$ $$=\int_0^x 4u\left[\frac{v^2}{2}\right]_0^y du =\int_0^x 2uy^2\,du$$ $$=2y^2\left[\frac{u^2}{2}\right]_0^x =x^2y^2.$$

Hence the joint distribution function is

$$F(x,y)= \begin{cases} 0, & x<0 \text{ or } y<0,\\[4pt] x^2y^2, & 0\le x<1,\ 0\le y<1,\\[4pt] x^2, & 0\le x<1,\ y\ge 1,\\[4pt] y^2, & x\ge 1,\ 0\le y<1,\\[4pt] 1, & x\ge 1,\ y\ge 1. \end{cases}$$

ii) (P\left(X<\frac12,\ Y<\frac34\right))

$$P\left(X<\frac12,\ Y<\frac34\right) =F\left(\frac12,\frac34\right)$$ $$=\left(\frac12\right)^2\left(\frac34\right)^2 =\frac14\cdot\frac{9}{16} =\frac{9}{64}.$$ $$\boxed{P\left(X<\frac12,\ Y<\frac34\right)=\frac{9}{64}}$$

Chi-square distribution tends to normal distribution as $n \to \infty$

Let $X \sim \chi^2_n$, the chi-square distribution with $n$ degrees of freedom. Its probability density function is

$f(x) = \frac{1}{2^{n/2}\,\Gamma\!\left(\frac{n}{2}\right)}\, x^{\frac{n}{2}-1} e^{-x/2}, \qquad x > 0.$

Mean and variance. For the chi-square distribution,

$E(X) = n, \qquad \operatorname{Var}(X) = 2n.$

Standardization. Define the standardized variable

$Z = \frac{X - E(X)}{\sqrt{\operatorname{Var}(X)}} = \frac{X - n}{\sqrt{2n}}.$

Then $E(Z) = 0$ and $\operatorname{Var}(Z) = 1$.

Moment generating function. The mgf of $X \sim \chi^2_n$ is

$M_X(t) = (1 - 2t)^{-n/2}, \qquad t < \tfrac{1}{2}.$

The mgf of the standardized variable $Z = \dfrac{X - n}{\sqrt{2n}}$ is

$$M_Z(t) = E\!\left(e^{tZ}\right) = e^{-\frac{nt}{\sqrt{2n}}}\, M_X\!\left(\frac{t}{\sqrt{2n}}\right) = e^{-\frac{nt}{\sqrt{2n}}}\, \left(1 - \frac{2t}{\sqrt{2n}}\right)^{-n/2}.$$

Let $u = \dfrac{t}{\sqrt{2n}}$, so that $u \to 0$ as $n \to \infty$. Then

$$M_Z(t) = e^{-nu}\, (1 - 2u)^{-n/2}.$$

Taking the limit. Write

$$\ln M_Z(t) = -nu - \frac{n}{2}\ln(1 - 2u).$$

Using the expansion $\ln(1 - 2u) = -2u - \dfrac{(2u)^2}{2} - \dfrac{(2u)^3}{3} - \cdots$ for small $u$,

$$\ln M_Z(t) = -nu - \frac{n}{2}\left(-2u - 2u^2 - \frac{8u^3}{3} - \cdots\right) = -nu + nu + n u^2 + \frac{4n u^3}{3} + \cdots$$ $$= n u^2 + \frac{4n u^3}{3} + \cdots$$

Now substitute $u = \dfrac{t}{\sqrt{2n}}$:

$$\ln M_Z(t) = n\cdot\frac{t^2}{2n} + \frac{4n}{3}\cdot\frac{t^3}{(2n)^{3/2}} + \cdots = \frac{t^2}{2} + O\!\left(n^{-1/2}\right).$$

Hence, as $n \to \infty$,

$$\lim_{n \to \infty} \ln M_Z(t) = \frac{t^2}{2} \quad\Longrightarrow\quad \lim_{n \to \infty} M_Z(t) = e^{t^2/2}.$$

Conclusion. The limiting mgf $e^{t^2/2}$ is exactly the moment generating function of the standard normal distribution $N(0,1)$. By the uniqueness theorem of moment generating functions, the distribution of

$Z = \frac{X - n}{\sqrt{2n}}$

converges to the standard normal distribution as $n \to \infty$. Equivalently, for large $n$,

$\chi^2_n \;\dot\sim\; N(n,\, 2n).$

Hence the chi-square distribution with $n$ degrees of freedom tends to the normal distribution as $n \to \infty$. $\blacksquare$

Answer:

Let (X_1,X_2,\dots,X_n) be a random sample from (N(\mu,\sigma^2)). We examine the existence of the minimum variance bound (MVB) estimator of (\mu) using the Cramer–Rao lower bound.

The likelihood function is [ L(\mu,\sigma^2;x)=\left(2\pi\sigma^2\right)^{-n/2} \exp\left[-\frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2\right]. ] The log-likelihood is [ \ell=\ln L=-\frac n2\ln(2\pi)-\frac n2\ln\sigma^2 -\frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2. ]

Case 1: (\sigma^2) known

The score function for (\mu) is [ \frac{\partial \ell}{\partial \mu} =\frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu) =\frac{n(\bar x-\mu)}{\sigma^2}. ] The Fisher information about (\mu) is [ I(\mu)=-E\left(\frac{\partial^2 \ell}{\partial \mu^2}\right) =\frac{n}{\sigma^2}. ] Hence the Cramer–Rao lower bound is [ \operatorname{Var}(T)\ge \frac{1}{I(\mu)}=\frac{\sigma^2}{n} ] for any unbiased estimator (T) of (\mu).

Now take [ T=\bar X=\frac1n\sum_{i=1}^n X_i. ] Then [ E(\bar X)=\mu,\qquad \operatorname{Var}(\bar X)=\frac{\sigma^2}{n}. ] Thus (\bar X) attains the Cramer–Rao lower bound. Therefore, the MVB estimator of (\mu) exists and is [ \boxed{\hat\mu_{\text{MVB}}=\bar X=\frac1n\sum_{i=1}^n X_i.} ]

Case 2: (\sigma^2) unknown

Treat (\theta=(\mu,\sigma^2)) as a two-parameter vector. The Fisher information matrix is [ I(\theta)= \begin{pmatrix} \dfrac{n}{\sigma^2} & 0\[6pt] 0 & \dfrac{n}{2\sigma^4} \end{pmatrix}. ] Its inverse is [ I^{-1}(\theta)= \frac1n \begin{pmatrix} \sigma^2 & 0\[4pt] 0 & 2\sigma^4 \end{pmatrix}. ] Therefore the Cramer–Rao lower bound for (\mu) is [ [I^{-1}(\theta)]_{11}=\frac{\sigma^2}{n}. ] Again (\bar X) is unbiased for (\mu) and has variance (\sigma^2/n), so it attains the bound.

Hence, even when (\sigma^2) is unknown, the MVB estimator of (\mu) exists and is the sample mean [ \boxed{\bar X=\frac1n\sum_{i=1}^n X_i.} ]

Conclusion: The minimum variance bound estimator of (\mu) exists for the normal population (N(\mu,\sigma^2)), and it is the sample mean (\bar X). Its variance is (\sigma^2/n), which equals the Cramer–Rao lower bound.

Question. Let $X_1, X_2, \ldots, X_n$ be iid Bernoulli $B(1,p)$ random variables. Show that $t_n(x) = \sum_{i=1}^n X_i$ is a sufficient statistic for $p$.

Answer.

Joint pmf of the sample. Since each $X_i$ takes the value $0$ or $1$ with $P(X_i = x_i) = p^{x_i}(1-p)^{1-x_i}, \qquad x_i \in \{0,1\},$ and the variables are independent, the joint pmf of $X = (X_1, \ldots, X_n)$ is $f(x; p) = \prod_{i=1}^{n} p^{x_i}(1-p)^{1-x_i} = p^{\sum_{i=1}^{n} x_i}(1-p)^{\,n - \sum_{i=1}^{n} x_i}.$

Factorization. Write $t_n(x) = \sum_{i=1}^{n} x_i$. Then $f(x; p) = p^{\,t_n(x)}(1-p)^{\,n - t_n(x)}.$

This is already in the required product form $f(x; p) = g\big(t_n(x),\, p\big)\, h(x),$ where we may take $g(t, p) = p^{t}(1-p)^{\,n-t}, \qquad h(x) = 1.$

Here $g$ depends on the sample $x$ only through the value of $t_n(x)$, and $h(x) = 1$ is free of the parameter $p$.

Conclusion. By the Neyman factorization criterion, a statistic $T$ is sufficient for $p$ if and only if the joint pmf factorises as $f(x; p) = g(t(x), p)\,h(x)$ with $h$ independent of $p$. Since the above factorisation holds with $t_n(x) = \sum_{i=1}^{n} X_i$, it follows that $t_n(x) = \sum_{i=1}^{n} X_i$ is a sufficient statistic for the parameter $p$. $\blacksquare$

Remark. The statistic $t_n$ counts the total number of successes in $n$ trials, and the conditional distribution of the sample given $t_n(x) = t$ is free of $p$, which confirms sufficiency directly.

Neyman–Pearson Lemma

Statement.
Let (X=(X_1,X_2,\dots,X_n)) be a random sample from a distribution with p.d.f. or p.m.f. (f(x;\theta)). Consider the simple null hypothesis against a simple alternative:

[ H_0:\theta=\theta_0 \quad \text{against} \quad H_1:\theta=\theta_1. ]

Let

[ f_0(x)=f(x;\theta_0), \qquad f_1(x)=f(x;\theta_1) ]

be the joint densities under (H_0) and (H_1). For a given size (\alpha), (0<\alpha<1), the most powerful test is the likelihood ratio test:

[ \phi^*(x)= \begin{cases} 1, & \text{if } \dfrac{f_1(x)}{f_0(x)}>k,\[6pt] 0, & \text{if } \dfrac{f_1(x)}{f_0(x)}<k,\[6pt] \gamma, & \text{if } \dfrac{f_1(x)}{f_0(x)}=k, \end{cases} ]

where (k\ge 0) and (\gamma\in[0,1]) are chosen so that

[ E_0[\phi^*(X)]=\alpha. ]

That is, reject (H_0) when the likelihood ratio is large, accept (H_0) when it is small, and randomize when it equals (k).

---

Proof.
Let (\phi) be any other test with size not exceeding (\alpha), i.e.

[ E_0[\phi(X)]\le \alpha. ]

Consider the product

[ (\phi^*(x)-\phi(x))(f_1(x)-kf_0(x)). ]

If (f_1(x)>kf_0(x)), then (\phi^(x)=1), so (\phi^(x)-\phi(x)\ge 0), and also (f_1(x)-kf_0(x)>0).
If (f_1(x)<kf_0(x)), then (\phi^(x)=0), so (\phi^(x)-\phi(x)\le 0), and also (f_1(x)-kf_0(x)<0).
If (f_1(x)=kf_0(x)), the product is zero.
Hence, for every (x),

[ (\phi^*(x)-\phi(x))(f_1(x)-kf_0(x))\ge 0. ]

Therefore,

[ \int (\phi^*-\phi)(f_1-kf_0),dx\ge 0. ]

Expanding,

[ \int (\phi^*-\phi)f_1,dx

k\int (\phi^*-\phi)f_0,dx \ge 0. ]

That is,

[ E_1[\phi^]-E_1[\phi] \ge k\left(E_0[\phi^]-E_0[\phi]\right). ]

But (E_0[\phi^*]=\alpha) and (E_0[\phi]\le \alpha). Hence

[ E_0[\phi^*]-E_0[\phi]\ge 0. ]

Since (k\ge 0), we get

[ E_1[\phi^*]-E_1[\phi]\ge 0, ]

so

[ E_1[\phi^*]\ge E_1[\phi]. ]

Thus (\phi^*) has the greatest power among all tests of size at most (\alpha). Therefore, the likelihood ratio test is the most powerful test for (H_0) against (H_1). For discrete distributions, the integrals are replaced by sums. Hence the Neyman–Pearson Lemma is proved.

$H_0$: children of the different heights are equally enrolled (uniform over the 8 observed heights). $H_1$: they are not equally enrolled. Level $\alpha=0.05$.

Frequencies of the 20 heights, with the observed cumulative proportion $F_o$ and the expected (uniform, $1/8$ each) cumulative proportion $F_e$:

$D_{cal}=\max\lvert F_o-F_e\rvert=0.175$

Tabulated value for $n=20$ at 5%: $D_{0.05}=1.36/\sqrt{20}\approx0.294$.

Since $D_{cal}=0.175<0.294$, $H_0$ is not rejected. The data are consistent with children of different heights being equally enrolled.

Median Test for Two Samples (Small Samples)

The median test is a non-parametric test used to determine whether two independent samples come from populations having the same median. It is based on the chi-square test and is particularly useful for small samples where the normality assumption of the t-test cannot be met.

Procedure:

Step 1: State the hypotheses.

• Null hypothesis $H_0$: The two populations have the same median, i.e. $M_1 = M_2$.
• Alternative hypothesis $H_1$: The two populations have different medians, i.e. $M_1 \neq M_2$ (or one-sided, as required).

Step 2: Combine and arrange the data. Pool the observations of both samples together and arrange them in ascending order of magnitude. Let the total number of observations be $N = n_1 + n_2$.

Step 3: Find the grand median. Determine the median of the combined series. For $N$ observations, the median is the $\left(\dfrac{N+1}{2}\right)^{th}$ value when $N$ is odd, and the average of the $\left(\dfrac{N}{2}\right)^{th}$ and $\left(\dfrac{N}{2}+1\right)^{th}$ values when $N$ is even. This value is the grand median $\tilde{M}$.

Step 4: Classify the observations. Compare each observation of both samples with the grand median and classify them as:

• Above the median (values greater than $\tilde{M}$)
• Below or equal to the median (values not greater than $\tilde{M}$)

Step 5: Prepare the $2 \times 2$ contingency table.

Step 6: Compute the chi-square statistic. Since the sample is small, apply Yates' correction for continuity:

$$\chi^2 = \frac{N\left(|ad - bc| - \dfrac{N}{2}\right)^2}{(a+b)(c+d)(a+c)(b+d)}$$

Step 7: Determine the critical value and decide. The test statistic follows a chi-square distribution with $df = (2-1)(2-1) = 1$. Obtain the critical value $\chi^2_{0.05,\,1} = 3.84$ at the 5% level of significance (or $\chi^2_{0.01,\,1} = 6.63$ at 1%).

• If $\chi^2_{cal} > \chi^2_{tab}$, reject $H_0$; the medians differ significantly.
• If $\chi^2_{cal} \le \chi^2_{tab}$, do not reject $H_0$; the medians do not differ significantly.

Note: For very small samples where any expected frequency falls below 5, the chi-square approximation is unreliable; in such cases the Fisher's exact test should be used instead.

Laplace distribution (also called the double exponential distribution) is a continuous probability distribution whose probability density function is

$f(x)=\frac{1}{2b}\,e^{-\frac{|x-\mu|}{b}},\qquad -\infty<x<\infty,\ b>0$

where $\mu$ is the location parameter (mean, median and mode all equal $\mu$) and $b$ is the scale parameter. Its shape is a sharp peak at $x=\mu$ with tails decaying exponentially on both sides — more peaked at the centre and heavier in the tails than the normal distribution. Its mean is $\mu$ and variance is $2b^2$. It is used in modelling data with outliers and in robust statistics (e.g. as the prior in LASSO regression).

Probability mass function of a bivariate random variable

Let $(X, Y)$ be a bivariate discrete random variable taking the possible values $(x_i, y_j)$, $i = 1, 2, \dots, j = 1, 2, \dots$. The function

$p(x_i, y_j) = P(X = x_i,\; Y = y_j)$

is called the joint probability mass function (joint pmf) of $(X, Y)$ if it satisfies:

1. $p(x_i, y_j) \ge 0$ for all $i, j$,
2. $\displaystyle\sum_i \sum_j p(x_i, y_j) = 1$.

The individual (marginal) probability mass functions are then

$p_X(x_i) = \sum_j p(x_i, y_j), \qquad p_Y(y_j) = \sum_i p(x_i, y_j).$

For the beta distribution of the second kind, the probability density function is $f(x) = \frac{1}{B(m,n)} \cdot \frac{x^{m-1}}{(1+x)^{m+n}}, \quad x > 0$

Mean: $E(X) = \frac{1}{B(m,n)}\int_0^\infty \frac{x^m}{(1+x)^{m+n}}dx = \frac{B(m+1, n-1)}{B(m,n)}$

Using $B(m+1,n-1) = \frac{\Gamma(m+1)\Gamma(n-1)}{\Gamma(m+n)}$ and $B(m,n) = \frac{\Gamma(m)\Gamma(n)}{\Gamma(m+n)}$:

$E(X) = \frac{\Gamma(m+1)\Gamma(n-1)}{\Gamma(m)\Gamma(n)} = \frac{m}{n-1}, \quad n>1$

Second moment: $E(X^2) = \frac{B(m+2, n-2)}{B(m,n)} = \frac{\Gamma(m+2)\Gamma(n-2)}{\Gamma(m)\Gamma(n)} = \frac{m(m+1)}{(n-1)(n-2)}, \quad n>2$

Variance: $\text{Var}(X) = E(X^2) - [E(X)]^2 = \frac{m(m+1)}{(n-1)(n-2)} - \frac{m^2}{(n-1)^2}$

$= \frac{m(m+1)(n-1) - m^2(n-2)}{(n-1)^2(n-2)} = \frac{m[(m+1)(n-1) - m(n-2)]}{(n-1)^2(n-2)}$

$= \frac{m[mn - m + n - 1 - mn + 2m]}{(n-1)^2(n-2)} = \frac{m(m+n-1)}{(n-1)^2(n-2)}, \quad n>2$

Result:

$E(X)=\frac{m}{n-1}\ (n>1),\qquad \operatorname{Var}(X)=\frac{m(m+n-1)}{(n-1)^2(n-2)}\ (n>2)$

Negative hypergeometric distribution is the distribution of the number of failures obtained before the $r$-th success when items are drawn without replacement from a finite population. It is the without-replacement counterpart of the negative binomial distribution, just as the hypergeometric distribution is the counterpart of the binomial.

Let a population of size $N$ contain $K$ successes and $N-K$ failures, and let items be drawn one at a time without replacement until $r$ successes ($r\le K$) appear. If $X$ is the number of failures before the $r$-th success, then

$P(X=x)=\frac{\binom{x+r-1}{r-1}\binom{N-x-r}{K-r}}{\binom{N}{K}},\qquad x=0,1,\dots,N-K$

Its mean is

$E(X)=\frac{r(N-K)}{K+1}$

Standard error of sample proportion

If a random sample of size (n) is drawn from a population in which the proportion of successes is (p), and (q=1-p), then the standard error of the sample proportion (\hat p) is

[ SE(\hat p)=\sqrt{\frac{pq}{n}}=\sqrt{\frac{p(1-p)}{n}} ]

When the population proportion (p) is unknown, it is estimated by the sample proportion (\hat p), so

[ SE(\hat p)=\sqrt{\frac{\hat p(1-\hat p)}{n}} ]

For sampling without replacement from a finite population of size (N), the finite population correction is used:

[ SE(\hat p)=\sqrt{\frac{p(1-p)}{n}}\sqrt{\frac{N-n}{N-1}} ]

Thus, the standard error of the sample proportion measures the standard deviation of the sampling distribution of (\hat p).

Uses of F-distribution:

1. Testing equality of two population variances — the F-test uses the ratio $F = s_1^2/s_2^2$ to decide whether two normal populations have equal variances.

2. ANOVA (Analysis of Variance) — it tests the equality of three or more population means by comparing between-group variance with within-group variance.

3. Testing the overall significance of a regression model — in regression analysis, the F-statistic tests whether the regression as a whole is significant (i.e., whether the independent variables jointly explain the dependent variable).

4. Comparing nested models — it is used to test whether adding more parameters (variables) significantly improves the fit of a model.

Cramer-Rao inequality: Let $X_1,\dots,X_n$ be a random sample from $f(x;\theta)$ and let $T$ be an unbiased estimator of $\theta$. Under the regularity conditions, the variance of $T$ is bounded below:

$\operatorname{Var}(T)\ \ge\ \frac{1}{nI(\theta)},\qquad I(\theta)=E\left[\left(\frac{\partial\ln f(X;\theta)}{\partial\theta}\right)^2\right]=-E\left[\frac{\partial^2\ln f(X;\theta)}{\partial\theta^2}\right]$

where $I(\theta)$ is the Fisher information in a single observation. More generally, for an estimator $T$ with $E(T)=\tau(\theta)$, $\operatorname{Var}(T)\ge \dfrac{[\tau'(\theta)]^2}{nI(\theta)}$.

Regularity conditions: the range of $X$ does not depend on $\theta$; $f$ is differentiable with respect to $\theta$; differentiation and integration can be interchanged; and $0<I(\theta)<\infty$.

An unbiased estimator whose variance equals $1/(nI(\theta))$ is a minimum variance (efficient) estimator.

Properties of a good estimator:

1. Unbiasedness — Its expected value equals the true population parameter, i.e. $E(\hat{\theta}) = \theta$.
2. Consistency — As the sample size $n$ increases, the estimate approaches the true parameter value.
3. Efficiency — Among all unbiased estimators, it has the smallest variance.
4. Sufficiency — It uses all the information in the sample relevant to the parameter.

The most powerful test is the test that has the greatest power among all tests of the same size (same significance level α) for testing a given null hypothesis against a given alternative hypothesis.

In other words, for a fixed level of significance α, a test is said to be most powerful if its power function β(θ) is at least as large as that of any other test of the same size, for every value of the parameter under the alternative hypothesis.

By the Neyman–Pearson Lemma, for a simple null hypothesis H₀ against a simple alternative H₁, the most powerful test at level α is the likelihood ratio test, whose critical region is of the form

$\frac{L(\theta_1)}{L(\theta_0)} > k$

where k is chosen so that the size of the test equals α.

Advantages of non-parametric tests:

1. No assumption about the population distribution: They do not require the data to follow a normal distribution (or any specific distribution), so they can be used when the normality assumption of parametric tests fails.

2. Suitable for small samples: They remain valid for very small sample sizes where parametric tests are not reliable.

3. Applicable to ordinal and nominal data: They can handle data that are merely ranks, categories, or scores, not just interval/ratio measurements.

4. Robust to outliers and extreme values: Since they often use ranks or signs rather than raw magnitudes, extreme observations do not distort the result much.

5. Simple and easy to compute: The calculations are usually straightforward and require less computational effort.

6. Fewer and weaker assumptions: Only mild assumptions (such as independence of observations or a continuous symmetric distribution in some tests) are needed, making conclusions more general.

Setting up the MGF:

$M_X(t) = \int_{-\infty}^{\infty} e^{tx}\frac{1}{2}e^{-|x|}\,dx = \frac{1}{2}\int_{-\infty}^{0} e^{(t+1)x}\,dx + \frac{1}{2}\int_{0}^{\infty} e^{(t-1)x}\,dx$

First integral: For convergence need $t+1>0$, i.e., $t>-1$: $\frac{1}{2}\cdot\frac{1}{t+1} = \frac{1}{2(1+t)}$

Second integral: For convergence need $t-1<0$, i.e., $t<1$: $\frac{1}{2}\cdot\frac{1}{1-t} = \frac{1}{2(1-t)}$

So $M_X(t) = \frac{1}{2}\left(\frac{1}{1+t}+\frac{1}{1-t}\right) = \frac{1}{2}\cdot\frac{2}{1-t^2} = \frac{1}{1-t^2}$, valid for $|t|<1$.

Moments: Expanding $\frac{1}{1-t^2} = \sum_{k=0}^{\infty} t^{2k}$.

The $r$-th moment is $\mu_r' = M_X^{(r)}(0)$. Since only even powers appear, all odd moments vanish. For even $r=2k$, the coefficient of $t^{2k}$ is 1, so $\mu_{2k}' = (2k)!$.

So $\mu_r' = r!$ for even $r$, and $0$ for odd $r$.

Particular moments: Mean = 0, Variance = 2, $\mu_4' = 24$.

Let (X) denote the lifetime of a component. Since (X) follows a negative exponential distribution, take its p.d.f. as

[ f(x;\theta)=\frac{1}{\theta}e^{-x/\theta},\qquad x>0,\ \theta>0 ]

where (\theta) is the mean lifetime.

For a random sample (x_1,x_2,\dots,x_{50}), the likelihood function is

[ L(\theta)=\prod_{i=1}^{50}\frac{1}{\theta}e^{-x_i/\theta} =\theta^{-50}e^{-\frac{1}{\theta}\sum x_i}. ]

Taking logarithms,

[ \log L(\theta)=-50\log\theta-\frac{1}{\theta}\sum x_i. ]

For maximum likelihood,

[ \frac{d}{d\theta}\log L(\theta) =-\frac{50}{\theta}+\frac{\sum x_i}{\theta^2}=0. ]

Hence

[ 50\theta=\sum x_i ]

so

[ \hat{\theta}=\frac{\sum x_i}{50}=\bar{x}. ]

Given (\bar{x}=200) hours,

[ \boxed{\hat{\theta}=200\text{ hours}} ]

Thus the maximum likelihood estimate of the parameter is (200) hours.

Now,

[ P(X>250)=1-P(X\le 250) =\int_{250}^{\infty}\frac{1}{200}e^{-x/200},dx =e^{-250/200}. ]

[ P(X>250)=e^{-1.25}\approx 0.2865. ]

Therefore, the probability that a component lasts more than (250) hours is approximately

[ \boxed{0.2865 \text{ or } 28.65%} ]

If the exponential distribution is written using the rate parameter (\lambda), then

[ f(x)=\lambda e^{-\lambda x}, ]

and the MLE is

[ \hat{\lambda}=\frac{1}{\bar{x}}=\frac{1}{200}=0.005. ]

Then

[ P(X>250)=e^{-0.005\times 250}=e^{-1.25}\approx 0.2865. ]

So the required probability is about (28.65%).

Given:

• Area A: $n_1 = 300$, $x_1 = 180$, so $\hat{p}_1 = 180/300 = 0.60$
• Area B: $n_2 = 250$, $x_2 = 150$, so $\hat{p}_2 = 150/250 = 0.60$

Difference: $\hat{p}_1 - \hat{p}_2 = 0$

Standard error: $SE = \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}} = \sqrt{\frac{0.6 \times 0.4}{300} + \frac{0.6 \times 0.4}{250}}$ $= \sqrt{0.0008 + 0.00096} = \sqrt{0.00176} \approx 0.04195$

95% CI: $0 \pm 1.96(0.04195) = 0 \pm 0.08222 = (-0.0822,\ 0.0822)$

The interval is $(-0.082, 0.082)$, which contains 0, so no significant difference.

Let the finite population consist of (N) units with values
[ Y_1,Y_2,\dots,Y_N, ] and let the population mean and population variance be [ \bar Y=\frac1N\sum_{i=1}^N Y_i,\qquad S^2=\frac{1}{N-1}\sum_{i=1}^N (Y_i-\bar Y)^2. ] A simple random sample without replacement (SRSWOR) of size (n) is drawn, giving sample values (y_1,\dots,y_n). The sample variance with divisor ((n-1)) is [ s^2=\frac{1}{n-1}\sum_{i=1}^n (y_i-\bar y)^2, \qquad \bar y=\frac1n\sum_{i=1}^n y_i. ]

We prove that [ E(s^2)=S^2. ]

Proof

For any (n) numbers (y_1,\dots,y_n), we have the identity [ \sum_{i<j}(y_i-y_j)^2

n\sum_{i=1}^n (y_i-\bar y)^2. ] Hence [ s^2

\frac{1}{n-1}\sum_{i=1}^n (y_i-\bar y)^2

\frac{1}{n(n-1)}\sum_{i<j}(y_i-y_j)^2. ]

Now consider any two population units (i,j). Under SRSWOR, the probability that both units appear in the sample is [ P(\text{both }i,j\text{ in sample})

\frac{\binom{N-2}{n-2}}{\binom{N}{n}}

\frac{n(n-1)}{N(N-1)}. ] Therefore, [ E(s^2)

\frac{1}{n(n-1)} \sum_{i<j} P(\text{both }i,j\text{ in sample})(Y_i-Y_j)^2. ] Substituting the probability, [ E(s^2)

\frac{1}{n(n-1)} \cdot \frac{n(n-1)}{N(N-1)} \sum_{i<j}(Y_i-Y_j)^2

\frac{1}{N(N-1)} \sum_{i<j}(Y_i-Y_j)^2. ]

Next, for the whole population, [ \sum_{i<j}(Y_i-Y_j)^2

N\sum_{i=1}^N (Y_i-\bar Y)^2. ] Since [ S^2=\frac{1}{N-1}\sum_{i=1}^N (Y_i-\bar Y)^2, ] we get [ \sum_{i<j}(Y_i-Y_j)^2

N(N-1)S^2. ] Thus [ E(s^2)

\frac{1}{N(N-1)}\cdot N(N-1)S^2

S^2. ]

Hence [ \boxed{E(s^2)=S^2} ] so the sample variance with divisor ((n-1)) is an unbiased estimator of the population variance under SRSWOR.

Note: If population variance is defined as (\sigma^2=\frac1N\sum (Y_i-\bar Y)^2), then
[ E(s^2)=\frac{N}{N-1}\sigma^2, ] so (s^2) is unbiased for (S^2), the usual finite-population variance with divisor (N-1).

Each draw is made with replacement, so the 4 draws are independent. On each draw (12 balls in all):

$P(\text{blue}) = \frac{5}{12},\quad P(\text{green}) = \frac{4}{12} = \frac13,\quad P(\text{yellow}) = \frac{3}{12} = \frac14.$

Joint distribution

$(X, Y, 4-X-Y)$ follows a trinomial (multinomial) distribution:

$P(X=x,\,Y=y) = \frac{4!}{x!\,y!\,(4-x-y)!}\left(\frac{5}{12}\right)^{x}\left(\frac{4}{12}\right)^{y}\left(\frac{3}{12}\right)^{4-x-y}, \qquad x,y \ge 0,\ x+y \le 4.$

Since $12^4 = 20736$, this equals $\dfrac{4!}{x!\,y!\,(4-x-y)!}\cdot\dfrac{5^x\,4^y\,3^{4-x-y}}{20736}$. Selected values:

All other cells follow from the same formula, and all 15 probabilities sum to 1.

Marginal distributions

Summing over $y$ (or noting that "blue" versus "not blue" is a Bernoulli trial on each draw):

$P(X = x) = \binom4x\left(\frac{5}{12}\right)^{x}\left(\frac{7}{12}\right)^{4-x}, \qquad x = 0,1,\dots,4,$

so $X \sim \text{Binomial}(4,\ 5/12)$, with $E(X) = 4\cdot\frac{5}{12} = \frac53$.

$P(Y = y) = \binom4y\left(\frac{1}{3}\right)^{y}\left(\frac{2}{3}\right)^{4-y}, \qquad y = 0,1,\dots,4,$

so $Y \sim \text{Binomial}(4,\ 1/3)$, with $E(Y) = \frac43$.

The population variance is unknown and the sample is small ($n = 16 < 30$) from a normal population, so we use the $t$-distribution.

Given: $n = 16$, $\bar x = 74$, $s = 8$, confidence level 95%, so $\alpha = 0.05$.

Degrees of freedom $= n - 1 = 15$, and $t_{0.025,\,15} = 2.131$.

Standard error:

$SE = \frac{s}{\sqrt n} = \frac{8}{\sqrt{16}} = 2$

Margin of error:

$E = t_{\alpha/2}\cdot SE = 2.131 \times 2 = 4.262$

95% confidence interval:

$\bar x \pm E = 74 \pm 4.262 = (69.74,\ 78.26)$

We are 95% confident that the population mean lies between 69.74 and 78.26.

We have a 2×4 contingency table:

H₀: Machines do not differ significantly in producing defectives. H₁: Machines differ significantly.

Expected frequencies: E = (row total × column total)/grand total

Defectives:

• I: 54×100/420 = 12.857
• II: 54×120/420 = 15.429
• III: 54×90/420 = 11.571
• IV: 54×110/420 = 14.143

Non-defectives:

• I: 366×100/420 = 87.143
• II: 366×120/420 = 104.571
• III: 366×90/420 = 78.429
• IV: 366×110/420 = 95.857

χ² = Σ(O−E)²/E

Defectives:

• I: (12−12.857)²/12.857 = 0.7347/12.857 = 0.0571
• II: (18−15.429)²/15.429 = 6.612/15.429 = 0.4286
• III: (9−11.571)²/11.571 = 6.612/11.571 = 0.5714
• IV: (15−14.143)²/14.143 = 0.7347/14.143 = 0.0519

Non-defectives:

• I: (88−87.143)²/87.143 = 0.7347/87.143 = 0.0084
• II: (102−104.571)²/104.571 = 6.612/104.571 = 0.0632
• III: (81−78.429)²/78.429 = 6.612/78.429 = 0.0843
• IV: (95−95.857)²/95.857 = 0.7347/95.857 = 0.0077

χ² = 0.0571 + 0.4286 + 0.5714 + 0.0519 + 0.0084 + 0.0632 + 0.0843 + 0.0077 = 1.2726

df = (2−1)(4−1) = 3 χ²₀.₀₅,₃ = 7.815

Since 1.2726 < 7.815, fail to reject H₀. No significant difference among machines.

Set-up

A population has $N$ items, $K$ of which are successes and $N-K$ are failures. Items are drawn one at a time without replacement until the $r$-th success appears. Let $X$ be the number of failures drawn before the $r$-th success. Then $X$ has the negative hypergeometric distribution:

$P(X=x) = \frac{\dbinom{x+r-1}{r-1}\dbinom{N-x-r}{K-r}}{\dbinom{N}{K}}, \qquad x = 0,1,\dots,N-K.$

Let $m = N-K$ be the number of failures.

Mean

Label the failures $1,\dots,m$ and let $I_i = 1$ if failure $i$ is drawn before the $r$-th success, and $0$ otherwise. Then $X = \sum_{i=1}^{m} I_i$.

Consider only failure $i$ and the $K$ successes, $K+1$ items in a random order. Failure $i$ precedes the $r$-th success exactly when it is among the first $r$ of these $K+1$ items. By symmetry it is equally likely to be in any position, so

$P(I_i = 1) = \frac{r}{K+1}.$

$\boxed{E(X) = \frac{r\,(N-K)}{K+1}}$

Variance

$E(X^2) = \sum_i E(I_i) + \sum_{i\ne j} E(I_iI_j).$

For $i \neq j$, take the $K$ successes and the two failures $i, j$ ($K+2$ items in random order). Both failures precede the $r$-th success exactly when both lie among the first $r+1$ positions, so

$P(I_iI_j = 1) = \frac{\binom{r+1}{2}}{\binom{K+2}{2}} = \frac{r(r+1)}{(K+1)(K+2)}.$

Hence

$E(X^2) = \frac{mr}{K+1} + \frac{m(m-1)\,r(r+1)}{(K+1)(K+2)},$

$\operatorname{Var}(X) = \frac{mr}{K+1} + \frac{m(m-1)\,r(r+1)}{(K+1)(K+2)} - \frac{r^2m^2}{(K+1)^2}.$

Over the common denominator $(K+1)^2(K+2)$ the numerator is

$mr\big[(K+1)(K+2) + (m-1)(r+1)(K+1) - rm(K+2)\big].$

Expanding the bracket:

$(K+1)(K+2) + (K+1)(mr+m-r-1) - rm(K+2) = (K+1)(K+m+1-r) - mr.$

With $K + m = N$, this is $(K+1)(N+1-r) - r(N-K) = (N+1)(K+1-r)$.

Therefore

$\boxed{\operatorname{Var}(X) = \frac{r\,(N-K)\,(N+1)\,(K-r+1)}{(K+1)^2\,(K+2)}}$

For $r=1$: $E(X) = \dfrac{N-K}{K+1}$ and $\operatorname{Var}(X) = \dfrac{K(N-K)(N+1)}{(K+1)^2(K+2)}$.