Negative Exponential Distribution

Definition. A continuous random variable $X$ taking only non-negative values is said to follow the negative exponential distribution with parameter $\lambda > 0$ if its probability density function is

$$f(x) = \begin{cases} \lambda e^{-\lambda x}, & x \ge 0,\ \lambda > 0 \\[2pt] 0, & \text{otherwise} \end{cases}$$

It is the distribution of the waiting time until the first event in a Poisson process, and is the continuous analogue of the geometric distribution. Its cumulative distribution function is $F(x) = 1 - e^{-\lambda x}$, $x \ge 0$.

Mean

$$E(X) = \int_0^\infty x\,\lambda e^{-\lambda x}\,dx$$

Put $\lambda x = t \Rightarrow x = \dfrac{t}{\lambda},\ dx = \dfrac{dt}{\lambda}$:

$$E(X) = \int_0^\infty \frac{t}{\lambda}\,e^{-t}\,\frac{dt}{\lambda} = \frac{1}{\lambda}\int_0^\infty t\,e^{-t}\,dt = \frac{1}{\lambda}\,\Gamma(2) = \frac{1}{\lambda}$$ $$\boxed{E(X) = \frac{1}{\lambda}}$$

Variance

First find the second moment:

$$E(X^2) = \int_0^\infty x^2 \lambda e^{-\lambda x}\,dx = \frac{1}{\lambda^2}\int_0^\infty t^2 e^{-t}\,dt = \frac{1}{\lambda^2}\,\Gamma(3) = \frac{2}{\lambda^2}$$ $$V(X) = E(X^2) - [E(X)]^2 = \frac{2}{\lambda^2} - \left(\frac{1}{\lambda}\right)^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2}$$ $$\boxed{V(X) = \frac{1}{\lambda^2}}$$

Note. The standard deviation is also $1/\lambda$, so mean = standard deviation. The distribution has the memoryless property: $P(X > s+t \mid X > s) = P(X > t)$.

Negative exponential distribution

A continuous random variable $X$ is said to follow the negative exponential distribution with parameter $\lambda > 0$ if its probability density function is

$f(x) = \lambda e^{-\lambda x}, \qquad x \ge 0,\ \lambda > 0$

and $f(x) = 0$ otherwise. Here $\lambda$ is the rate parameter and the mean is $1/\lambda$.

Mean

$E(X) = \int_0^{\infty} x\, f(x)\, dx = \int_0^{\infty} x\, \lambda e^{-\lambda x}\, dx$

Integrating by parts, take $u = x$ and $dv = \lambda e^{-\lambda x}dx$, so that $du = dx$ and $v = -e^{-\lambda x}$:

$E(X) = \Big[-x e^{-\lambda x}\Big]_0^{\infty} + \int_0^{\infty} e^{-\lambda x}\, dx$

The first term vanishes at both limits (at $\infty$ because the exponential decays faster than $x$ grows), so

$E(X) = \left[\frac{e^{-\lambda x}}{-\lambda}\right]_0^{\infty} = 0 + \frac{1}{\lambda} = \frac{1}{\lambda}$

$\boxed{\text{Mean} = \frac{1}{\lambda}}$

Variance

First find $E(X^2)$:

$E(X^2) = \int_0^{\infty} x^2 \lambda e^{-\lambda x}\, dx$

Integrating by parts with $u = x^2$, $dv = \lambda e^{-\lambda x}dx$, so $du = 2x\,dx$ and $v = -e^{-\lambda x}$:

$E(X^2) = \Big[-x^2 e^{-\lambda x}\Big]_0^{\infty} + 2\int_0^{\infty} x e^{-\lambda x}\, dx = 0 + \frac{2}{\lambda}\int_0^{\infty} x \lambda e^{-\lambda x}\, dx = \frac{2}{\lambda}\,E(X)$

$E(X^2) = \frac{2}{\lambda} \cdot \frac{1}{\lambda} = \frac{2}{\lambda^2}$

Now

$\text{Var}(X) = E(X^2) - [E(X)]^2 = \frac{2}{\lambda^2} - \left(\frac{1}{\lambda}\right)^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}$

$\boxed{\text{Variance} = \frac{1}{\lambda^2}}$

Result: For the negative exponential distribution, mean $= \dfrac{1}{\lambda}$ and variance $= \dfrac{1}{\lambda^2}$; the standard deviation is also $\dfrac{1}{\lambda}$, so the mean and standard deviation are equal.

For Beta distribution of the first kind with parameters (m) and (n), the probability density function is

[ f(x)=\frac{x^{m-1}(1-x)^{n-1}}{B(m,n)}, \qquad 0<x<1 ]

where (B(m,n)) is the Beta function.

Given:

[ m=3,\qquad n=4 ]

1. Mean

[ \text{Mean}=\frac{m}{m+n} ]

[ =\frac{3}{3+4}=\frac{3}{7} ]

[ \boxed{\text{Mean}=\frac{3}{7}} ]

2. Mode

Since (m>1) and (n>1), the mode is

[ \text{Mode}=\frac{m-1}{m+n-2} ]

[ =\frac{3-1}{3+4-2} ]

[ =\frac{2}{5} ]

[ \boxed{\text{Mode}=\frac{2}{5}} ]

3. Variance

[ \text{Variance}=\frac{mn}{(m+n)^2(m+n+1)} ]

[ =\frac{3\times 4}{(3+4)^2(3+4+1)} ]

[ =\frac{12}{7^2\times 8} ]

[ =\frac{12}{49\times 8} ]

[ =\frac{12}{392} ]

[ =\frac{3}{98} ]

[ \boxed{\text{Variance}=\frac{3}{98}} ]

Hence,

[ \boxed{\text{Mean}=\frac{3}{7},\quad \text{Mode}=\frac{2}{5},\quad \text{Variance}=\frac{3}{98}} ]

Let (X\sim \text{Cauchy}(0,1)), so its pdf is

[ f_X(x)=\frac{1}{\pi(1+x^2)},\qquad -\infty<x<\infty. ]

Let

[ Y=X^2. ]

For (y>0),

[ F_Y(y)=P(Y\le y)=P(X^2\le y)=P(-\sqrt y\le X\le \sqrt y) ]

[ =F_X(\sqrt y)-F_X(-\sqrt y). ]

Differentiating with respect to (y),

[ f_Y(y)=f_X(\sqrt y)\frac{1}{2\sqrt y}+f_X(-\sqrt y)\frac{1}{2\sqrt y}. ]

Since the Cauchy pdf is even,

[ f_X(\sqrt y)=f_X(-\sqrt y)=\frac{1}{\pi(1+y)}. ]

Hence

[ f_Y(y)=\frac{1}{2\sqrt y}\cdot \frac{2}{\pi(1+y)} ]

[ \boxed{f_Y(y)=\frac{1}{\pi\sqrt y(1+y)}},\qquad y>0. ]

So the pdf of (X^2) is

[ \boxed{f_{X^2}(y)= \begin{cases} \dfrac{1}{\pi\sqrt y(1+y)}, & y>0,\[6pt] 0, & y\le 0. \end{cases}} ]

This is the Beta prime distribution with parameters (\alpha=\frac12,\ \beta=\frac12). Equivalently,

[ X^2\sim F(1,1), ]

the Fisher–Snedecor (F)-distribution with (1) and (1) degrees of freedom. Also, if

[ Z=\frac{X^2}{1+X^2}, ]

then

[ Z\sim \text{Beta}\left(\frac12,\frac12\right). ]

Thus the square of a standard Cauchy variable follows the (F(1,1)) distribution.

Given the joint density

$$f(x,y)= \begin{cases} 4xy, & 0<x<1,\ 0<y<1,\\ 0, & \text{elsewhere.} \end{cases}$$

i) Joint distribution function (F(x,y))

The joint cumulative distribution function is

$F(x,y)=P(X\le x,\ Y\le y)=\int_{-\infty}^{x}\int_{-\infty}^{y} f(u,v)\,dv\,du.$

For (0<x<1,\ 0<y<1),

$$F(x,y)=\int_0^x\int_0^y 4uv\,dv\,du$$ $$=\int_0^x 4u\left(\frac{y^2}{2}\right)du =\int_0^x 2uy^2\,du = x^2y^2.$$

Hence,

$$F(x,y)= \begin{cases} 0, & x\le 0 \text{ or } y\le 0,\\[4pt] x^2y^2, & 0<x<1,\ 0<y<1,\\[4pt] x^2, & 0<x<1,\ y\ge 1,\\[4pt] y^2, & x\ge 1,\ 0<y<1,\\[4pt] 1, & x\ge 1,\ y\ge 1. \end{cases}$$

ii) (P\left(X<\frac12,\ Y<\frac34\right))

Since (\frac12) and (\frac34) lie between (0) and (1),

$$P\left(X<\frac12,\ Y<\frac34\right) =F\left(\frac12,\frac34\right)$$ $$=\left(\frac12\right)^2\left(\frac34\right)^2 =\frac14\cdot\frac{9}{16} =\frac{9}{64}.$$

Therefore,

$$\boxed{P\left(X<\frac12,\ Y<\frac34\right)=\frac{9}{64}}$$

Chi-square distribution tends to normal distribution as $n \to \infty$

Let $X \sim \chi^2_{(n)}$, i.e. $X$ is a chi-square variate with $n$ degrees of freedom. Its probability density function is

$f(x) = \frac{1}{2^{n/2}\,\Gamma\!\left(\frac{n}{2}\right)}\, x^{\frac{n}{2}-1} e^{-x/2}, \qquad x > 0.$

Mean and variance. For the chi-square distribution,

$E(X) = n, \qquad \text{Var}(X) = 2n.$

Standardisation. Define the standardised variate

$Z = \frac{X - E(X)}{\sqrt{\text{Var}(X)}} = \frac{X - n}{\sqrt{2n}}.$

Moment generating function of $Z$. The mgf of $X$ is

$M_X(t) = (1 - 2t)^{-n/2}, \qquad t < \tfrac{1}{2}.$

Hence

$$M_Z(t) = E\!\left(e^{tZ}\right) = e^{-tn/\sqrt{2n}}\, M_X\!\left(\frac{t}{\sqrt{2n}}\right) = e^{-t\sqrt{n/2}}\left(1 - \frac{2t}{\sqrt{2n}}\right)^{-n/2}.$$

Taking logarithms.

$$\ln M_Z(t) = -\frac{t\sqrt{n}}{\sqrt{2}} - \frac{n}{2}\ln\!\left(1 - \frac{2t}{\sqrt{2n}}\right).$$

Put $u = \dfrac{2t}{\sqrt{2n}} = \dfrac{\sqrt{2}\,t}{\sqrt{n}}$, so that $u \to 0$ as $n \to \infty$. Using the expansion

$\ln(1-u) = -u - \frac{u^2}{2} - \frac{u^3}{3} - \frac{u^4}{4} - \cdots,$

we get

$$\ln M_Z(t) = -\frac{t\sqrt{n}}{\sqrt{2}} - \frac{n}{2}\left[-\frac{\sqrt{2}\,t}{\sqrt{n}} - \frac{1}{2}\cdot\frac{2t^2}{n} - \frac{1}{3}\cdot\frac{2\sqrt{2}\,t^3}{n^{3/2}} - \frac{1}{4}\cdot\frac{4t^4}{n^{2}} - \cdots\right].$$

The first two terms cancel:

$$-\frac{t\sqrt{n}}{\sqrt{2}} + \frac{t\sqrt{n}}{\sqrt{2}} = 0,$$

and the remaining terms give

$$\ln M_Z(t) = \frac{t^2}{2} + \frac{\sqrt{2}\,t^3}{3\sqrt{n}} + \frac{t^4}{2n} + \cdots$$

Limit as $n \to \infty$. Every term beyond the first contains a positive power of $1/\sqrt{n}$ or $1/n$, so all of them vanish:

$$\lim_{n \to \infty} \ln M_Z(t) = \frac{t^2}{2}.$$

Therefore

$$\lim_{n \to \infty} M_Z(t) = e^{t^2/2},$$

which is exactly the moment generating function of the standard normal variate $N(0,1)$.

Conclusion. Since the mgf of $Z = \dfrac{X-n}{\sqrt{2n}}$ converges to the mgf of $N(0,1)$, by the uniqueness theorem of moment generating functions

$$\frac{X - n}{\sqrt{2n}} \xrightarrow{\ d\ } N(0,1) \quad \text{as } n \to \infty,$$

i.e. the chi-square distribution with $n$ degrees of freedom tends to the normal distribution with mean $n$ and variance $2n$ as $n \to \infty$. Hence shown. $\blacksquare$

Let (X_1,X_2,\dots,X_n) be a random sample from (N(\mu,\sigma^2)).

The likelihood function is [ L(\mu)=\prod_{i=1}^n \frac{1}{\sigma\sqrt{2\pi}} \exp\left[-\frac{(x_i-\mu)^2}{2\sigma^2}\right]. ]

The log-likelihood is [ \ell(\mu)=-\frac n2\log(2\pi\sigma^2) -\frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2. ]

The score function for (\mu) is [ U(\mu)=\frac{\partial \ell}{\partial \mu} =\frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu) =\frac{n(\bar X-\mu)}{\sigma^2}. ]

Hence the Fisher information about (\mu) is [ I(\mu)=E[U(\mu)^2]=\frac{n}{\sigma^2}. ]

By the Cramer–Rao inequality, for any unbiased estimator (T) of (\mu), [ \operatorname{Var}(T)\ge \frac{1}{I(\mu)}=\frac{\sigma^2}{n}. ]

Now the sample mean [ \bar X=\frac1n\sum_{i=1}^n X_i ] is unbiased for (\mu), and [ \operatorname{Var}(\bar X)=\frac{\sigma^2}{n}. ]

Thus (\bar X) attains the Cramer–Rao lower bound. Therefore the minimum variance bound (MVB) estimator of (\mu) exists, and it is

[ \boxed{\hat\mu_{\text{MVB}}=\bar X=\frac1n\sum_{i=1}^n X_i} ]

with minimum variance

[ \boxed{\operatorname{Var}(\bar X)=\frac{\sigma^2}{n}}. ]

If (\sigma^2) is unknown, the joint information matrix is diagonal, and the (\mu\mu)-element remains (n/\sigma^2), so the same MVB estimator (\bar X) is obtained.

Answer

Let $X_1, X_2, \ldots, X_n$ be iid Bernoulli$(1, p)$ random variables. The probability mass function of each $X_i$ is

$P(X_i = x_i) = p^{x_i}(1-p)^{1-x_i}, \qquad x_i \in \{0, 1\}.$

Since the variables are independent, the joint pmf of the sample $\mathbf{X} = (X_1, \ldots, X_n)$ is

$P(\mathbf{X} = \mathbf{x}) = \prod_{i=1}^n p^{x_i}(1-p)^{1-x_i} = p^{\sum_{i=1}^n x_i}(1-p)^{n - \sum_{i=1}^n x_i}.$

Let $t_n(\mathbf{x}) = \sum_{i=1}^n x_i$. Then the joint pmf can be written as

$P(\mathbf{X} = \mathbf{x}) = p^{\,t_n(\mathbf{x})}(1-p)^{\,n - t_n(\mathbf{x})}.$

This is already in the required factorised form

$P(\mathbf{X} = \mathbf{x}) = g\big(t_n(\mathbf{x}),\, p\big) \cdot h(\mathbf{x}),$

where

$g\big(t_n(\mathbf{x}), p\big) = p^{\,t_n(\mathbf{x})}(1-p)^{\,n - t_n(\mathbf{x})}$

depends on the sample $\mathbf{x}$ only through $t_n(\mathbf{x})$, and

$h(\mathbf{x}) = 1$

is a function of $\mathbf{x}$ alone, independent of the parameter $p$.

Conclusion. By the Neyman–Fisher factorisation criterion, $t_n(\mathbf{X}) = \sum_{i=1}^n X_i$ is a sufficient statistic for the parameter $p$. That is, the total number of successes in the sample contains all the information about $p$ that the sample provides; the order in which the successes occur is irrelevant.

Neyman–Pearson Lemma

Let $X_1,X_2,\dots,X_n$ be a random sample from a population with p.d.f. or p.m.f. $f(x;\theta)$. Consider testing the simple null hypothesis

$$H_0:\theta=\theta_0$$

against the simple alternative hypothesis

$$H_1:\theta=\theta_1.$$

Let the joint p.d.f. or p.m.f. of the sample be

$$f(\mathbf{x};\theta)=f(x_1,x_2,\dots,x_n;\theta).$$

Statement: For a given level of significance $\alpha$, the most powerful test for testing $H_0$ against $H_1$ is the likelihood ratio test:

$$\text{Reject }H_0 \text{ if } \frac{f(\mathbf{x};\theta_1)}{f(\mathbf{x};\theta_0)}>k,$$ $$\text{Accept }H_0 \text{ if } \frac{f(\mathbf{x};\theta_1)}{f(\mathbf{x};\theta_0)}<k,$$

and if the ratio equals $k$, reject $H_0$ with probability $\gamma$, where $k$ and $\gamma$ are chosen so that

$$P(\text{Reject }H_0\mid H_0)=\alpha.$$

Proof:
Let

$$L(\mathbf{x})=\frac{f(\mathbf{x};\theta_1)}{f(\mathbf{x};\theta_0)}.$$

Define the likelihood ratio test function $\phi^*(\mathbf{x})$ by

$$\phi^*(\mathbf{x})= \begin{cases} 1, & L(\mathbf{x})>k,\\ \gamma, & L(\mathbf{x})=k,\\ 0, & L(\mathbf{x})<k. \end{cases}$$

Here $\phi^*(\mathbf{x})$ denotes the probability of rejecting $H_0$ when $\mathbf{x}$ is observed. Its size is

$$E_0[\phi^*(\mathbf{X})]=P(\text{Reject }H_0\mid H_0)=\alpha.$$

Let $\phi(\mathbf{x})$ be any other test of $H_0$ against $H_1$ with size not exceeding $\alpha$, i.e.

$$E_0[\phi(\mathbf{X})]\le \alpha.$$

Now consider the product

$$(\phi^*(\mathbf{x})-\phi(\mathbf{x}))(L(\mathbf{x})-k).$$

If $L(\mathbf{x})>k$, then $\phi^*(\mathbf{x})=1$ and $\phi(\mathbf{x})\le 1$, so

$$\phi^*(\mathbf{x})-\phi(\mathbf{x})\ge 0,\qquad L(\mathbf{x})-k>0.$$

If $L(\mathbf{x})<k$, then $\phi^*(\mathbf{x})=0$ and $\phi(\mathbf{x})\ge 0$, so

$$\phi^*(\mathbf{x})-\phi(\mathbf{x})\le 0,\qquad L(\mathbf{x})-k<0.$$

If $L(\mathbf{x})=k$, the product is zero. Hence in all cases,

$$(\phi^*(\mathbf{x})-\phi(\mathbf{x}))(L(\mathbf{x})-k)\ge 0.$$

Taking expectation under $H_0$,

$$E_0\left[(\phi^*(\mathbf{X})-\phi(\mathbf{X}))(L(\mathbf{X})-k)\right]\ge 0.$$

Therefore,

$$E_0[(\phi^*-\phi)L]-kE_0[\phi^*-\phi]\ge 0.$$

But

$$L(\mathbf{x})f(\mathbf{x};\theta_0)=f(\mathbf{x};\theta_1).$$

So,

$$E_1[\phi^*-\phi]\ge kE_0[\phi^*-\phi].$$

Now,

$$E_0[\phi^*-\phi]=\alpha-E_0[\phi]\ge 0$$

because $E_0[\phi]\le \alpha$. Since $k>0$,

$$E_1[\phi^*-\phi]\ge 0.$$

Thus,

$$E_1[\phi^*]\ge E_1[\phi].$$

That is, the power of the likelihood ratio test is at least as large as the power of any other test of size $\alpha$. Hence the likelihood ratio test is the most powerful test. This proves the Neyman–Pearson Lemma.

Kolmogorov–Smirnov goodness-of-fit test

Given data (n = 20):

Hypotheses:

• $H_0$: The children of different heights are equally enrolled (the heights are uniformly distributed over the distinct values).
• $H_1$: The children of different heights are not equally enrolled.

Test statistic: $D = \max|F_o(x) - F_e(x)|$

Level of significance: $\alpha = 0.05$

Critical value: For $n = 20$ at 5%, $D_{0.05} = \dfrac{1.36}{\sqrt{20}} = 0.304$

Computation:

There are $k = 8$ distinct heights, so the expected relative frequency of each is $1/8 = 0.125$.

| Height $x$ | $f$ | $F_o(x)$ | $F_e(x)$ | Difference $D_i$ | |---|---|---|---|---|---| | 2.6 | 1 | 0.05 | 0.125 | 0.075 | | 2.7 | 1 | 0.10 | 0.250 | 0.150 | | 2.8 | 2 | 0.20 | 0.375 | 0.175 | | 2.9 | 5 | 0.45 | 0.500 | 0.050 | | 3.0 | 6 | 0.75 | 0.625 | 0.125 | | 3.1 | 3 | 0.90 | 0.750 | 0.150 | | 3.2 | 1 | 0.95 | 0.875 | 0.075 | | 3.3 | 1 | 1.00 | 1.000 | 0.000 | | Total | 20 | | | |

$D = \max|F_o(x) - F_e(x)| = 0.175$

Decision: Since $D = 0.175 < D_{0.05} = 0.304$, we fail to reject $H_0$ at the 5% level of significance.

Conclusion: There is not enough evidence to conclude that the children of different heights are unequally enrolled; the data are consistent with the hypothesis that children of different heights are equally enrolled in the school.

Median test for two samples (small samples)

The median test checks the null hypothesis $H_0$ that two independent samples come from populations with the same median (against $H_1$: the medians differ). It uses only whether each observation lies above or below the pooled median, so it needs no assumption of normality.

Procedure

1. State $H_0$: both samples are drawn from populations with the same median; $H_1$: the medians are different (or one is larger, for a one-tailed test). Choose the level of significance $\alpha$ (usually 5%).
2. Pool the two samples (sizes $n_1$ and $n_2$, $N = n_1 + n_2$) and find the combined median $M$.
3. For each sample count the observations above $M$ and at or below $M$ (observations equal to $M$ are put in the "not above" group, or dropped). Arrange the counts in a $2\times 2$ table:

4. Choose the test statistic according to size.
   • For small samples ($N<20$, or any expected cell frequency below 5) the chi-square approximation is unreliable, so use Fisher's exact test. Under $H_0$ the probability of the observed table is $P=\frac{\binom{n_1}{a}\binom{n_2}{b}}{\binom{N}{a+b}}$ and the p-value is the sum of the probabilities of this table and all more extreme tables.
   • When Fisher's test is not used, apply Yates' correction for continuity: $\chi^2=\frac{N\left(|ad-bc|-\tfrac{N}{2}\right)^2}{(a+b)(c+d)(a+c)(b+d)}$ with 1 degree of freedom.
5. Decision: reject $H_0$ if the p-value is below $\alpha$ (or if $\chi^2$ exceeds the tabulated value, 3.84 at 5% with 1 d.f.). Otherwise do not reject $H_0$ and conclude there is no significant difference between the medians.

i. Laplace distribution A continuous distribution (also called the double exponential distribution) with pdf $f(x)=\frac{1}{2\sigma}e^{-|x-\mu|/\sigma},\qquad -\infty<x<\infty,\ \sigma>0.$ It is symmetric about $\mu$, has mean $\mu$ and variance $2\sigma^2$, and arises as the distribution of the difference of two independent exponential variables. It is used when data have heavier tails than the normal.

ii. Probability mass function of a bivariate random variable If $(X,Y)$ is a discrete bivariate random variable, its joint pmf is $p(x,y)=P(X=x,\ Y=y),$ satisfying (a) $p(x,y)\ge 0$ for all $(x,y)$, and (b) $\sum_x\sum_y p(x,y)=1$. The marginal pmfs are $p_X(x)=\sum_y p(x,y)$ and $p_Y(y)=\sum_x p(x,y)$.

iii. Mean and variance of Beta distribution of second kind $B_2(m,n)$ For the pdf $f(x)=\dfrac{x^{m-1}}{B(m,n)(1+x)^{m+n}},\ x>0$: $\text{Mean}=\frac{m}{n-1}\ (n>1),\qquad \text{Variance}=\frac{m(m+n-1)}{(n-1)^2(n-2)}\ (n>2).$

iv. Negative hypergeometric distribution It is the distribution of the number of failures $X$ obtained before the $r$-th success when sampling without replacement from a finite population of size $N$ containing $K$ successes. Its pmf is $P(X=x)=\frac{\binom{r+x-1}{x}\binom{N-r-x}{K-r}}{\binom{N}{K}},\qquad x=0,1,\dots,N-K,$ with mean $\dfrac{r(N-K)}{K+1}$.

v. Standard error of sample proportion If $X\sim B(n,p)$ and $\hat p=X/n$, then $SE(\hat p)=\sqrt{\frac{p(1-p)}{n}}\ \approx\ \sqrt{\frac{\hat p(1-\hat p)}{n}}.$ For finite populations the finite population correction $\sqrt{\dfrac{N-n}{N-1}}$ is also applied.

vi. Uses of F-distribution

1. Testing equality of two population variances ($F=S_1^2/S_2^2$).
2. Analysis of variance (ANOVA) for comparing three or more means.
3. Testing the overall significance of a multiple regression model.
4. Comparing nested models and constructing confidence intervals for the ratio of variances. It is the distribution of the ratio of two independent chi-square variables each divided by its degrees of freedom.

vii. Cramer–Rao inequality For a regular family $f(x;\theta)$ with Fisher information per observation $I(\theta)=E\left[\left(\frac{\partial \ln f}{\partial \theta}\right)^2\right],$ any unbiased estimator $T$ of $\theta$ satisfies $Var(T)\ \ge\ \frac{1}{n\,I(\theta)}.$ (For a biased estimator, $Var(T)\ge \dfrac{[1+b'(\theta)]^2}{n\,I(\theta)}$, where $b(\theta)$ is the bias.)

viii. Properties of a good estimator

1. Unbiasedness: $E(\hat\theta)=\theta$.
2. Consistency: $\hat\theta \xrightarrow{P} \theta$ as $n\to\infty$.
3. Efficiency: it has the smallest variance among unbiased estimators (attains the Cramer–Rao lower bound).
4. Sufficiency: it uses all the information in the sample about $\theta$.
5. Minimum mean square error and, ideally, robustness to small departures from assumptions.

ix. Most powerful test For testing a simple null $H_0:\theta=\theta_0$ against a simple alternative $H_1:\theta=\theta_1$ at a fixed level of significance $\alpha$, the test that has the maximum power $(1-\beta)$ among all tests of size $\alpha$ is called the most powerful (MP) test. By the Neyman–Pearson lemma, the likelihood ratio test $\lambda=\frac{L(\theta_1)}{L(\theta_0)}>k$ is the most powerful test.

x. Advantages of non-parametric tests

1. Distribution-free: no assumption about the form of the population distribution (normality etc.).
2. Suitable for small samples and for nominal/ordinal (ranked) data.
3. Simple to understand and easy to compute.
4. Robust to outliers, extreme values and heterogeneous variances.
5. Applicable when parametric assumptions are violated or the data are only ranks/scores.

Derivation of moments about origin of the standard Laplace distribution using its MGF

The standard Laplace distribution has pdf $f(x)=\frac{1}{2}e^{-|x|},\qquad -\infty<x<\infty.$

Moment generating function. By definition, $M_X(t)=E(e^{tX})=\frac{1}{2}\int_{-\infty}^{\infty}e^{tx}e^{-|x|}\,dx.$

Split the integral at $x=0$: $M_X(t)=\frac{1}{2}\left[\int_{-\infty}^{0}e^{tx}e^{x}\,dx+\int_{0}^{\infty}e^{tx}e^{-x}\,dx\right]$ $=\frac{1}{2}\left[\int_{-\infty}^{0}e^{(t+1)x}\,dx+\int_{0}^{\infty}e^{(t-1)x}\,dx\right].$

The first integral converges for $t+1>0$, i.e. $t>-1$: $\int_{-\infty}^{0}e^{(t+1)x}\,dx=\frac{1}{t+1}.$

The second integral converges for $t-1<0$, i.e. $t<1$: $\int_{0}^{\infty}e^{(t-1)x}\,dx=\frac{1}{1-t}.$

Hence, for $-1<t<1$,

$$M_X(t)=\frac{1}{2}\left(\frac{1}{1+t}+\frac{1}{1-t}\right) =\frac{1}{2}\cdot\frac{(1-t)+(1+t)}{1-t^{2}} =\frac{1}{1-t^{2}}.$$

So $\boxed{M_X(t)=(1-t^{2})^{-1},\qquad |t|<1.}$

Moments about origin. The $r$-th moment about origin is $\mu_r'=E(X^{r})=M_X^{(r)}(0).$

Expand the MGF as a binomial series: $M_X(t)=(1-t^{2})^{-1}=1+t^{2}+t^{4}+t^{6}+\cdots=\sum_{k=0}^{\infty}t^{2k}.$

Comparing with $M_X(t)=\sum_{r=0}^{\infty}\mu_r'\dfrac{t^{r}}{r!}$, the coefficient of $t^{r}$ gives $\mu_r'=0$ for odd $r$, and for even $r=2k$, $\frac{\mu_{2k}'}{(2k)!}=1 \quad\Longrightarrow\quad \mu_{2k}'=(2k)!.$

Therefore the moments about origin are

$$\boxed{\mu_r'= \begin{cases} 0, & r \text{ odd},\\[4pt] r!, & r \text{ even}, \end{cases}} \qquad\text{i.e.}\qquad \mu_0'=1,\ \mu_1'=0,\ \mu_2'=2,\ \mu_3'=0,\ \mu_4'=24,\ \dots$$

In particular:

• Mean: $\mu_1'=0$
• Second moment about origin: $\mu_2'=2!=2$
• Variance: $\mu_2=\mu_2'-(\mu_1')^{2}=2-0=2$
• $\mu_4'=4!=24$, so kurtosis $\beta_2=\dfrac{\mu_4}{\sigma^{4}}=\dfrac{24}{4}=6$ (leptokurtic).

Let the lifetime be
[ X\sim \text{Exponential}(\theta) ] with p.d.f. [ f(x)=\frac{1}{\theta}e^{-x/\theta},\qquad x>0,\ \theta>0 ] so that the mean lifetime is (E(X)=\theta).

Given: [ n=50,\qquad \bar x=200 ]

Maximum likelihood estimate of (\theta)

For a random sample (x_1,x_2,\dots,x_n), the likelihood function is [ L(\theta)=\prod_{i=1}^n \frac{1}{\theta}e^{-x_i/\theta} =\frac{1}{\theta^n}e^{-\sum x_i/\theta} ]

Taking logarithms, [ \log L(\theta)=-n\log\theta-\frac{\sum x_i}{\theta} ]

Differentiating with respect to (\theta), [ \frac{d}{d\theta}\log L(\theta) =-\frac{n}{\theta}+\frac{\sum x_i}{\theta^2} ]

For maximum likelihood, [ -\frac{n}{\theta}+\frac{\sum x_i}{\theta^2}=0 ] [ \frac{\sum x_i}{\theta^2}=\frac{n}{\theta} ] [ \theta=\frac{\sum x_i}{n}=\bar x ]

Hence the MLE of (\theta) is the sample mean: [ \hat{\theta}=\bar x=200 ]

So, [ \boxed{\hat{\theta}=200\text{ hours}} ]

Probability that a component lasts more than 250 hours

[ P(X>250)=\int_{250}^{\infty}\frac{1}{\theta}e^{-x/\theta},dx ] [ =e^{-250/\theta} ]

Using (\hat{\theta}=200), [ P(X>250)=e^{-250/200}=e^{-1.25} ]

[ \boxed{P(X>250)=0.2865} ]

Therefore, the maximum likelihood estimate of (\theta) is 200 hours, and the probability that a component lasts more than 250 hours is approximately 0.2865.

Given: Area A: $n_1=300$, $x_1=180$, so $\hat p_1 = 0.60$. Area B: $n_2=250$, $x_2=150$, so $\hat p_2 = 0.60$.

Point estimate: $\hat p_1 - \hat p_2 = 0.60 - 0.60 = 0$.

Standard error:

$SE = \sqrt{\frac{\hat p_1\hat q_1}{n_1}+\frac{\hat p_2\hat q_2}{n_2}} = \sqrt{\frac{0.6\times0.4}{300}+\frac{0.6\times0.4}{250}} = \sqrt{0.0008+0.00096} = \sqrt{0.00176} = 0.0420$

Critical value: for 95% confidence, $z_{0.025} = 1.96$.

Margin of error: $1.96\times0.0420 = 0.0822$.

95% confidence interval:

$(\hat p_1-\hat p_2)\pm 1.96\,SE = 0 \pm 0.0822 = (-0.0822,\ 0.0822)$

Interpretation: We are 95% confident that the true difference between the proportions favouring the policy in the two areas lies between $-0.082$ and $0.082$. Since the interval contains 0, there is no significant difference between the two areas.

Let the finite population consist of (N) units with values
[ Y_1,Y_2,\dots,Y_N ] and population mean [ \bar Y=\frac1N\sum_{i=1}^{N}Y_i. ] Define the population variance as [ S^2=\frac{1}{N-1}\sum_{i=1}^{N}(Y_i-\bar Y)^2. ]

Draw a simple random sample without replacement of size (n) from this population. Let the sample values be (y_1,y_2,\dots,y_n), with sample mean [ \bar y=\frac1n\sum_{i=1}^{n}y_i. ] The sample variance with divisor ((n-1)) is [ s^2=\frac{1}{n-1}\sum_{i=1}^{n}(y_i-\bar y)^2. ]

We prove that [ E(s^2)=S^2. ]

Use the identity [ \sum_{i=1}^{n}(y_i-\bar y)^2

\frac{1}{2n}\sum_{i=1}^{n}\sum_{j=1}^{n}(y_i-y_j)^2. ]

Let (I_i) be the indicator variable such that [ I_i= \begin{cases} 1, & \text{if population unit } i \text{ is in the sample},\ 0, & \text{otherwise}. \end{cases} ] Then [ \sum_{i=1}^{n}(y_i-\bar y)^2

\frac{1}{2n}\sum_{i=1}^{N}\sum_{j=1}^{N}I_iI_j(Y_i-Y_j)^2. ]

Taking expectation, [ E\left[\sum_{i=1}^{n}(y_i-\bar y)^2\right]

\frac{1}{2n}\sum_{i=1}^{N}\sum_{j=1}^{N}E(I_iI_j)(Y_i-Y_j)^2. ]

For (i=j), the term ((Y_i-Y_j)^2=0). For (i\neq j), [ E(I_iI_j)=P(\text{both } i \text{ and } j \text{ are in the sample})

\frac{n(n-1)}{N(N-1)}. ] Therefore, [ E\left[\sum_{i=1}^{n}(y_i-\bar y)^2\right]

\frac{1}{2n}\cdot \frac{n(n-1)}{N(N-1)} \sum_{i\neq j}(Y_i-Y_j)^2. ] So [ E\left[\sum_{i=1}^{n}(y_i-\bar y)^2\right]

\frac{n-1}{2N(N-1)} \sum_{i\neq j}(Y_i-Y_j)^2. ]

Now, [ \sum_{i\neq j}(Y_i-Y_j)^2

2N\sum_{i=1}^{N}(Y_i-\bar Y)^2. ] But [ \sum_{i=1}^{N}(Y_i-\bar Y)^2=(N-1)S^2. ] Hence, [ \sum_{i\neq j}(Y_i-Y_j)^2

2N(N-1)S^2. ]

Substituting this value, [ E\left[\sum_{i=1}^{n}(y_i-\bar y)^2\right]

\frac{n-1}{2N(N-1)}\cdot 2N(N-1)S^2

(n-1)S^2. ]

Therefore, [ E(s^2)

\frac{1}{n-1}E\left[\sum_{i=1}^{n}(y_i-\bar y)^2\right]

\frac{1}{n-1}(n-1)S^2

S^2. ]

Hence, [ \boxed{E(s^2)=S^2} ] so the sample variance with divisor ((n-1)) is an unbiased estimator of the population variance under simple random sampling without replacement.

Total balls (=5+4+3=12).

[ p_B=\frac{5}{12},\qquad p_G=\frac{4}{12}=\frac13,\qquad p_Y=\frac{3}{12}=\frac14 ]

Since the sample is drawn with replacement, the 4 draws are independent. Hence ((X,Y)) follows a multinomial distribution with (n=4). Let (Z=4-X-Y) be the number of yellow balls. Then

[ P(X=x,Y=y)=\frac{4!}{x!,y!,(4-x-y)!} \left(\frac{5}{12}\right)^x \left(\frac13\right)^y \left(\frac14\right)^{4-x-y}, ]

where (x,y\ge 0,\ x+y\le 4), and (P(X=x,Y=y)=0) otherwise.

Joint probability distribution of (X) and (Y)

Marginal distribution of (X)

Since (X) is the number of blue balls in 4 draws,

[ X\sim \text{Binomial}\left(4,\frac{5}{12}\right) ]

[ P(X=x)=\binom{4}{x}\left(\frac{5}{12}\right)^x\left(\frac{7}{12}\right)^{4-x}, \qquad x=0,1,2,3,4. ]

Marginal distribution of (Y)

Since (Y) is the number of green balls in 4 draws,

[ Y\sim \text{Binomial}\left(4,\frac13\right) ]

[ P(Y=y)=\binom{4}{y}\left(\frac13\right)^y\left(\frac23\right)^{4-y}, \qquad y=0,1,2,3,4. ]

Thus the required joint and marginal probability distributions are as above.

Given: $n = 16$, $\bar x = 74$, $s = 8$; population normal, $\sigma$ unknown and $n<30$, so the $t$-distribution is used.

$df = n-1 = 15,\qquad t_{0.025,15} = 2.131$

Standard error:

$SE = \frac{s}{\sqrt n} = \frac{8}{\sqrt{16}} = 2$

Margin of error: $t\cdot SE = 2.131\times2 = 4.262$.

95% confidence interval:

$\bar x \pm t_{0.025}\frac{s}{\sqrt n} = 74 \pm 4.262 = (69.74,\ 78.26)$

Interpretation: We are 95% confident that the population mean lies between 69.74 and 78.26.

Data:

Hypotheses: $H_0$: the machines do not differ in producing defectives ($p_1=p_2=p_3=p_4$); $H_1$: they differ significantly.

Test: chi-square test for homogeneity of proportions, $\alpha = 0.05$.

Expected defectives $E_i = \dfrac{54}{420}\times n_i$:

$E_1 = 12.857,\quad E_2 = 15.429,\quad E_3 = 11.571,\quad E_4 = 14.143$

$\chi^2_{cal} = 1.109$

(Using defectives only is the usual textbook treatment; the non-defective cells give the same conclusion.)

Degrees of freedom: $4-1 = 3$; critical value $\chi^2_{0.05,3} = 7.815$.

Decision: Since $1.109 < 7.815$, $H_0$ is not rejected.

Conclusion: The machines do not differ significantly in producing defectives; the observed differences are due to chance.

Setup. A population of $N$ items contains $k$ successes and $N-k$ failures. Items are drawn one by one without replacement until the $r$-th success. Let $Y$ be the number of draws needed (so $X = Y - r$ is the number of failures before the $r$-th success).

PMF. The $r$-th success occurs at draw $y$ if there are $r-1$ successes among the first $y-1$ draws and draw $y$ is a success:

$P(Y=y) = \frac{\binom{y-1}{r-1}\binom{N-y}{k-r}}{\binom{N}{k}},\qquad y=r,\dots,N-k+r$

We use the identity $\displaystyle\sum_j \binom{j}{a}\binom{m-j}{b} = \binom{m+1}{a+b+1}$.

Mean. Since $y\binom{y-1}{r-1} = r\binom{y}{r}$,

$E(Y) = \frac{r}{\binom Nk}\sum_y \binom yr\binom{N-y}{k-r} = \frac{r\binom{N+1}{k+1}}{\binom Nk} = \frac{r(N+1)}{k+1}$

Hence $E(X) = E(Y)-r = \frac{r(N+1)}{k+1}-r = \boxed{\frac{r(N-k)}{k+1}}$

Variance. Use $(y+1)\binom yr = (r+1)\binom{y+1}{r+1}$:

$E[Y(Y+1)] = \frac{r(r+1)}{\binom Nk}\sum_y \binom{y+1}{r+1}\binom{N-y}{k-r} = \frac{r(r+1)\binom{N+2}{k+2}}{\binom Nk} = \frac{r(r+1)(N+1)(N+2)}{(k+1)(k+2)}$

Then $\operatorname{Var}(Y) = E[Y(Y+1)] - E(Y) - [E(Y)]^2 = \frac{r(r+1)(N+1)(N+2)}{(k+1)(k+2)} - \frac{r(N+1)}{k+1} - \frac{r^2(N+1)^2}{(k+1)^2}$

Taking out $\dfrac{r(N+1)}{k+1}$ and putting the bracket over $(k+1)(k+2)$, the numerator simplifies to

$(r+1)(N+2)(k+1)-(k+1)(k+2)-r(N+1)(k+2) = (N-k)(k+1-r)$

so $\boxed{\operatorname{Var}(X)=\operatorname{Var}(Y)=\frac{r(N+1)(N-k)(k+1-r)}{(k+1)^2(k+2)}}$

Check: $N=10,\ k=5,\ r=2$: $E(X)=\dfrac{2\cdot5}{6}=\dfrac53$ and $\operatorname{Var}=\dfrac{2\cdot11\cdot5\cdot4}{36\cdot7}=\dfrac{110}{63}$, which agrees with direct summation over the PMF.