Inferences about a Mean Vector

Definition:

• Given a set of vectors as dataset, what can we infer about the mean

Simple Mean Vector Test:

• Consider the test in ${\mathbb{R}}^p$ with $H_0:\mu ={\mu }_0$ and $H_1:\mu \ne {\mu }_0$
• Define $T^2=n(\bar{X}-{\mu }_0{)}^{⊺}{S}^{-1}(\bar{X}-{\mu }_0)$ , T-squared Statistic
  • Where $\bar{X}=\frac{1}{n}{\sum }_{j=1}^n{X}_j$ and $S=\frac{1}{n-1}{\sum }_{j=1}^n(x_j-\bar{x})(x_j-\bar{x}{)}^{⊺}$
• $T^2\sim \frac{(n-1)p}{n-p}{F}_{p,n-p}$
  • whe re $F_{p,n-p}$ denotes F-distribution with $p$ and $n-p$ degrees of freedom
• At $\alpha$ level of significant, we reject $H_0$ if $T^2>\frac{(n-1)p}{n-p}{F}_{p,n-p}(\alpha )$
  • where $F_{p,n-p}(\alpha )$ is the upper (100$\alpha$)-th percentile of the $F_{p,n-p}$ distribution
  • If $T^2$ is too large, meaning $\bar{x}$ is too far from ${\mu }_0$

Confidence Regions of Component Means:

• A $100(1-\alpha )\%$ confidence region for the mean of a $p$-dimension normal distribution is the ellipsoid determined by all $\mu$ such that $n(\bar{x}-\mu {)}^{⊺}{S}^{-1}(\bar{x}-\mu )\le \frac{(n-1)p}{n-p}{F}_{p,n-p}(\alpha )$
  • where $\bar{x}=\frac{1}{n}{\sum }_{j=1}^n{x}_j$ and $S=\frac{1}{n-1}{\sum }_{j=1}^n(x_j-\bar{x})(x_j-\bar{x}{)}^{⊺}$
• The confidence ellipsoid is $\bar{x}\pm {\lambda }_i\sqrt{\frac{(n-1)p}{n-p}{F}_{p,n-p}(\alpha )e_i}$
  • where $S{e}_i=\lambda e_i$ for $i=1,...p$
  • Eigenvalue-Eigenvector pairs
• Let $X_1,..,X_2$ be a normal random sample from an $N_p(\mu ,\lambda )$ population with $\sum$ positive definite.
  • Then, simultaneously for all $a$, the interval $a^{⊺}\bar{X}\mp \sqrt{\frac{(n-1)p}{n(n-p)}{F}_{p,n-p}(\alpha )a^{⊺}Sa}$ will contain $a^{⊺}\mu$ with probability $(1-\alpha )$
  • these simultaneous intervals are also referred as $T^2$-invervals
  • ex: We can make statements about the differences ${\mu }_i-{\mu }_k$ by choosing $a^{⊺}=[0,0,...,a_i,...,0,0,....,a_k,....,0,0]$ where $a_i=1$ and $a_k=-1$.
    • In this case $a^{⊺}Aa=s_{ii}-2s_{ik}+s_{kk}$ and the interval ${\bar{x}}_i-{\bar{x}}_k\pm \sqrt{\frac{(n-1)p}{n(n-p)}{F}_{p,n-p}(\alpha )(s_{ii}-2s_{ik}+s_{kk})}$ contains ${\mu }_i-{\mu }_k$ with probability $1-\alpha$
• By choosing $a^{⊺}=[0,0,...,a_i,...,0]$ where $a_i=1$, $1\le i\le p$, the interval ${\bar{x}}_i\mp \sqrt{\frac{(n-1)p}{n(n-p)}{F}_{p,n-p}(\alpha )S_{ii}}$ contains ${\mu }_i$ with probability $1-\alpha$

Large sample inferences about a population mean:

• When the sample size is large, tests of hypotheses and confidence regions for $\mu$ can be constructed without the assumption of a normal distribution
• Let $X_1,...,X_n$ be a random sample from a population with mean $\mu$ and positive definite covariance matrix $\sum$ and $n-p$ is large:
  1. $n(\bar{x}-{\mu }_0{)}^{⊺}{S}^{-1}(\bar{x}-{\mu }_0)\ge {\chi }_p^2(\alpha )$, reject $H_0:\mu ={\mu }_0$
     • for level of significant $\alpha$
  2. $a^{⊺}\bar{X}\pm \sqrt{{\chi }_p^2(\alpha )}\sqrt{\frac{a^{⊺}Sa}{n}}$ will contain $a^{⊺}\mu$ for every $a$, with probability approximately $1-\alpha$
     • Consequently, we can make the $100\%(1-\alpha )$ simultaneous confidence statements ${\bar{x}}_i\pm \sqrt{{\chi }_p^2(\alpha )}\sqrt{\frac{S_{ii}}{n}}$ contains ${\mu }_i$ for $i=1,..,p$

Multivariate Quality Control Chart, $\bar{X}$ chart:

• Control charts make the variation visible
• Allow one to distinguish common from special causes of variation.
• One useful control chart is the $\bar{X}$ chart:
  1. Plot the individual observations or sample means in time order.
  2. Create and plot the centerline $\overline{\stackrel{ˉ}{x}}$, the sample mean of all of the observations.
  3. Calculate and plot the control limits given by
     • Upper control limit $(UCL)=\overline{\stackrel{ˉ}{x}}$ + 3(standard deviation)
     • Lower control limit $(LCL)=\overline{\stackrel{ˉ}{x}}$ - 3(standard deviation)
  4. Plot with data points in order from 1; draw 3 sets of identical points for UCL, mean and LCL then connect them

Ellipsoid Format Chart:

• only extract 2 dimensions, therefore $p=2$
  • the 95% quality ellipse consists of all x that satisfy $(x-\bar{x}{)}^T{S}^{-1}(x-\bar{x})\le {\chi }_2^2(0.05)$
• Find Cov, Cov${}^{-1}$
• Find mean
• Determinant = $||\sum ||={\prod }_i{\lambda }_i$
• Trace $=tr(\sum )=\sum {\lambda }_i$
• Find eigenvalues from det and trace (use quadratic)
• Find $a,b$
  • $=\sqrt{{\lambda }_i}\ast \frac{(n-1)p}{n(n-p)}{F}_{p,n-p}(\alpha )$ for small sample
  • $=\sqrt{{\lambda }_i}\ast \sqrt{chi-sq(\alpha ,2)}$. Chi-sq $=CHI.INV(\alpha ,2)$
• Theta(rad) $=\theta =atan2({\sigma }_{12},{\lambda }_1-{\sigma }_{11})$
• Calculate rotation matrix $R=\left[\begin{array}{cc}\cos (\theta )& -\sin (\theta )\\ \sin (\theta )& \cos (\theta )\end{array}\right]$
• Sample for points on esclipse:
  • Generate n points from $0$ to $2\pi$: $i=\frac{k\ast 2\pi }{n}$
  • x coordinate of that point $=R_{11}\ast a\ast \cos (i)+R_{12}\ast b\ast \sin (i)+{\bar{x}}_1$
  • y coordinate of that point $=R_{21}\ast a\ast \cos (i)+R_{22}\ast b\ast \sin (i)+{\bar{x}}_2$
  • plot points and connect them
• Plot data:
  • For each data point: minus mean to normalize it and plot on same graph with ellipse
  • then find if that point is inside $\alpha$ confidence interval
    • $(Co{v}_{11}^{-1}\ast (x_i-\bar{x}{)}_1+Co{v}_{12}^{-1}\ast (x_i-\bar{x}{)}_2)\ast (x_i-\bar{x}{)}_1+(Co{v}_{21}^{-1}\ast (x_i-\bar{x}{)}_1+Co{v}_{22}^{-1}\ast (x_i-\bar{x}{)}_2)\ast (x_i-\bar{x}{)}_2$
    • If $>chi-sq$ : outside, otherwise inside

$T^2$ chart:

• For more than 2 dimensions
• When a point is out of the control region, individual $\bar{X}$ charts are constructed.
• When the lower control limit is less than zero for data that must be nonnegative, LCL is generally set to zero.
• Points are displayed in time order rather than as a scatter plot, and this makes patterns and trends visible.
• For the $j$th points, we calculate the T-squared Statistic: $T_j^2=(x-\bar{x}{)}^{⊺}{S}^{-1}(x-\bar{x})$
• Then plot the $T^2$-values on a time axis, the lower limit is 0, and upper limt is $UCL={\chi }_p^2(\alpha )$, there is no centerline in $T^2$-chart
• When the multivariable $T^2$-chart signals that the $j$-th unit is out of order, it should be determined which variables are responsible
• A region based on Bonferoni Interval is frequently chosen for this purpose. The k-th variable is out of control if $x_{jk}$ does not lie in the interval ${\bar{x}}_k\mp t_{n-1}(0.005/p)\sqrt{s_{kk}}$ where $p$ is the total nb of measured variables

Inference when some observations are missing:

• Often, some components of a vector observation are unavailable. We treat situations where data are missing at random.
• To estimate the incomplete data, we use the EM algorithm.
  1. Prediction step. Given some estimate $\tilde{\theta }$ of the unknown parameters, predict the contribution of any missing observation to the (complete-data) sufficient statistics.
  2. Estimation step. Use the predicted sufficient statistics to compute a revised estimate of the parameters.
• When the observations $X_1,...,X_n$ are a random sample from a p-variate normal population, the prediction–estimation algorithm is based on the complete data sufficient statistics
  • $T_1={\sum }_{j=1}^n{X}_j=n\bar{X}$
  • and $T_2=\sum _{j-1}^2{X}_j{X}_j^{⊺}=(n-1)S+n\bar{X}{\bar{X}}^{⊺}$
• We assume that the population mean $\mu$ and variance $\sum$ are unknown and estimated with $\tilde{\mu }$ and $\tilde{\Sigma }$
• Estimation:
  • $\tilde{\mu }=\frac{{\tilde{T}}_1}{n}$ and $\tilde{\Sigma }=\frac{1}{n}{\tilde{T}}_2-\tilde{\mu }{\tilde{\mu }}^{⊺}$
• Prediction step:
  • for each vector $x_j$ with missing values, let $x_j^{(1)}$ denotes the vector of missing components and $x_j^{(2)}$ denotes vector of available components
  • Contribution estimation of $x_j^{(1)}$ to $T_1:\widetilde{x_j^{(1)}}=E(X_j^{(1)}|x_j^{(2)};\tilde{\mu },\tilde{\Sigma })={\tilde{\mu }}^{(1)}+{\tilde{\Sigma }}_{12}{\tilde{\Sigma }}_{22}^{-1}(x_j^{(2)}-{\tilde{\mu }}^{(2)})$
  • Predicted contribution of $x_j^{(1)}$ to $T_2$ is
    • $\widetilde{x_j^{(1)}{x}_j^{(1)⊺}}=E(X_j^{(1)}{X}_j^{(1)⊺}|x_j^{(2)};\tilde{\mu },\tilde{\Sigma })={\tilde{\Sigma }}_{11}-{\tilde{\Sigma }}_{12}{\tilde{\Sigma }}_{22}^{-1}{\tilde{\Sigma }}_{21}+{\widetilde{x_j}}^{(1)}{\widetilde{x_j}}^{(1)⊺}$
    • $\widetilde{x_j^{(1)}{x}_j^{(2)⊺}}=E(X_j^{(1)}{X}_j^{(2)⊺}|x_j^{(2)};\tilde{\mu },\tilde{\Sigma })={\widetilde{x_j}}^{(1)}{\widetilde{x_j}}^{(2)⊺}$

---