Positive Semi-definite Matrix

Description:

• $A⪰0$
• Symmetric matrix with non-negative eigenvalues
• If and only if it can be expressed as a product of the form $A=R{R}^{⊺}$ for some matrix $R$ (having $n$ rows).
• The PSD property implies $\forall x\in {\mathbb{R}}^n:x^{⊺}Ax=||R^{⊺}x|{|}_2^2\ge 0$
• Negative semidefinite Matrix, $A⪯0$
  • If $-A⪰0$
  • and negative definite: $A\prec 0$
• If moreover, $x^{⊺}Ax>0,\forall 0\ne x\in {\mathbb{R}}^n$ then $A$ is said to be Positive Definite Matrix
  • $A\succ 0$
  • Mean all eigenvalues are positive
• A positive semidefinite matrix is actually positive definte if and only if it is invertible

Eigenvalues of PSD matrix:

• Eigenvalue
• Consider a PSD matrix $C=R{R}^T$. Let $R=US{V}^T$ be an SVD of $R$, we have $C=U\Lambda U^T\text{ where }\Lambda =S{S}^T=\mathrm{diag}\left({\sigma }_1^2,\dots ,{\sigma }_r^2,0,\dots ,0\right)$
• Hence, the eigenvalues of $C$ are non-negative.
• Conversely, if $C=U\wedge U^T$ with $\Lambda =\mathrm{diag}\left({\lambda }_1,\dots ,{\lambda }_n\right),{\lambda }_i\ge 0$ for every $i$, then $C=R{R}^T$, with $R=U\mathrm{diag}\left(\sqrt{{\lambda }_1},\dots ,\sqrt{{\lambda }_n}\right)$

Congruence transformation:

• For any matrix $A\in {\mathbb{R}}^{m,n}$ it holds that:
  1. $A^{T}A⪰0$, and $A{A}^T⪰0$;
  2. $A^{T}A\succ 0$ if and only if $A$ is full-column rank, i.e., $\mathrm{rank}A=n$;
  3. $A{A}^T\succ 0$ if and only if $A$ is full-row rank, i.e., $\mathrm{rank}A=m$

Matrix square-root

• Let $A\in {\mathbb{S}}^n$. Then $\begin{array}{rl}& A⪰0\iff \exists B⪰0:A=B^2\\ & A\succ 0\iff \exists B\succ 0:A=B^2.\end{array}$
• Matrix $B=A^{1/2}$ is called the matrix square-root of $A$.
• Any $A⪰0$ admits the spectral factorization $A=U\wedge U^T$, with $U$ orthogonal and $\Lambda =\mathrm{diag}\left({\lambda }_1,\dots ,{\lambda }_n\right),{\lambda }_i\ge 0,i=1,\dots ,n$.
  • Defining ${\Lambda }^{1/2}=\mathrm{diag}\left(\sqrt{{\lambda }_1},\dots ,\sqrt{{\lambda }_1}\right)$ and $B=U{\Lambda }^{1/2}{U}^T$ : $\begin{array}{rl}& A⪰0\iff \exists B:A=B^{T}B\\ & A\succ 0\iff \exists B\text{ nonsingular : }A=B^{T}B\end{array}$
• $A$ is positive definite if and only if it is congruent to the identity

As Ellipsoids:

• Positive-definite matrices are intimately related to geometrical objects called ellipsoids.
• A full-dimensional, bounded ellipsoid with center in the origin can indeed be defined as the set $\mathcal{E}=\left\{x\in {\mathbb{R}}^n:x^T{P}^{-1}x\le 1\right\},P\succ 0.$
• The eigenvalues ${\lambda }_i$ and eigenvectors $u_i$ of $P$ define the orientation and shape of the ellipsoid: $u_i$ the directions of the semi axes of the ellipsoid, while their lengths are given by $\sqrt{{\lambda }_i}$.
• Using the Cholesky Decomposition $P^{-1}=A^{T}A$, the previous definition of ellipsoid $\mathcal{E}$ is also equivalent to: $\mathcal{E}=\left\{x\in {\mathbb{R}}^n:\Vert Ax{\Vert }_2\le 1\right\}$.

Schur Complement

• Let $A\in {\mathbb{S}}^n,B\in {\mathbb{S}}^m,X\in {\mathbb{R}}^{n,m}$, with $B\succ 0$. Consider the symmetric block matrix $M=\left(\begin{array}{cc}A& X\\ X^T& B\end{array}\right),$
  • and define the so-called Schur complement matrix of $A$ in $M:S\doteq A-X{B}^{-1}{X}^T$
• Then, $M⪰0\text{ (resp., }M\succ 0\text{ ) }\iff S⪰0\text{ (resp., }S\succ 0)$
• Geometric interpretation:
  • Since $M$ is PSD, the set $\mathcal{E}:=\left\{\left(\genfrac{}{}{0ex}{}{x}{y}\right):{\left(\genfrac{}{}{0ex}{}{x}{y}\right)}^T\left(\begin{array}{cc}A& X\\ X^T& B\end{array}\right)\left(\genfrac{}{}{0ex}{}{x}{y}\right)\le 1\right\}$ is an ellipsoid.
  • Its projection on the space of $x$-variables is the set ${\mathcal{E}}^{\text{proj }}:=\left\{x:{\min }_y{\left(\genfrac{}{}{0ex}{}{x}{y}\right)}^T\left(\begin{array}{cc}A& X\\ X^T& B\end{array}\right)\left(\genfrac{}{}{0ex}{}{x}{y}\right)\le 1\right\}$