Least Square Classification

Definition:

• Adding new features:
  • We artificially can add new dimensions with random values in order to seperate the data points into classes more separately, reducing error rate
• Fit model $\tilde{f}(x)$ to encoded $(\pm 1)y^{(i)}$ values using standard least squares data fitting
• $\tilde{f}(x)$ is a scalar number
  • should be near +1 when $y=+1$ and -1 when $y=-1$
• Skewed decision threshold:
  • the funcion is then $\hat{f}(x)=\text{sign}(\tilde{f}(x)-\alpha )$
  • $\hat{f}(x)=+1$ if $\tilde{f}(x)\ge \alpha$
  • $\hat{f}(x)=-1$ if $\tilde{f}(x)<\alpha$
  • $\alpha$ is decision threshold
  • but there will be trade off, ex: if $\alpha$ too low then there will be more false positive
    • the trade off curve of trye positive vs false positive rate is called Receiver Operating Characteristic