An Application of Linear Algebra
Kevin McGoldrick - Math 345, Fall 2009
Least Squares Estimation
The Necessary Linear Algebra
Here we present some results allowing us to derive a formula for χ.
Theorem 1: Suppose V is a finite dimensional inner product space, with subspace W. Let u be any vector in V. The “best approximation” from u to W is the orthogonal projection of u onto W, denoted projW u. In other words, for every w in W such that w is not projW u, we have
||u-projW u|| < ||u-w||.
Proof: For any w in W, we have
u-w = (u-(projW u)) + ((projW u)-w).
Note that ((projW u)-w) is a vector in W, and that
(u-(projW u)) is orthogonal to any vector in W. Now,
||u-w||2 = ||(u-(projW u)) + ((projW u)-w)||2
= ||(u-(projW u))||2 + ||((projW u)-w)||2
Since ||((projW u)-w)||2 > 0, then
||u-w||2 > ||(u-(projW u))||2 ,
||u-w|| > ||(u-(projW u))||
Here, P is the orthogonal projection of V onto the plane.
(P = projW v)