An Application of Linear Algebra

Kevin McGoldrick - Math 345, Fall 2009

Least Squares Estimation

More Linear Algebra

Theorem 2: Suppose A is an n x m matrix with linearly independent columns. Then ATA is invertible.




Proof: It suffices to show that ATAx = 0 has only the solution x = 0. If we suppose that ATAx = 0, this implies that Ax = o. Indeed, recall that the null space of AT and the column space of A are orthogonal complements. As Ax exists in both, it must be the zero vector. Let c1,,cm denote the columns of A. Then,

Ax = x1c1 + + xmcm


As the columns of A are linearly independent, we must have x = 0. Thus, ATAx = 0 only when x = 0. As such, ATA must be invertible.

Matrix A

Matrix AT

Then, ATA is invertible, quickly seen as it is symmetric.