An Application of Linear Algebra

Kevin McGoldrick - Math 345, Fall 2009

Least Squares Estimation

A Basic Example

Return to the collection of points presented earlier. To find the least squares solution, we find ATA, compute (ATA)-1 and calculate the following product

(ATA)-1ATb.

Doing so gives the result to the right.

 

We can also calculate the error vector given by this least squares solution. We find that

ε = b Ax = (2.383, -1.050, 0.983, -2.317)

and

||ε|| = 3.622.

As noted before, any other combination of slope and intercept will produce an error vector of greater magnitude than this one.

 

The scatter plot of points with the line corresponding to the least squares solution seems to match what we would intuitively expect. Any other line would have an error vector with greater magnitude than the one achieved here.

Least squares solution

Scatter plot of points with best fit line