An Application of Linear Algebra |

Kevin McGoldrick - Math 345, Fall 2009 |

Least Squares Estimation The Problem Defined |

Suppose we have the following points of data, believed to be linearly related: (2,5), (3,1), (-4,7), (-1,2),
Let m denote the slope of such a line and b denote the intercept. For any point (x,y) we wish to have y = mx + b. Hence, we have the system depicted to the right. Then, we have a system of the form Ax = b.
In general, suppose we are given an inconsistent solution to the system of equations Ax = b. Define the error vector ε = b— Ax. It seems natural that the best solution to our system will minimize the norm, or magnitude of the error, ||ε|| (Note: norms are computed with the Euclidean inner product, e.g. ||(e
As such, let χ be the solution which makes ||ε|| as small as possible. The vector χ is known as the least squares solution. |

The vertical distance between a point and the pictured best fit line is the magnitude of each individual error. |