An Application of Linear Algebra
 Kevin McGoldrick - Math 345, Fall 2009
 Least Squares Estimation The Problem Defined
 Suppose we have the following points of data, believed to be linearly related: (2,5), (3,1), (-4,7), (-1,2),   Let m denote the slope of such a line and b denote the intercept.  For any point (x,y) we wish to have     y = mx + b.  Hence, we have the system depicted to the right.  Then, we have a system of the form Ax = b.   In general, suppose we are given an inconsistent solution to the system of equations Ax = b.  Define the error vector ε = b— Ax.  It seems natural that the best solution to our system will minimize the norm, or magnitude of the  error, ||ε|| (Note: norms are computed with the Euclidean inner product, e.g. ||(e1,…,en)|| = (e12 + … + en2)(1/2))   As such, let χ be the solution which makes ||ε|| as small as possible.  The vector χ is known as the least squares solution.
 The vertical distance between a point and the pictured best fit line is the magnitude of each individual error.