An Application of Linear Algebra

Kevin McGoldrick - Math 345, Fall 2009

Least Squares Estimation

Advanced Least Squares

Least squares estimation can be used to do more than simply estimate the relationship between two variables.  In many cases, a dependent variable may be affected by multiple independent variables.  For example, rather than income depending only on education, as it obviously does not, we may hypothesize that age, gender, and experience all play a role in determining the level of income.

 

In this case, least squares estimation can be used to estimate the effect of each variable on income, when the others are held constant.  This is extremely useful as dependent variables almost always are explained by more than one independent variable.  Of course, with multiple explanatory variables, we lose the ability to depict the least squares solution in a standard plot.

 

It should also be noted that least squares estimation can be expanded to estimate non-linear effects as well.  This can be achieved by observing an independent variable, and supposing that the dependent variable is also related to some other power of the independent variable (e.g. the square).  Thus, we can still represent the model with a matrix, and compute the coefficient on each term.

A non-linear least squares estimation line