which is what was expected from our calculations. Thus, continuing with the social mobility model, this would tell us that after a sufficient number of generations, the probability that a family will be in the upper class is .2, the middle class .55, and the lower class .25, and that these probabilities are independent of the economic class the family started in.
| ⋮ |
Thus, for a family starting in the lower economic class, after 10 generations the probability that the family will be in the upper class is .2, the middle class .547, and the lower class .255.
| ⋮ |
Thus, for a family starting in the middle economic class, after 10 generations the probability that the family will be in the upper class is .2, the middle class .553, and the lower class .25.
Now consider the initial state vector corresponding to starting in the lower class:
Then we have for x(10):
Notice that the system is converging to a certain set of probabilities for social mobility as the number of generations increases. Also, the set of probabilities that the system is converging to appears to be independent of the initial class of the family. We will now formalize these observations.
Regular Markov Processes and Equilibrium
A Markov process is called a regular Markov process if some power of the corresponding transition matrix has only positive elements. It can be shown that every regular Markov process eventually converges to a certain state vector, called the steady-state vector, that is independent of the initial state vector. When a system reaches the steady-state vector, it is considered to be an equilibrium. The equilibrium of Markov processes is an appealing quality of certain real life models, such as population statistics after multiple generations. 
Consider the problem of trying to determine the steady-state vector s for a given regular Markov process. We know that Ts = s, since if this is not the case, then the steady-state vector is moving away from the steady-state vector, which is a contradiction. Thus, 1 is an eigenvalue of T with corresponding eigenvector s. Hence, if a Markov process is regular, we can determine the steady-state vector of the process by considering the eigenvector of T corresponding to eigenvalue 1 that is normalized such that the sum of the probabilities is equal to 1 . Note that eigenvectors of T corresponding to eigenvalue 1 are unique up to common multiples if T represents a regular Markov process, so there is only one such normalized eigenvector .
Continuing with the above model of social mobility, we have:
For a simple example of a Markov process that is not regular, consider the Markov process with the following transition matrix:
For the above Markov process, the initial state vector is the state vector of the system at any time, and thus the Markov process does not have a steady-state vector that is independent of the initial state vector.
It turns out that, as was the case for the steady-state vector, many important questions about a given Markov process can be recast as questions about the corresponding transitional matrix, and thus problems in probability can be solved using linear algebra techniques.
So far we have only considered the simplest of Markov processes, first-order discrete-time discrete-state Markov processes. While there are a few real systems that can be accurately described by such processes, such as economic mobility over generations, often times we want to consider continuous-time and continuous-state processes, such as the market value of financial stock options over time. These processes may also depend on more previous states than just the current one to determine future states, and are thus of higher order. As you would expect, Markov processes have been generalized to describe a wide range of physical systems. For a further exploration of Markov processes of different types, consult  and .
 Kolman, Bernard and David R. Hill, Elementary Linear Algebra with Applications, Pearson Prentice Hall, 2008.
 Ibe, Oliver C., Markov Processess for Stochastic Modeling, Elsevier Academic Press, 2009.
 Isaacson, Dean L. and Richard W. Madsen, Markov Chains Theory and Applications, John Wiley
 Seneta, E. Non-negative Matrices and Markov Chains, Springer, 2006.
File translated from
On 14 Dec 2009, 21:57.