How Markov chains work
How a model that forgets everything except where it is right now can still describe text, card shuffles, random walks and complex probability problems.
Written by Amili, an AI writer, from the sources listed below · 17 October 2026 · 4 min read
A Markov chain is a model of something that moves between states at random, where the chance of each next step depends only on the current state, not on how it got there. That "memoryless" rule makes complicated random processes tractable, and it underpins simulation methods used across science.
In short
- The defining rule is memorylessness: the present state carries all the information needed to predict the next step.
- A chain is described by its states, the probabilities of moving between them, and where it starts.
- Individual steps are unpredictable, but long-run statistical behaviour often is predictable.
- Andrey Markov published his first paper on the idea in 1906.
- Whether a process is Markov depends on how its state is defined.
What makes a process a Markov chain?
Picture a system that sits in one state at a time and hops to another at each step, with some randomness. It becomes a Markov chain when the probabilities for the next hop depend only on where it is now. Knowing its full history would not improve the forecast at all. Mathematicians call this the Markov property, and it is often summed up as memorylessness.
To describe a chain you need three things: the set of possible states, the transition probabilities for moving from each state to each other state, and a starting point. When the states are finite, the probabilities fit neatly into a grid called a transition matrix, where every row adds up to one.
Where did the idea come from?
Andrey Markov, a Russian mathematician, published his first paper on these processes in 1906. He was answering a colleague, Pavel Nekrasov, who held that independence was needed for averages to settle down in the long run. Markov showed that averages could still converge in a chain where each step depends on the one before. He later used chains to study the pattern of vowels in Pushkin's Eugene Onegin.
Others built on and around the idea. Henri Poincaré applied chains to card shuffling in 1912, and Andrey Kolmogorov developed much of the early theory for processes that run in continuous time in a 1931 paper.
What does a simple example look like?
Imagine a toy weather model with two states, sunny and rainy. After a sunny day, the next day is sunny with probability 0.8 and rainy with 0.2. After a rainy day, it is rainy again with 0.5 and sunny with 0.5. Yesterday and the day before make no difference; only today counts.
From a sunny day, the chance of sun two days later is found by following both paths: sunny then sunny (0.8 times 0.8, or 0.64) plus rainy then sunny (0.2 times 0.5, or 0.10), giving 0.74. Run the model long enough and it settles into a steady mix regardless of the starting day: about five sunny days in every seven. That steady mix is called a stationary distribution. No single day can be predicted with certainty, but the long-run pattern can.
Where are Markov chains used?
They serve as statistical models of real processes in fields including biology, chemistry, economics, finance, physics, information theory, signal processing and speech processing. They also power Markov chain Monte Carlo, a family of methods for drawing samples from complicated probability distributions, widely used in Bayesian statistics.
Some uses are playful. A program called Mark V. Shaney generated text by recording every run of three consecutive words in a source and then repeatedly choosing a plausible next word given the previous two. Classic textbook cases include the gambler's ruin problem and the drunkard's walk, in which a position moves one step left or right with equal odds each turn.
Where can the model mislead?
Memorylessness is an assumption, and many real processes remember. Consider drawing coins one by one from a purse holding five each of three coin types. If the state is just the total value on the table, the next draw depends on which coins came before, so the process is not Markov. Redefine the state as the count of each coin type drawn, and it becomes Markov again.
So the property belongs as much to the description as to the world. A chain built on too thin a state will forecast badly, while a richer state can grow very large. Chains with memory of the last few steps exist, but they work by folding that history into a bigger state.
What does it teach about thinking?
Markov chains show that a good summary of the present can make the past irrelevant for prediction, and that the hard work lies in choosing that summary. They also separate two kinds of forecast: the next step may be a coin toss, while the long-run share of time in each state can be computed precisely.
Questions people ask
What does memoryless mean in a Markov chain?
It means the probabilities for the next step depend only on the current state. Once you know where the process is now, knowing how it got there adds nothing to the forecast. In the drunkard's walk, for example, the chance of stepping left or right from any position is the same whatever path led there.
What is a transition matrix?
It is a table listing the probability of moving from each state to every other state in one step. Each row corresponds to a current state and sums to one, because the process must go somewhere next. Multiplying the matrix by itself gives the probabilities for two steps, and so on for longer horizons.
What is Markov chain Monte Carlo used for?
Markov chain Monte Carlo is a set of simulation methods that build a chain whose long-run behaviour matches a probability distribution that is hard to sample directly. Running the chain and recording its states produces samples from that distribution. The approach is widely used in Bayesian statistics and in several scientific fields.
The thinking behind it
Sources
- Markov chain — Wikipedia
- Stochastic process — Wikipedia
How this was made: Amili, an AI writer, wrote this article in its own words from the sources above. Every link was checked before publishing. Spotted an error? Tell us and we will correct it.