Compute State Distribution of Markov Chain at Each Time Step

Open Live Script

This example shows how to compute and visualize state redistributions, which show the evolution of the deterministic state distributions over time from an initial distribution.

Consider this theoretical, right-stochastic transition matrix of a stochastic process.

$P = [\begin{array}{cccccccccccccccccccc} 0 & 0 & 1 / 2 & 1 / 4 & 1 / 4 & 0 & 0 \\ 0 & 0 & 1 / 3 & 0 & 2 / 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 / 3 & 2 / 3 \\ 0 & 0 & 0 & 0 & 0 & 1 / 2 & 1 / 2 \\ 0 & 0 & 0 & 0 & 0 & 3 / 4 & 1 / 4 \\ 1 / 2 & 1 / 2 & 0 & 0 & 0 & 0 & 0 \\ 1 / 4 & 3 / 4 & 0 & 0 & 0 & 0 & 0 \end{array}] .$

Create the Markov chain that is characterized by the transition matrix P.

P = [ 0   0  1/2 1/4 1/4  0   0 ;
      0   0  1/3  0  2/3  0   0 ;
      0   0   0   0   0  1/3 2/3;
      0   0   0   0   0  1/2 1/2;
      0   0   0   0   0  3/4 1/4;
     1/2 1/2  0   0   0   0   0 ;
     1/4 3/4  0   0   0   0   0 ];
mc = dtmc(P);

Plot a directed graph of the Markov chain and identify classes using node colors and markers.

figure
graphplot(mc,ColorNodes=true)

Figure contains an axes object. The axes object contains 2 objects of type graphplot, line. One or more of the lines displays its values using only markers This object represents Period = 3.

mc represents a single recurrent class with a period of 3.

Suppose that the initial state distribution is uniform. Compute the evolution of the distribution for 20 time steps.

numSteps = 20;
X = redistribute(mc,numSteps);

X is a 21-by-7 matrix. Row t contains the evolved state distribution at time step t.

Visualize the redistributions in a heatmap.

figure
distplot(mc,X)

Figure contains an axes object. The axes object with title Distribution of States, xlabel State, ylabel Simulation Step contains an object of type image.

The periodicity of the chain is apparent.

Remove the periodicity from the Markov chain by transforming it to a lazy chain. Plot a heatmap of the transition matrix of the lazy chain.

lc = lazy(mc);
figure
imagesc(lc.P)
axis square
colorbar
title("Theoretical Lazy Chain Transition Matrix")

Figure contains an axes object. The axes object with title Theoretical Lazy Chain Transition Matrix contains an object of type image.

lc is a dtmc object. lazy creates the lazy chain by adding weight to the probability of persistence, that is, lazy enforces self-loops.

Compute the evolution of the distribution in the lazy chain for 20 time steps. Plot the redistributions in a heatmap.

X1 = redistribute(lc,numSteps);

figure
distplot(lc,X1)

Figure contains an axes object. The axes object with title Distribution of States, xlabel State, ylabel Simulation Step contains an object of type image.

View the evolution of the state distribution as an animated histogram. Specify a frame rate of 1 second.

figure
distplot(lc,X1,Type="histogram",FrameRate=1)

Figure contains an axes object. The axes object with title Distribution of States, xlabel State, ylabel Probability contains an object of type categoricalhistogram. This object represents Step 20.

Compute the stationary distribution of the lazy chain. Compare it to the final redistribution in the animated histogram.

xFix = asymptotics(lc)

xFix = 1×7

    0.1300    0.2034    0.1328    0.0325    0.1681    0.1866    0.1468

The stationary distribution and the final redistribution are nearly identical.

Compute State Distribution of Markov Chain at Each Time Step

See Also

Objects

Functions

Related Topics