Markov Chains

math concept 11 topics use this

Math concept

Markov Chains

Core equation

$$P(X_{n+1}=j \mid X_0,\ldots,X_n) = P(X_{n+1}=j \mid X_n)$$

A Markov chain is a stochastic process where the future depends only on the present, not the past. This "memoryless" property makes them mathematically tractable and widely applicable — from finance to biology to machine learning and computational physics.

Transition matrix

For a finite state space ${1,\ldots,n}$, the chain is characterised by a transition matrix $P$ where $P_{ij} = P(X_{t+1}=j \mid X_t=i)$. Each row sums to 1 (stochastic matrix).

The $n$-step transition probabilities are given by $P^n$ — matrix powers.

Stationary distribution

A distribution $\pi$ is stationary if $\pi = \pi P$, i.e., it is a left eigenvector of $P$ with eigenvalue 1. For irreducible, aperiodic chains, $\pi$ is unique and the chain converges to it:

\[P^n_{ij} \to \pi_j \text{ as } n\to\infty\]

Cross-field connections

Markov chains connect: ML (hidden Markov models), finance (credit ratings, interest rate models), ecology (population dynamics), bioinformatics (sequence alignment), and physics (Monte Carlo simulation).

Fields that use this concept

Life sciences Bioinformatics

Comparative Genomics

Measuring evolutionary divergence between genomes through synteny, substitution rates, and neutrality tests.

Hidden Markov Models in Bioinformatics

Probabilistic sequence models for gene finding, CpG island detection, and profile-based database search.

Phylogenetics

Inferring evolutionary trees from molecular sequences using substitution models and likelihood methods.

Physical sciences Computational chemistry

Molecular Dynamics

Classical simulation of atomic motion by integrating Newton's equations with empirical force fields.

Monte Carlo Methods in Chemistry