Key classes

Stationary processes: statistics invariant under time shift — $F_{t_1+\tau,\ldots}(·) = F_{t_1,\ldots}(·)$.

Wide-sense stationary (WSS): $\mathbb{E}[X_t] = \mu$ and $\text{Cov}(X_t, X_{t+\tau}) = R(\tau)$ depend only on lag $\tau$.

Ergodic: time averages equal ensemble averages almost surely.

Autocorrelation and power spectrum

For a WSS process: $R(\tau) = \mathbb{E}[X_t X_{t+\tau}]$.

By the Wiener–Khinchin theorem, the power spectral density is the Fourier transform of $R$:

\[S(\omega) = \int_{-\infty}^\infty R(\tau)\,e^{-i\omega\tau}\,d\tau\]

White noise and filtered processes

White noise: $R(\tau) = \sigma^2\delta(\tau)$, $S(\omega) = \sigma^2$ (flat spectrum).

Passing white noise through an LTI filter $H(\omega)$ gives output PSD $S_Y(\omega) =

H(\omega)

^2 S_X(\omega)$ — the basis of ARMA spectral representations.