Kalman Filter: Mean Reversion

The Kalman filter generalizes OU mean-reversion to handle time-varying parameters. Where OU assumes a fixed long-run mean μ, the Kalman version lets μ drift slowly over time, tracked online from market data. This is the right model for pairs and spreads whose equilibrium relationships evolve.

The Idea

Imagine a hedge ratio between two stocks. Yesterday it was 0.85; this morning, after a 5% move in one of them, it might be 0.84 or 0.86. A static OU model assumes one true ratio for all of history. The Kalman filter lets the ratio be a slowly-evolving "hidden state" that it estimates from observations.

At each bar:

Predict: the filter forecasts what the hidden state and observation should be, given yesterday's estimate.
Update: the actual observation arrives. The filter computes how surprising it was (the "innovation") and adjusts the hidden state estimate to reduce future surprise.

The result is a state estimate that adapts to new information without overreacting to single-day noise.

The Two Tuning Parameters That Matter

Q (process noise) controls how fast the hidden state can change per bar. Larger Q → state adapts faster but is more jittery. R (observation noise) controls how much the filter trusts new observations. Larger R → filter believes its prior more, smooths more aggressively. The Q/R ratio is everything.

Mean-Reversion Setup

For a single-asset Z-score strategy, the state is the current "fair value" mean. The model says:

μ(t) = μ(t−1) + w(t), w ~ N(0, Q) price(t) = μ(t) + v(t), v ~ N(0, R)

So the fair value is a random walk (slow drift), and the observed price is the fair value plus i.i.d. noise. The filter's job is to recover μ(t) from observed prices and report a Z-score on (price − μ).

For pairs, the state is the hedge ratio β(t). The model says:

β(t) = β(t−1) + w(t), w ~ N(0, Q) y(t) = β(t) × x(t) + v(t), v ~ N(0, R)

where x and y are the two prices. The filter tracks β over time, and the residual y − β × x is the spread.

Tuning Q and R in Practice

QuanterLab's grid search over Q and R is the principled way to find the right values. Some pragmatic priors:

Q very small (e.g., 1e-5): state barely moves. Reduces to nearly-static OU. Good if you believe the relationship is stable.
Q moderate (1e-3 to 1e-2): state adapts to slow regime drift. Typical sweet spot for pairs.
Q large (> 0.1): state tracks aggressively. Almost no mean-reversion edge — you're just smoothing prices.
R proportional to historical price-noise variance. A reasonable initial guess is the sample variance of first differences.

The grid search produces a Sharpe surface across (Q, R). The stable plateau is what you want — the single best cell is overfit.

Why Use Kalman Over Static OU?

Adapts to regime change without retraining. When two stocks' equilibrium relationship shifts, the filter slowly updates β. A static OU strategy keeps trading the old relationship until you manually refit.
Online computable. Only the current state estimate is needed for the next bar — no full retrain.
Quantifies uncertainty. The filter outputs not just an estimate but the variance of that estimate. This can be used for sizing (smaller positions when the filter is uncertain).

Where Kalman MR Breaks Down

If the assumed model is wrong — e.g., if β is stable and you set Q high — the filter wastes information chasing noise as drift.
Non-Gaussian shocks. Earnings gaps, regime breaks, and tail events violate the Gaussian assumption and can permanently distort the state estimate.
Initialization sensitivity. The initial state estimate matters; bad initialization can take many bars to wash out.
Overfitting Q and R. A 100-cell grid search on Q and R is just as susceptible to curve-fitting as any other parameter sweep. Robustness-sweep before trusting.

The Bottom Line

Kalman mean-reversion is OU made adaptive. It is the right model when you believe the equilibrium relationship slowly evolves; it is overkill (and risky) when you believe it is stable. The Q/R tuning is the same parameter-selection problem as anything else in quant — sweep it, find the plateau, trade somewhere in the middle, and walk-forward validate before deploying.

Kalman, R. E. (1960). A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82(1), 35–45.
Avellaneda, M. & Lee, J.-H. (2010). Statistical Arbitrage in the U.S. Equities Market. Quantitative Finance, 10(7), 761–782.
Elliott, R. J., van der Hoek, J. & Malcolm, W. P. (2005). Pairs Trading. Quantitative Finance, 5(3), 271–276.

Textbook references

Harvey, A. C. (1990). Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge University Press.
Hamilton, J. D. (1994). Time Series Analysis. Princeton University Press.

Try it in QuanterLab

In SC001STCB run the Kalman MR method with the default Q and R, then sweep Q ∈ [1e-5, 1e-2] and R ∈ [1e-3, 1.0]. The plateau in the Sharpe heatmap is your robust parameter region — pick somewhere in its interior.