Per-Regime Optimization

Markets behave differently in different regimes. The same strategy that earns Sharpe 2.0 in a low-volatility uptrend may earn Sharpe 0.5 in a high-volatility chop. Per-regime optimization formalizes this: detect regimes, then run a separate parameter optimization within each, and switch parameters live based on which regime is currently active.

The Two-Stage Process

Per-regime optimization proceeds in two distinct stages:

Stage 1: Regime Detection

Use a Hidden Markov Model (or similar regime classifier) to label each historical bar with a regime tag — typically "low-volatility trending," "high-volatility choppy," "bear," "recovery." The HMM is fit on the full sample to discover the regimes; the inference produces per-bar regime probabilities.

Stage 2: Per-Regime Grid Search

For each regime, run a separate static grid search using only the bars labeled with that regime. The result: a different optimal parameter cell for each regime. The live strategy reads the current regime probability and switches to the appropriate parameter set.

Why It Beats a Single Static Optimum

A single global optimum is a compromise — parameters that work passably in all regimes but optimally in none. Per-regime optimization lets each regime have its own best parameters, capturing genuine regime-conditional structure when it exists. The key word is "when it exists" — without genuine regime-dependence, per-regime optimization just adds parameters that overfit.

What Per-Regime Optimization Buys You

  • Higher per-regime Sharpe when regimes have genuinely different optimal parameters. Each regime's strategy is tuned for that regime's dynamics.
  • Drawdown reduction in unfavorable regimes — the strategy can become more conservative (or stop entirely) when classified into a hostile regime.
  • Honest regime decomposition in performance attribution. You can see how much each regime contributes to overall returns.

The Costs and Pitfalls

  • Each regime gets less data. If your sample has 5 years and the HMM identifies 4 regimes, each regime has roughly 1.25 years of data. Grid search results on small samples are noisier and more overfit-prone.
  • The number of effective parameters multiplies. A 2-parameter strategy with 4 regimes has 8 effective parameters — you've quadrupled your degrees of freedom. DSR correction must account for this.
  • Regime classification errors propagate. If the HMM mislabels a bar, the strategy applies the wrong parameters. Live regime classification (filtered, not smoothed) is noisier than the in-sample classification used to fit the model.
  • Regime instability. Regimes detected in 2010–2015 may not exist (or may have shifted) in 2020–2025. The classifier itself drifts.

When to Use Per-Regime Optimization

Per-Regime Is Worth the Complexity When
  • You can articulate why regimes matter for this strategy. Mean-reversion that fails in trending markets is a clear story; momentum that fails in choppy markets is a clear story. Generic "more parameters might help" is not.
  • Per-regime grid search shows materially different optima. If the per-regime optima all converge to similar parameters, regime-conditioning isn't adding information — use static.
  • Regimes are stable over multi-year periods. If the HMM regime classifications change every few months due to noise, the regime structure is unreliable.
  • You have enough data per regime. ≥ 200 trades per regime is a reasonable minimum; less, and the per-regime statistics are too noisy.

The Honest Workflow

  1. Run static optimization first. Establish the global optimum and the static-baseline performance.
  2. Fit the regime classifier. 2–4 regimes is the sweet spot. More regimes overfit.
  3. Run per-regime grid search. Compare each regime's optimum to the static optimum.
  4. Diff the results. If per-regime optima are close to the static optimum, the complexity isn't earning its keep — stay with static. If they diverge meaningfully and reflect distinct market behaviors, per-regime is justified.
  5. Walk-forward the per-regime strategy. Crucially, the regime classifier must be re-fit within each fold (using only past data) — fitting it on the full sample and applying per fold leaks future information.
  6. Compare composite OOS Sharpe. Per-regime should outperform static on walk-forward. If it doesn't, the static version wins by Occam's razor.

The Practical Test

The honest comparison: walk-forward static vs walk-forward per-regime. If per-regime's composite OOS Sharpe is, say, 1.6 and static's is 1.2, per-regime is earning its complexity. If per-regime is 1.3 and static is 1.2, the complexity isn't worth it — the small improvement is within sampling error and the simpler model is more robust to regime drift.

The Bottom Line

Per-regime optimization is the right tool when markets genuinely behave differently in different regimes and the regimes can be reliably detected with enough data per regime. When all three conditions hold, it produces meaningful improvement over static. When any condition fails, the complexity becomes a liability rather than an asset. Always justify the move from static to per-regime by demonstrated walk-forward improvement, not by theoretical appeal.

Further Reading

Foundational papers

  • Hamilton, J. D. (1989). A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle. Econometrica, 57(2), 357–384.
  • Ang, A. & Bekaert, G. (2002). International Asset Allocation with Regime Shifts. Review of Financial Studies, 15(4), 1137–1187.
  • Guidolin, M. & Timmermann, A. (2007). Asset Allocation under Multivariate Regime Switching. Journal of Economic Dynamics and Control, 31(11), 3503–3544.

Textbook references

  • Hamilton, J. D. (1994). Time Series Analysis. Princeton University Press.
  • López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.

Related QuanterLab articles

Try it in QuanterLab

In RG001RGMO select Per-Regime Optimizer mode. Compare its composite walk-forward OOS Sharpe to the Static optimizer's on the same strategy. If per-regime doesn't materially outperform static, the regime conditioning isn't earning its complexity.

Back to QuanterLab
Report
Loading report...
Article
Loading article...