Verdict Thresholds: From Score to Action

The Strategy Health Card converts a numeric composite into a four-level verdict: HEALTHY, MONITOR, REVIEW, RECONSIDER. The bucketing is intentionally coarse because a single score does not have enough resolution to support fine distinctions. This article explains the choice of cutoffs, what each verdict implies for what to do next, and why a "RECONSIDER" verdict on a high-return backtest is often the correct read.

The cutoffs

• ≥ 70 — HEALTHY. Move to paper trading; design risk overlay.
• 50–69 — MONITOR. Identify the weakest sub-score; targeted fix.
• 30–49 — REVIEW. Two or more pillars weak; redesign, do not tune.
• < 30 — RECONSIDER. Not deployable; abandon or reframe entirely.

Why these cutoffs?

The cutoffs are calibrated against empirical observation across thousands of factor strategies in academic literature and practitioner backtests. The boundaries align with three implicit thresholds:

• 70 is the level at which all three pillars must be at least adequate. Below 70, a strategy is either failing one pillar entirely or weak across the board.
• 50 is the level at which the strategy passes a coin-flip test: it has at least one pillar that is genuinely strong, but at least one that is genuinely weak.
• 30 is the level at which most strategies are below the threshold of "would even pass robust statistical scrutiny." Bailey, Borwein, López de Prado & Zhu (2014) document that the vast majority of published backtests, when stress-tested for backtest overfitting, fall into this band.

What HEALTHY does and does not mean

A HEALTHY verdict means: this strategy is structurally sound across factor signal, risk discipline, and cost survivability, in the period sampled, with the configuration sampled. It does NOT mean:

• The strategy will work going forward (out-of-sample performance is not measured here).
• The strategy is appropriate for your capital base, time horizon, or tax status (those are situational).
• The strategy is original (you may be re-discovering a known anomaly that is already arbitraged out).
• The strategy will work in your live execution environment (slippage, latency, fill quality differ).

HEALTHY is necessary, not sufficient. The next step is paper trading (see Paper Trading) or walk-forward validation if applicable.

RECONSIDER on a high-return backtest

The most common surprise to new users is a RECONSIDER verdict on a backtest reporting +120% total return. The composite catches this because the score weighs structural soundness, not return level. Three classic patterns produce this:

1. Single-regime miracle. All gains compressed into one regime (e.g., 2020 Q2 rebound). Risk sub-score collapses on regime-breadth penalty.
2. Concentrated three-name bet. Effective N near 3 even with 30 holdings (one stock has 80% weight via market-cap weighting + a runaway). HHI penalty hits Risk sub-score hard.
3. Factor-rotation overfit. Backtest accidentally enables rotation that depends on knowing the future macro regime. Factor sub-score collapses on negative IC + low specific return.

In all three, the headline return is real (it happened in the data) but the structure that produced it is not repeatable. The verdict catches this where the headline number does not.

Sub-score weighting

A useful intuition: a strategy with sub-scores (factor=80, risk=50, cost=40) lands at:

The same total return profile with (factor=50, risk=80, cost=70) lands at:

Both land in MONITOR — but the actionable next step differs. The first case demands a redesign of the risk overlay; the second demands deeper investigation of where the factor signal actually lives.

The verdict is not a destination

The most disciplined users treat the verdict as a triage decision: HEALTHY → continue, others → diagnose. The mistake to avoid is treating MONITOR as "almost HEALTHY" and tweaking parameters until the verdict flips. Parameter-tuning to flip a verdict is the textbook mechanism of backtest overfitting (Bailey & López de Prado 2014). The correct response to MONITOR is to understand the weak sub-score, not to tune around it.