Lenses: asking better questions of your research

The most dangerous moment in research is the one right after you see a good number. A backtest prints a Sharpe of 1.8, your eyes light up, and some quiet part of your brain starts looking for reasons it must be real. That reflex — to confirm rather than to test — is not a character flaw. It is how everyone is built. The whole craft of honest research is a set of habits for working against it.

One habit we keep coming back to: never look at a result through a single lens. A good result is not one that survived your enthusiasm. It is one that survived a deliberate attempt to break it from several different angles — would it survive trading costs, is it just one market regime, did the parameters get fit to noise, did information from the future leak into the past? Each of those is a separate question, and each could falsify the result on its own. We call them lenses, and the rest of this article is about why they matter and where they show up in QuanterLab.

Why one view is never enough

Popper (1963) made the point as sharply as anyone: a claim earns its keep not by how many times it is confirmed but by how it stands up to attempts to refute it. A theory that nothing could disprove tells you nothing. The same is true of a strategy. If you can't name the test that would make you abandon it, you haven't tested it — you've admired it. The lens that refutes is the one you learn from. The five confirmations you collected beforehand were mostly you talking yourself into it.

Tetlock and Gardner (2015) found something complementary studying forecasters: the people who were actually well-calibrated were not the ones with the strongest single thesis. They were the ones who held several views at once, weighted them, updated as evidence arrived, and stayed humble about any one of them. The lens habit is a way of operationalising that for quant research — instead of one confident story about why your strategy works, you carry several competing worries about why it might not, and you make each one earn a clean refutation before you set it down.

This matters more in finance than almost anywhere, because the data fights back. Test enough strategies and some will look brilliant by chance alone — the multiple-testing problem that Harvey and Liu (2015) documented across the published cross-section of returns, and that Bailey, Borwein, López de Prado and Zhu (2014) called out as the pseudo-mathematics behind a lot of backtest claims. A single good-looking result is the expected output of trying many things. The lenses are how you tell the rare real edge from the many lucky-looking ones.

Quantin's lenses: the same idea, made literal

Quantin — QuanterLab's in-platform assistant — is built around exactly this framing, and it is not a metaphor we bolted on afterward. Quantin runs in one of five lenses, each a distinct mode that changes what kind of question it is set up to ask. The lenses are defined in wrapper/quantin.py, and you switch between them from a chip in the chat header. The five are:

• Tutor — explains concepts and points you to the right Knowledge Base article. The default; where you orient.
• Librarian — reads your own saved backtests and configs and surfaces patterns you've stopped noticing: redundant variants, a config you haven't re-run in months, three of your best results that all secretly lean on the same filter.
• Mathematician — gives formal, step-by-step derivations (rendered in LaTeX) grounded in the platform's actual implementations rather than textbook generalities.
• Skeptic — the adversarial reviewer.
• Agent — drives the platform end-to-end through the MCP toolset, starting and summarising real workflows on your behalf.

The reason this is more than a UI convenience is what switching costs you: nothing, and that's deliberate. Each lens is a small system-prompt extension stacked after a shared, cached prompt prefix, so changing lens doesn't throw away the platform context — it just re-aims the same assistant at a different question. The friction of looking at your work from a new angle is supposed to be one click, because the whole point of lenses is that you actually use more than one.

The Skeptic lens, specifically

The Skeptic lens is the one that most directly embodies this article. Its instructions describe it, in its own words, as your adversarial reviewer whose job is to find the weakness — and, memorably, as the platform's anti-overfit instinct turned into a person. It is told not to be a yes-man. Its playbook is to find the most fragile claim in what you just said and probe it with a specific, uncomfortable question: why this parameter and not the neighbouring values you didn't test; whether a stop sits inside ordinary noise; whether a fourteen-trade sample is anywhere near enough to mean anything; whether you've actually run walk-forward yet or are still mistaking a headline number for validation. After a few rounds it gives a blunt verdict — Defended, Borderline, or Concerning — and tells you what would change its mind.

Two constraints are worth stating plainly, because they are written into the lens. The Skeptic never recommends a trade and never claims a future return. Its job is to challenge, not to bless. That is the whole posture of the platform compressed into one mode: a tool for not fooling yourself, not an oracle for what to buy.

A starter set of lenses

You don't need Quantin to think in lenses; you need the questions. A practical set, each tied to a way the platform lets you actually check it rather than just worry about it:

• Did I peek? Could information from the future have leaked into a past decision? This is the lens the whole point-in-time machinery in Primitives exists to serve — filings admitted only by their acceptance date, regimes read as-of each bar and acted on one bar later, the Crash-Test and Fog-of-War timeline modes that keep you honestly blind to what came next.
• Is this just one regime? A strategy that only worked because 2017 was calm isn't a strategy, it's a description of 2017. The autopsy's per-regime attribution and the regime detectors exist to make you ask whether the edge survives a different weather.
• Would it survive costs? Gross returns are a fiction. The autopsy's gross-to-net friction waterfall and the cost waterfall in the forward testers exist so the question can't be waved away.
• Is it distinguishable from luck? The Forward-Test Autopsy's Monte-Carlo test doesn't bootstrap your result against itself; it ranks it against a zero-edge null — the same volatility and streak structure with the average return demeaned out, so a no-skill version of your own path becomes the thing you have to beat. The output is a rank and a one-sided p-value: how often pure luck would have matched or beaten you.
• Did I fit the noise? Walk-forward efficiency — out-of-sample over in-sample — and the per-fold parameter surface are the overfitting lens made concrete. A strategy whose out-of-sample performance collapses relative to its in-sample fit was optimised, not discovered.

None of these lenses, passed individually, proves a strategy works. That asymmetry is the honest part. A lens can refute; it can't bless. What surviving all of them buys you is narrower and more truthful: fewer of the specific ways you know you could have been fooling yourself. The edge might still be illusory for a reason none of your lenses covered — markets are good at finding the angle you forgot. But the discipline of deliberately trying to break your own result, from several directions, before you commit a cent to it, is the closest thing the craft has to a defence.

The easiest person to fool is yourself, and the cheapest defence is to keep asking, from a fresh angle, "what would prove me wrong?" — and then to actually go and look. The lenses just make sure you don't only ever look from the angle that already agrees with you.