Two of the most convincing track records you’ll ever see are convincing because something was quietly left out. Survivorship bias and cherry-picking both work the same way: show the winners, hide the losers, and let you assume you’re seeing the whole picture.
Survivorship bias is what happens when the things that failed have already dropped out of the data before you see it. Example (illustrative): test a stock-picking rule on the companies in today’s S&P 500 and it might look brilliant — but today’s index only contains the survivors. Every company that was dropped, went bankrupt, or got delisted over the years has silently disappeared from the test. The rule was graded on a roster of winners chosen with hindsight. The same trap hides in fund performance: families quietly close or merge their worst funds, so the surviving list looks stronger than the full history ever was.
Cherry-picking is choosing which evidence to show. It takes many shapes: highlighting the one stock the tool nailed while ignoring the ten it missed; quoting the single time window where results shine and skipping the rest; screenshotting the winning trades and never the losing ones. Example (illustrative): “our pick returned 300%!” tells you nothing unless you also know how the other 49 picks did over the same period. One spectacular call is a story, not a track record.
What’s missing? Ask for the complete record — every pick over a fixed period, not a highlight reel. If losers can’t be produced, assume they exist. Does the test include the ones that died? A credible backtest accounts for companies that were delisted or removed, not just today’s survivors. Is the time window fixed in advance, or hand-picked? A record that only works on one convenient stretch is the same overfitting warning from the previous lesson, wearing a different hat. Are the misses shown alongside the hits? A record with no visible losses has almost certainly been filtered.
The opposite of a cherry-picked reel is a fixed, complete, misses-included record. Quantustik’s committed backtest is scored on a pre-declared TOP-20 set and reports the full picture — including the names and horizons where the model is weak, such as stocks with a structural break in their business that its confidence bands cover poorly at the 1-year mark. Publishing where you fail is the clearest signal a track record hasn’t been filtered to flatter.
A filtered track record makes a tool look far more reliable than it is, so you trust it with larger positions than it has earned. The defence is a habit, not a formula: whenever you see an impressive result, ask what you’re not being shown. The answer separates a genuine edge from a well-edited highlight reel.
This lesson is investor education, not personalized advice. It teaches you to interrogate a track record, not to trust one — and no track record, filtered or complete, is a promise of future returns.
It’s when the failures have already dropped out of the data before you see it — for example, testing a rule on today’s index members ignores every company that was delisted or went bankrupt, so the surviving roster looks far stronger than the full history was.
Cherry-picking is actively choosing which evidence to show — the one winning pick, the one flattering time window — while quietly omitting the rest. Survivorship bias is more passive: the losers dropped out of the dataset on their own. Both leave you seeing only winners.
A fixed, pre-declared set of picks over a set period, with the misses shown next to the hits. A tool that publishes where it fails — specific names and horizons — is far more credible than one showing an unbroken string of wins.
Investor education only — not investment advice, and never a promise of profit. Every investment can lose value.