“95% accurate” is the headline every forecasting product wants you to read — but “accuracy” is not one thing, and knowing which kind is being claimed is the fastest way to tell a real edge from a dressed-up coin flip.
The directional hit-rate is the share of forecasts that got the direction right — up called up, down called down. It’s the easiest number to advertise and the easiest to misread. Example (illustrative): a tool that says “this stock will rise” and is right 6 times out of 10 has a 60% directional hit-rate. Sounds like an edge — but you can’t judge it until you know one more thing.
The base rate is how often the thing would happen anyway, with no skill involved. Over the long run the U.S. stock market has risen in far more years than it has fallen — so “it will go up” is right most of the time by default. Example (illustrative): if a stock actually rises on 58 of every 100 days, a tool that always predicts “up” scores a 58% hit-rate while knowing nothing at all. A 60% hit-rate only beats that by two points. The question is never “how often is it right?” — it’s “how much more often than an always-up guess?” A hit-rate quoted without its base rate is marketing, not evidence.
Direction is only half the story. The other half is calibration: when a tool says it’s 90% confident, is it actually right about 90% of the time? A well-calibrated tool that admits “I don’t know” when it doesn’t know is far more useful than a confident one that’s wrong at the worst moments. Scoring rules like the Brier score fold direction and confidence into a single honest number precisely so an overconfident tool can’t hide behind a good-looking hit-rate. The next lesson shows how to read a calibration table so you can check this yourself.
Pick a tool on its advertised hit-rate alone and you can easily pay for something that does no better than assuming the market drifts up — then get badly surprised the first time it confidently calls a move that doesn’t happen. Knowing the difference between hit-rate, base rate, and calibration is the single most useful filter for separating a real edge from a coin flip. It’s the standard we hold ourselves to: Quantustik publishes its backtest accuracy and calibration, weak spots included, rather than a single flattering headline.
This lesson is investor education, not personalized advice. Nothing here rates a specific product; it teaches you to read accuracy claims yourself. No forecasting tool, including Quantustik, promises a return.
Not on its own. “Accurate” usually means a directional hit-rate, which is only meaningful compared to the base rate — how often the move would happen anyway. If a stock rises most days, always predicting “up” scores a high hit-rate with no skill at all.
Hit-rate is how often the direction was right. Calibration is whether the tool’s stated confidence matches reality — when it says 90% confident, is it actually right about 90% of the time? A tool can have a decent hit-rate and still be badly overconfident.
“What’s the base rate?” If the tool won’t tell you how often the outcome happens without its forecast, you can’t tell whether its accuracy reflects skill or just the market’s natural tendency to drift upward.
Investor education only — not investment advice, and never a promise of profit. Every investment can lose value.