Directional hit rate measures one narrow thing: across the walk-forward backtest windows, how often did the median forecast call the correct up-or-down sign by the horizon end? 50% is a coin flip — no directional skill at all.
Direction and calibration are independent. CI90 coverage tells you whether the uncertainty band is the right width; directional hit rate tells you whether the forecast leaned the right way. A model can have well-calibrated 90% bands and still be a coin flip on direction, or vice versa. This metric isolates the direction question so the two never get conflated.
The committed backtest contains only a few dozen effective windows — the Fear & Greed leaderboard scores direction over roughly 33 horizon-end predictions per model. With a sample that small, a gap of a few percentage points between two models is well within random chance, not evidence one is better. The windows also overlap in the underlying data, so they are not fully independent and the effective sample is smaller still. On that leaderboard the leading model’s directional hit rate sits in the low-to-mid 60s percent — suggestive, but read it with the sample size firmly in mind.
Calling the sign correctly is not the same as making money. Directional hit rate says nothing about how big the up moves were versus the down moves, entry and exit timing, transaction costs, or risk. A model can be right on direction more than half the time and still lose money if the losers are bigger than the winners. This is a research diagnostic, not a promise of profit.
It means no directional skill — a coin flip. Only a hit rate durably above 50% indicates the forecast leans the right way more often than chance.
No. Calibration (CI90 coverage) is about whether the uncertainty band is the right width; directional hit rate is about whether the forecast pointed the right way. A model can score well on one and poorly on the other.
Be cautious. The committed backtest has only a few dozen effective windows, and those windows overlap, so small differences between models are within the range of noise rather than proof of skill. It is also not a trading track record — calling direction right does not by itself mean a profit.
Browse all S&P 500 tickers to see this metric applied to individual companies.
Educational research only — not investment advice.