Forecast accuracy

Forecast accuracy scorecard

SUZU Monitor logs every forecast it publishes and aims to score each one once its outcome is known. While the audit below is open, this page publishes no scores.

Methodology

This is the scoring method as documented. The audit found places where the scoring did not follow it.

Brier/log score over resolved YES/NO published forecast windows; VOID and pending entries are counted for coverage but excluded from accuracy math. The ledger totals and the maturity funnel count published forecasts only: shadow bets, scored for evidence but never shown, are left out of them. Each window is one question, and it is scored on the probability published when the window opened; a later re-emission of the same question does not change it. While an outcome-fitted calibration gate passes, that probability is calibrated, and the API does not mark which ones are; after a switch between raw and calibrated publication, the rolling window mixes forecasts published under both. 120 forecasts scored against a data feed we could not read correctly are voided and left out of every score (issue #5233). 322 shadow bet windows that opened on a base-rate placeholder instead of a model forecast are left out of every count, VOID included (issue #8990). Since 8 October 2026, 18 of 31 resolved published forecasts were void (58.1%, 95% interval 40.8% to 73.6%); the target is under 15%. This counts published forecasts first issued on or after that date, so it leaves out unpublished shadow bets and the voids relabelled in the issue #8990 audit.

Recently resolved forecasts, unverified

Market alerts: did the news follow?

The audit did not cover this section. Market alerts are scored from a separate ledger, and these figures have not been rechecked.

SUZU Monitor raises a market alert when a market or a prediction market makes an unusual move. Some alerts fire when there is no news behind the move, and one type fires when related news is already out. Each alert is checked 6 hours later. It counts as a hit if an established news outlet published a new story about the same company, commodity or topic in that time. The same check also runs on the same market for a stretch of the same length one day earlier, when no alert was raised, and that gives the base rate. An alert type is useful only when its hit rate is clearly above the base rate on the same alerts. The figures cover the last 30 days and were generated 2026-11-02 04:50:17 UTC.

How an alert is scored

An emission resolves HIT when a story first tracked by the English news digest within six hours after the emission has at least one Tier 1 or Tier 2 source and its title names the same entity: for a market, an alias match for the symbol's registry entity or a whole-word match of the market name of four or more characters; for a prediction market, a topic keyword that appears in the question itself. It resolves MISS when six hours pass with no such story. It resolves VOID, and is not counted, when the story archive for its window expired before the resolver read it. Lead time is the gap between emission and the matching story's first tracking. The base rate applies the same rule to a control window of the same length for the same entity that starts 24 hours before each scored emission, counted only when the archive covers the control window, no alert for that entity was open during it, and the emission and the control fall on the same side of a weekend in UTC.

Related reference

• Why we publish the ledger
• Country Instability Index
• Country Resilience Index
• Crisis trackers
• CII methodology

Download: scorecard.json. Source: SUZU Monitor forecast scorecard snapshot. The raw figures in the download were generated 2026-11-01 06:00:11 UTC and read on 2026-11-02, and are under audit. Live results come from the credentialed forecast scorecard endpoint, which this page freezes so it can be read without one.