AI Forecast Ledger · Methodology

How the ledger works

How a grade is reached, where the evidence comes from, who owns and pays for the ledger, and how to get something fixed. The sections follow the five principles of the International Fact-Checking Network code. We are not an IFCN signatory.

Claims tracked

11

Graded

10

Corrections logged

8

Rubric changes

none

1 · Fairness

• One process for everyone. A lab chief, a sceptic, a research report and a forecasting community go through the same rubric, the same evidence standard and the same two-model check. Bold timelines and sceptical calls are graded alike.
• We grade the claim, not the person. A verdict says whether a dated prediction came true by its own rubric. It says nothing else about the person who made it.
• Named forecasters only. Every claim names who made it. The build refuses a placeholder such as “a leading expert”.

2 · Sources

• The prediction, from where it was said. We aim to quote each prediction from its primary source: a recording, a transcript or the speaker’s own post. Some claims still quote a news report of the prediction while we replace it; every receipt names the outlet its quote comes from, so you can always see which.
• Two independent sources for a grade. A grade needs at least two sources from different organisations, both rated tier 2 or better in our source registry, and at least one of them a measurement rather than an opinion.
• Receipts. Every quote is stored verbatim and bound to its text by a SHA-256 hash, next to an archived snapshot of the page, so a later edit or deletion cannot change what the grade rests on.
• We never cite ourselves. A link to this site or our own accounts as evidence fails the build.

3 · Ownership and funding

The ledger is owned and run by Setpat Technologies, Unipessoal Lda. (Lisboa, Portugal), whose director, Benjamin Niemack, is responsible for its content. Setpat pays for it. Setpat also builds AI products and does AI consulting. Nobody whose prediction is on the ledger pays for, reviews or approves a grade. No advertiser or sponsor influences a grade, and any sponsorship is labelled on the page where it appears and listed here. Company details are on the imprint.

Sponsors: none yet.

4 · Method

• The rubric is fixed when the claim is logged. Each claim carries a target date and a written rubric saying what counts as met and what counts as missed. Rubric changes since logging: none. The corrections page keeps the count and lists each one with its old wording.
• Two models, blind. One model grades against the hashed evidence. A second model from a different vendor checks the same evidence without seeing the first one’s reasoning. A grade publishes only when both agree. When they disagree, the claim is published as contested and nothing is averaged away.
• A person approves every grade before it goes live.
• Re-reviews. A later re-read may refresh a claim’s headline figure from a new receipt, checked by one model. That never changes a verdict. A new verdict needs a new two-model grade.

Every claim shows one of these labels:

• Hit: settled; the claim has resolved
• On track: still open; may still change
• At risk: still open; may still change
• Missed: settled; the claim has resolved
• Pending: logged, not yet graded
• Overdue: the target date has passed and the claim is not yet resolved. The site shows this from the dates; no grader assigns it.

5 · Corrections

• Tell us. If a grade, a quote or a source is wrong, write to hello@setpat.com. Predictors who think a grade misrepresents them can dispute it, and the claim can be marked contested while we look.
• Every correction is public. Re-grades, withdrawn grades, withdrawn claims, rubric changes and attribution fixes are listed on the corrections page with the date. Each claim page also keeps its own dated history.
• Nothing is quietly deleted. A withdrawn claim stays on the corrections page with the reason it was withdrawn.

Page dated 2026-11-05

← Back to the ledger