Proof

Proof in progress.

There are no public results on this site yet. This page says so plainly, and it shows how a result will be scored before one is shown.

The method asks you to write the rule, state the odds, and prove the cause. The same three tests apply to the method itself.

Status
In progress
Calls recorded
0
Calls scored
0

What is here today

The method is written down. The scoring rules on this page are written down. The scorecard is public and it is empty.

No client results, no testimonials, and no figures are shown, because none have been published. An empty record is the honest record today.

How a call gets scored in public

Three tests, in this order. A result that skips one is not shown as proof.

  1. Stated odds

    Each call is written down with a date and the odds, before the outcome is known. The entry does not change after that.

  2. Brier score against a baseline

    When the outcome is known, the stated odds are scored. A Brier score is the average squared gap between the odds you stated and what happened. Lower is better. The score sits next to a baseline, such as always stating the base rate, so you can see whether the calls beat a simple guess.

  3. Effect measured on held out data

    Held out data is set aside before the work starts and is not used to build the rule or the model. A claim that something caused a lift has to hold there. A lift that only shows up on the data it was built from is not proof.

The calibration chart

Stated odds run one way. How often those calls happened runs the other. Honest odds sit on the diagonal. The chart below is empty because no calls have been scored in public yet.

How often it happened

No calls scored yet

Dashed line: the line you are aiming at

Stated odds, low to high

Empty on purpose. No calls have been scored in public yet, so nothing is plotted. The diagonal is the target, not a result. Scored calls will be listed on the scorecard first.

What this page will not show

  • A figure with no source.
  • A score with no baseline next to it.
  • A claim of cause that was only tested on the data it came from.
  • A client name, a logo, or a testimonial used in place of a scored result.

Where the record lives

The scorecard is the dated record. Each call goes in with its stated odds and gets scored later. It is empty today.

Open the scorecardHow a forecast is scoredAsk what caused it