Every morning bayesRatings publishes a win probability for each game, and we compare it with the Kalshi price recorded before the game started (the earliest quote we hold for that game). Once games finish, both are scored with log loss, the standard measure for probability forecasts.
01How the scoring works
For each finished game, log loss is −ln(p), where p is the probability a forecast gave to what actually happened. Confident and right scores near 0; confident and wrong is punished hard. A coin flip scores 0.693. Lower is better. We treat the Kalshi price (bid/ask midpoint) as the market's implied number for comparison only; a price is not a probability, and includes fees and market frictions.
The comparison only uses games where both numbers existed in the same morning snapshot. The difference column is model minus Kalshi (negative means the model did better), with a 95% interval; until that interval excludes zero, neither side has shown an edge.