Is there a neat way to use something like the Brier score to score an infinite set of forecasts/outcomes?
Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)
|
1
|
|
|
|
|
5
|
A general form of the Brier score, for an essentially arbitrary outcome space $\cal X$, is as follows. Let $q(\cdot)$ be your quoted density for a random quantity $X$, with respect to a dominating measure $\mu$ over $\cal X$. Then your score, when outcome $X=x$ is realised, is $$S(x, q(\cdot)) = \int q(t)^2 d\mu(t) - 2q(x).$$ The Brier score is just one of an infinity of proper scoring rules. For some alternatives, see e.g.: Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102, 359-378. |
||
|
|

