1
$\begingroup$

Fix $n > 0$, and consider the space $\cal P$ of probability functions defined over the Boolean closure of a fixed $\cal S = \{ s_1, \ldots, s_n \}$. The Brier score of $P \in \cal P$ at $s_i \in \cal S$ is given by $$BS(P,s_i) = \sum_{1 \leq j \leq n} (P(s_i) - \mathbb 1_{i = j})^2,$$ where $\mathbb 1_{i=j}$ is 1 if $i=j$ and 0 otherwise. The Brier score is strictly proper in that, for any $P \neq Q \in \cal P$,

$$\sum_i P(s_i) BS(P,s_i) < \sum_i P(s_i) BS(Q,s_i).$$

Fix $\lambda_i$, with $(1 \leq i \leq n)$, $\lambda_i > 0$. The following variant of the Brier score is also proper: $$BS_\lambda(P,s_i) = \sum_{1 \leq j \leq n} \lambda_j \cdot (P(s_j) - \mathbb 1_{i = j})^2.$$ This is essentially because $$a\cdot(1-x)^2 + (1-a)\cdot x^2$$ takes its minimum at $x=a$, and because $$\sum_i P(s_i) \cdot BS_\lambda(Q,s_i) = \sum_i \lambda_i \left(P(s_i)\cdot(1-Q(s_i))^2 + (1 - P(s_i))\cdot Q(s_i)^2\right).$$

Now, let $\cal F$ denote the Boolean closure of $\cal S$, and set $$BS_{\cal F}(P,s_i) = \sum_{X \in \cal F} (P(X) - \chi_X(s_i))^2,$$ where $\chi_X$ is the characteristic function of $X$. It is easy to see that $BS_{\cal F}$ is also strictly proper.

But again, fix $\lambda_X > 0$, $X \in \cal F$, and set $$BS^\lambda_{\cal F}(P,s_i) = \sum_{X \in \cal F} \lambda_X \cdot(P(X) - \chi_X(s_i))^2.$$

Is $BS^\lambda_{\cal F}$ proper?

If so, given $\lambda_X > 0$, $X \in \cal F$, is it possible to find $\mu_i > 0$, $1 \leq i \leq n$ such that:

$$BS^\lambda_{\cal F}(P,s_i) = BS^\mu(P,s_i).$$


Update: The answer to the first question is yes, since we can equally well write down the expectation of $BS^\lambda_{\cal F}(Q,s_i)$ relative to $P$ as

$$\sum_{X \in \cal F} \lambda_X \cdot \left(P(X)\cdot(1 - Q(X))^2 + (1-P(X)\cdot Q(X)^2\right).$$

The remaining question then is:

Given $\lambda_X > 0$, $X \in \cal F$, is it possible to find $\mu_i > 0$, $1 \leq i \leq n$ such that:

$$BS^\lambda_{\cal F}(P,s_i) = BS^\mu(P,s_i).$$

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.