4
$\begingroup$

Let $P,Q$ be two distributions on a finite set $X$. Consider the following metric* ​$$ d(P,Q) = \frac12\max_{\emptyset\neq A\subseteq X} \|P​(\cdot\mid A)-Q(\cdot\mid A)\|_1. $$ Obviously, the total variation metric $\frac12\|P-Q\|_1$ is majorized by $d(P,Q)$.

Question: has anyone encountered $d(P,Q)$ in the literature? Does it have a name?

*It's not immediately obvious that $d$ satisfies the triangle inequality, but I think this can be shown.

$\endgroup$
2
  • $\begingroup$ Do you mean to require that $P,Q$ have full support? If not then some of these conditional probabilities will be undefined. (And in any case, to be pedantic, you need to exclude $A = \emptyset$.) $\endgroup$ Jul 2, 2018 at 13:35
  • $\begingroup$ Yes, I just realized this myself -- this metric will only be defined on the interior of the distribution simplex. $\endgroup$ Jul 2, 2018 at 14:00

1 Answer 1

1
$\begingroup$

I think it is just called (scaled) supremum $L_1$ norm and it is mostly studied in Bayesian nonparametric estimation literature, especially posterior consistency. The following paper investigate conditional density yet it is obvious that corresponding probability measure is exactly what you wrote down.

De Blasi, Pierpaolo, and Stephen G. Walker. "Posterior asymptotics in the supremum $ L_ {1} $ norm for conditional density estimation." Electronic Journal of Statistics 10.2 (2016): 3219-3246.

$\endgroup$
5
  • $\begingroup$ It seems that their metric is defined on distributions over a product space. I don't see how their definition yields mine, would you be able to elaborate? $\endgroup$ Jul 2, 2018 at 15:14
  • $\begingroup$ Yes. $P,Q$ in OP are measure corresponding to posterior predictive densities in my mind, in that paper they only defined posteriors so in order to yield them to be defined on the same (finite) $\mathbb{X}$ you only have to yield the corresponding posterior predictive measure. The conditioned set $A$ is the observation. Of course you are right that $\mathbb{X}\neq \mathbb{R}$ for posteriors in Bayesian literature mostly. $\endgroup$
    – Henry.L
    Jul 2, 2018 at 15:31
  • $\begingroup$ If we take the definition here: en.wikipedia.org/wiki/Posterior_predictive_distribution Then $Y=\mathbb{R}$ in that paper is the parameter space and $\mathbb{X}$ is the sample space on which you usually draw samples. $\endgroup$
    – Henry.L
    Jul 2, 2018 at 15:35
  • $\begingroup$ But where is the maximum over subsets? I only see maximum over points. $\endgroup$ Jul 2, 2018 at 15:35
  • $\begingroup$ They consider one sample case so only one observation $x\in\mathbb{X}$, if you want a finite sequence of observations then $\mathbb{X}^n$ or even countably many observations $\mathbb{X}^\infty$. Then sample space can be chosen as you wish and the metric can be generalized easily. Or am I missing sth more subtle? $\endgroup$
    – Henry.L
    Jul 2, 2018 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.