4
$\begingroup$

Let $P,Q$ be two distributions on a finite set $X$. Consider the following metric* ​$$ d(P,Q) = \frac12\max_{\emptyset\neq A\subseteq X} \|P​(\cdot\mid A)-Q(\cdot\mid A)\|_1. $$ Obviously, the total variation metric $\frac12\|P-Q\|_1$ is majorized by $d(P,Q)$.

Question: has anyone encountered $d(P,Q)$ in the literature? Does it have a name?

*It's not immediately obvious that $d$ satisfies the triangle inequality, but I think this can be shown.

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you mean to require that $P,Q$ have full support? If not then some of these conditional probabilities will be undefined. (And in any case, to be pedantic, you need to exclude $A = \emptyset$.) $\endgroup$
    – Nate Eldredge
    Commented Jul 2, 2018 at 13:35
  • $\begingroup$ Yes, I just realized this myself -- this metric will only be defined on the interior of the distribution simplex. $\endgroup$
    – Aryeh Kontorovich
    Commented Jul 2, 2018 at 14:00

1 Answer 1

Reset to default
1
$\begingroup$

I think it is just called (scaled) supremum $L_1$ norm and it is mostly studied in Bayesian nonparametric estimation literature, especially posterior consistency. The following paper investigate conditional density yet it is obvious that corresponding probability measure is exactly what you wrote down.

De Blasi, Pierpaolo, and Stephen G. Walker. "Posterior asymptotics in the supremum $ L_ {1} $ norm for conditional density estimation." Electronic Journal of Statistics 10.2 (2016): 3219-3246.

Improve this answer
$\endgroup$
5
  • $\begingroup$ It seems that their metric is defined on distributions over a product space. I don't see how their definition yields mine, would you be able to elaborate? $\endgroup$
    – Aryeh Kontorovich
    Commented Jul 2, 2018 at 15:14
  • $\begingroup$ Yes. $P,Q$ in OP are measure corresponding to posterior predictive densities in my mind, in that paper they only defined posteriors so in order to yield them to be defined on the same (finite) $\mathbb{X}$ you only have to yield the corresponding posterior predictive measure. The conditioned set $A$ is the observation. Of course you are right that $\mathbb{X}\neq \mathbb{R}$ for posteriors in Bayesian literature mostly. $\endgroup$
    – Henry.L
    Commented Jul 2, 2018 at 15:31
  • $\begingroup$ If we take the definition here: en.wikipedia.org/wiki/Posterior_predictive_distribution Then $Y=\mathbb{R}$ in that paper is the parameter space and $\mathbb{X}$ is the sample space on which you usually draw samples. $\endgroup$
    – Henry.L
    Commented Jul 2, 2018 at 15:35
  • $\begingroup$ But where is the maximum over subsets? I only see maximum over points. $\endgroup$
    – Aryeh Kontorovich
    Commented Jul 2, 2018 at 15:35
  • $\begingroup$ They consider one sample case so only one observation $x\in\mathbb{X}$, if you want a finite sequence of observations then $\mathbb{X}^n$ or even countably many observations $\mathbb{X}^\infty$. Then sample space can be chosen as you wish and the metric can be generalized easily. Or am I missing sth more subtle? $\endgroup$
    – Henry.L
    Commented Jul 2, 2018 at 15:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .