1
$\begingroup$

Let $P$ and $Q$ be two distributions over a sample space $\Omega$ which I would like to show are close under some choice of distance function. So far I have managed to show that there exists a subset $S\subseteq \Omega$ such that:

  • $P(S)$ is large, say, at least $(1-\varepsilon)$
  • the conditional distributions of $P$ and $Q$ over $S$, denoted $P_S(x)=P(x)/P(S)$ and $Q_S(x)=Q(x)/Q(S)$ are pointwise close, i.e. $P_S(x)\in [1\pm\varepsilon]\;Q_S(x)$ for any $x\in S$.

My question is: does this correspond to a distributional closeness between $P$ and $Q$ under any well-known divergence? I realize that the above notion of closeness is asymmetric as the first point above is with respect to one of the distributions.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

$\newcommand\Om\Omega$No. E.g., for natural $n$, suppose that $\Om=[n]:=\{1,\dots,n\}$, $S=[n-1]$, $P(x)=\frac1n$ for $x\in\Om$, $Q(x)=\frac1{n^2}$ for $x\in S$, and $Q(n)=1-\frac{n-1}{n^2}$.

Then your conditions hold for $\varepsilon=\frac1n$, $P$ is uniform over $\Om$, but (for large $n$) almost all $Q$-mass is at the one point, $n$.

A probability distribution is a measure. So, you should write $P(\{x\})$ instead of $P(x)$, assuming that the singleton sets $\{x\}$ are in the underlying $\sigma$-algebra -- which, looking at the context of your post, appears to be the largest $\sigma$-algebra over a discrete set $\Om$. However, in the answer above I used your notations, such as $P(x)$. Also, $P_S(x)$ is a number, not a distribution, conditional or not.

Improve this answer
$\endgroup$
12
  • $\begingroup$ thanks, I agree that my notations are a bit informal. $\endgroup$
    – user43170
    Commented Jul 7, 2023 at 16:24
  • $\begingroup$ Then I think you should warn people about informal notations. $\endgroup$
    – Iosif Pinelis
    Commented Jul 7, 2023 at 16:25
  • 3
    $\begingroup$ @IosifPinelis I really disagree with you and think you are being a bit rude. It is very, very common to write $P(x)$ instead of $P(\{x\})$ for discrete probability distributions $P$. Also, writing $P_S(x) = P(x)/P(S)$ is analogous to writing $f(x) = \sin(x)$; you wouldn't object to someone saying "let $f$ denote the function $f(x) = \sin(x)$" by saying "$f(x)$ is a number, not a function". $\endgroup$
    – mathworker21
    Commented Jul 7, 2023 at 16:41
  • $\begingroup$ @mathworker21 : Yes, I would object to someone saying "function $f(x)=\sin(x)$", especially when one can simply say $f=\sin$ instead. In this case, it seems to have turned out that the OP was using notation "informally", but I have encountered many cases when the OP indeed did not know what a distribution is and thought that a formula like $f(x)=e^{-x}$ for $x\ge0$ is a probability distribution. $\endgroup$
    – Iosif Pinelis
    Commented Jul 7, 2023 at 17:03
  • $\begingroup$ Previous comment continued: Again in this case, I had an unpleasant choice to either change notations or use notations I am really uncomfortable with. I also had to assume the discreteness of the distribution. I think the OPs should try to spare readers of their questions of things like that, especially when it is so easy to do. $\endgroup$
    – Iosif Pinelis
    Commented Jul 7, 2023 at 17:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .