All Questions
25 questions with no upvoted or accepted answers
6
votes
0
answers
578
views
Maximal Correlation versus Correlation Coefficient When one RV is Gaussian
Let a pair of random variables $(X,Y)$ be continuous random variables (i.e., they both have density with respect to Lebesgue measure) with joint distribution $P_{XY}$. The maximal correlation $\rho_m(...
- 1,109
4
votes
0
answers
144
views
Exponential families closed under affine transformations
Let $(\Omega,\Sigma,\mu)$ be a probability space and let $\mathcal{M}$ be an exponential family of probability distributions for $\mu$ of the following form: There are $\varphi_1,\dots,\varphi_n:\...
- 5,405
4
votes
0
answers
573
views
An inequality involving conditional variance and its connection to information theory
Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$ ...
- 1,109
4
votes
1
answer
839
views
A balls into bins problem with combinatorial constraints
We are given $m$ balls and $n$ bins, with $m \ge n$. Each bin can contain at most $c$ balls (we assume that $c$ is an even integer). In a sequential fashion, at each time step, one ball is placed into ...
- 1,677
3
votes
0
answers
80
views
Seeking strong bounds on KL-divergence and martingales for a hypothesis-testing inequality
Let's say we have a finite set $\mathcal{O}$ of observations, and let $\mathcal{C}(\Delta\mathcal{O})$ denote the space of closed convex sets of probability distributions.
We have two hypotheses which ...
- 353
3
votes
0
answers
93
views
Asymptotic approximation of Fisher information matrix for small Gaussian perturbation
Let
$$
X=Y/a+b+\epsilon Z,
$$
where $Y\sim\operatorname{Poisson}(\lambda)$ and $Z\sim\mathcal N(0,1)$ are independent. Also define $\theta=(\lambda,a,b,\epsilon)$. The Fisher information matrix
$$
...
3
votes
0
answers
178
views
Partitioning the coupons collected in the classical coupon collector's problem
Suppose that there is an urn containing $n$ different coupons, from which $m$ coupons are being collected, equally likely, with replacement.
Let $C(m)$ be the whole set of the $m$ collected coupons. ...
- 1,677
3
votes
0
answers
158
views
How are these two multi-armed bandit problems similar?
I am reading the multi-armed bandit survey by Bubeck and Bianchi. This question is for the lower bound section (2.3) of the survey.
Let us define Kullback-Leibler divergence $kl(p, q) = p \log \frac{p}...
- 145
3
votes
0
answers
698
views
How does Jensen Shannon divergence and KL divergence correlate?
I am wondering if there is way to derive the correlation between Jensen Shannon divergence and KL divergence for two distributions: P and Q, in order to show that if JSD(P,Q) decreases, KLD(P,Q) ...
- 31
3
votes
0
answers
494
views
Maximization of a total variation distance subject to another total variation distance in Markov chain
Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
- 1,109
3
votes
0
answers
108
views
"Soft" Voronoi cells or statistical criterias
It is probably some basic statistics question, but...
Informally 1: How to choose "criteria", such that it will guarantee that error decision probability is less than "epsilon", and maximize ...
- 24.9k
3
votes
0
answers
213
views
Find a minimum entropy code for a simple gibbs random field.
Just to make precise what I am talking about, I will include the definition of a minimum entropy code. I will then define the precise markov random field I am asking about.
In the rest of this ...
- 131
3
votes
0
answers
125
views
Is a parametric family which is universally consistent for multiple quantiles impossible?
Suppose I am dead-set on using Bayesian inference on independent and identically distributed data, but I'm lazy and insist on using a parametric likelihood function come what may. I'd be reassured to ...
- 2,791
3
votes
0
answers
143
views
finding rank-3 tensors compatible with a rank-2 tensor projection
I am interested in the following problem: Consider a rank-3 symmetric tensor $\boldsymbol{\sigma}$ with $\sigma_{ijk}$ where $\sigma_{ijk}$ can be 0 or 1, and the symmetry is with respect to any ...
- 41
2
votes
0
answers
92
views
Construct a Bregman divergence from Wasserstein distance
I was wondering whether one has studied the Bregman divergence arising from a squared Wasserstein distance.
More precisely, let $\Omega\subset \mathbb{R}^d$ be a compact set and $c\in \Omega\times \...
- 503
2
votes
0
answers
68
views
What is an efficient non-adaptive group testing scheme if the number of defectives, $d$, grows proportionally to the number of items, $n$?
Suppose that for some $p \in \left(0, 1\right)$ and some $n \in \mathbb{N}$, we have $n$ independent Bernoulli random variables, $X_{1}, X_{2}, \dots, X_{n}$, each with mean $p$. We shall call $X_{1}, ...
2
votes
0
answers
149
views
Min Max Equality in Information Theory
Let $\mathcal{Y}$ and $\mathcal{X}$ be finite sets and let $Q_Y$ be a fixed probability mass function on $\mathcal{Y}$. Also, let $P_{X | Y}$ be some fixed conditional distribution on $\mathcal{X} \...
- 121
2
votes
0
answers
979
views
How to calculate/approximate expectation of function of a binomial random variable?
Hi,
I am stuck at following problem in my research.
Suppose that $M=m$ is a random variable with binomial distribution with parameters $n,p$. The constants $r$ and $\gamma$ are greater than zero. $\...
- 21
1
vote
0
answers
48
views
Sample complexity of estimating a doubly stochastic matrix
Let $P\in\mathbb{R}^{n\times n}$ be a doubly-stochastic matrix. That is:
$$P(x,y)\geq 0,\quad \sum_xP(x,y)=1,\quad \sum_yP(x,y)=1.$$
I would like to know if lower and upper bounds on the sample ...
- 111
1
vote
0
answers
65
views
Normalizing constants preserve metric entropy
Suppose $\mathcal{F}=\left\{f\in L^2([a,b]): 0<\underline{c}\leq f\leq\overline{c} \right\}$. Consider the following transformation
$$\tilde{\mathcal{F}} := \left\{\frac{f}{\int f d\mu}: f\in \...
- 11
1
vote
0
answers
212
views
A new notion of probability coupling
Let $X$ and $Y$ be two discrete random variables distributed according to $\mu$ and $\nu$, respectively. Consider the following optimization problems
$$\inf_{\pi\in \Pi(\mu, \nu)}\Pr(X\neq Y),$$
...
- 1,109
1
vote
0
answers
438
views
Chain rule for maximal correlation
Let a pair of random variables $(X,Y)$ be defined over finite alphabet $\mathcal{X}\times \mathcal{Y}$ with joint distribution $P_{XY}$. The maximal correlation $\rho(X;Y)$ between $X$ and $Y$ is ...
- 1,109
0
votes
0
answers
85
views
When is a family of distributions "closed" with respect to minimal sufficient statistics?
As in the title, I am interested in understanding how to express the idea that a parametric family of distribution is "closed" with respect to minimal sufficient statistics. Before giving ...
0
votes
0
answers
52
views
Classifier-specific lower bounds on the misclassification rate in binary classification
Consider a binary classification problem for $(X,Y)$, and let $\hat{f}$ be a proposed classifier. We wish to bound the misclassification rate $P(\hat{f}(X)\ne Y)$. There are many known lower bounds on ...
- 13
0
votes
0
answers
171
views
A basic property of maximal correlation
Let $𝑋$ and $𝑌$ be random variables. Then the maximal correlation $\rho_{m}(X;Y)$ is defined as:
$$\rho_{m}(X;Y):=\max_{f,g}\mathbb{E}[f(X)g(Y)],$$
where the maximization is taken over real-valued ...
- 11