Skip to main content

All Questions

Filter by
Sorted by
Tagged with
1 vote
1 answer
56 views

How to study the convergence of the sample mode for arbitrary probability spaces

(This is not the problem I actually care about, but an analogy with similar issues to the problem I'm actually considering.) Consider a probability space with i.i.d. random variables $X_i$ producing ...
cgmil's user avatar
  • 277
0 votes
0 answers
31 views

What is the Fisher information matrix of the von Mises-Fisher distribution?

Assuming the von Mises-Fisher distribution as $$f_{p}(\mathbf{x}; \boldsymbol{\mu}, \kappa) = C_{p}(\kappa) \exp \left( {\kappa \boldsymbol{\mu}^\mathsf{T} \mathbf{x} } \right),$$ where $\kappa \ge 0$,...
Math_Y's user avatar
  • 287
2 votes
1 answer
131 views

Almost sure convergence of double averages of IID random variables

Let $ \{X_i\}_{i=1}^{P} $ and $ \{Y_j\}_{j=1}^{Q} $ be two sequences of independent and identically distributed (i.i.d.) random variables. $X_i$ and $Y_j$ are independent between all pairs of $i$ and $...
CWC's user avatar
  • 433
1 vote
0 answers
148 views

conjecture for general form of minimax estimator

I had previously posed an overly ambitious version of this conjecture here, Form of minimax estimator, which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
Aryeh Kontorovich's user avatar
1 vote
1 answer
341 views

Form of minimax estimator

Let $\Delta$ be the set of all probability distributions over $\mathbb{N}=\{1,2,\ldots\}$ and fix some $\mathcal{P}\subseteq\Delta$. Suppose additionally that $\Delta$ is endowed with some norm $||\...
Aryeh Kontorovich's user avatar
0 votes
1 answer
940 views

Derivative of log-likelihood function for Gaussian distribution with parameterized variance

Suppose we have a parameter $\theta \in R^{n}$ that defines some noisy observation $z=\mu(\theta)+\eta, z\in R^{m}$ where the noise follows a Gaussian distribution whose covariance is a function of ...
JNL's user avatar
  • 75
7 votes
1 answer
569 views

Singular Fisher information matrix and existence of unbiased estimators

I'm doing some research into the Cramer-Rao bound for time of arrival localization and have come across a rather strange result: the FIM is singular, but there exists an unbiased estimator. My ...
JNL's user avatar
  • 75
2 votes
0 answers
87 views

A complex problem involving densities (likelihood functions) and optimization

Consider the following autoregressive process with normal errors: \begin{equation}\label{7YlUV4i8nuO}\tag{I} y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2) \end{equation} We ...
PSE's user avatar
  • 13
1 vote
1 answer
101 views

Estimating the average of two gaussians' mean with minimal squared error

This is a follow-up to my previous question. Assume that $X\sim \mathcal N(\mu_1,\sigma_1^2)$ and $Y\sim \mathcal N(\mu_2,\sigma_2^2)$. I want to estimate $\frac{\mu_1+\mu_2}{2}$ after observing $X,Y$....
R B's user avatar
  • 618
2 votes
1 answer
872 views

Estimating the average of two gaussians' mean

Assume that $X\sim \mathcal N(\sigma_1,\mu_1)$ and $Y\sim \mathcal N(\sigma_2,\mu_2)$. I want to estimate $\frac{\mu_1+\mu_2}{2}$ after observing $X,Y$. In my setting, $\sigma_1,\sigma_2$ are known ...
R B's user avatar
  • 618
1 vote
0 answers
81 views

Calculating the mean squared error for an estimate of a large sum

Consider the set of all Boolean function $f: \{0, 1\}^{n} \rightarrow \{-1, 1\}$. Now, let's pick a function uniformly at random from this set. Let $F$ be the random variable corresponding to the ...
RandomMatrices's user avatar
1 vote
0 answers
75 views

Percentile interval Lemma

Let $\theta$ be a parameter and $\hat{\theta}$ the plug-in estimate, I need a proof of the following lemma, as given in [1], p. 173, in the form of a reference or of a direct argument: Percentile ...
Roberto Palermo's user avatar
3 votes
1 answer
139 views

Design a random variable which has the maximal correlation with another random variable

$Y$ is a Gaussian distributed random variable with zero mean and known variance: $Y\sim N(0,\sigma_y)$. We measure $Y$ with a sensor, which is corrupted by white Gaussian noise: $Z=Y+V$; $V\sim N(0,\...
Jing Zhou's user avatar
-1 votes
1 answer
205 views

How to combine estimator with different variances?

Consider independent random variables $X_1,X_2,\ldots,$ that have the same expectation $\mathbb x=\mathbb E[X_1]=\mathbb E[X_2]=\ldots$ Further, assume that we know that $Var[X_i]=\sigma_i^2$. In the ...
M A's user avatar
  • 127
1 vote
0 answers
154 views

How to retrieve back the input using Bussgang theorem?

If we have a non-linear function $f$, that is applied to input $x$, we have then the output $y=f(x)$ Using Bussgang decomposition we can linearize this nonlinearity and express $y$ as $y=Bx+ η$, ...
e. sfe's user avatar
  • 39
6 votes
1 answer
434 views

Probability of complex eigenvalues

I find this is the best site to post this question, even though I considered cs. It is a Monte Carlo experiment over the set of 10.000 n×n matrices. If a single matrix eigenvalue is complex then ...
prosti's user avatar
  • 171
1 vote
0 answers
62 views

Distances between up and down crosses in Gaussian Processes

Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$, where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...
Kikolo's user avatar
  • 91
5 votes
1 answer
107 views

Estimating the size of the remainder in a random partition

Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...
Jan-Christoph Schlage-Puchta's user avatar
-2 votes
1 answer
92 views

Existence or impossibility of Gaussian factory

Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
Sebastian Nowozin's user avatar
3 votes
1 answer
87 views

Optimal linear measurement operator

Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
Aryeh Kontorovich's user avatar
2 votes
1 answer
676 views

Distribution of ratio between complex Gaussian and Chi-square R.V.s

What would be the distribution (p.d.f.) of the following ratio? $$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$ where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
Felipe Augusto de Figueiredo's user avatar
3 votes
1 answer
113 views

maximum likelihood estimation of X is better than that of f(X)?

Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...
Jeff's user avatar
  • 482
2 votes
2 answers
632 views

An alternative proof of Bayesian Cramer-Rao

My question is: Are there an alternative proof of Cramer-Rao lower bound that does not use Cauchy-Swartz inequality? Let me outline the classical proof and explain why I am interested in this ...
Boby's user avatar
  • 671
1 vote
1 answer
193 views

Fisher information with vanishing probability

I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as: $$...
WoofDoggy's user avatar
  • 237
1 vote
1 answer
124 views

Reconstructing the number of distinct elements from a random projection

Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$. We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...
R B's user avatar
  • 618
1 vote
0 answers
49 views

A question about the prediction error

I am reading about the prediction error estimation and I found the following: Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...
neda's user avatar
  • 11
3 votes
1 answer
96 views

Perturbation results for statistical estimators

Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...
Tom Solberg's user avatar
  • 4,049
3 votes
0 answers
82 views

Uniform mean-square-error estimates

Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...
Sam Cohen's user avatar
  • 111
4 votes
1 answer
203 views

Can samples be compressed?

The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is: $\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...
Daniel Moskovich's user avatar
2 votes
1 answer
444 views

Literature question on the convergence rate of the empirical distribution

Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...
user avatar
1 vote
1 answer
256 views

KL divergence Inequality

I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading. Consider two probability measures $P$ and $Q$ both ...
rajatsen91's user avatar
4 votes
1 answer
479 views

Cramér-Rao bound for randomized estimator

As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter. Consider the case when the parameter is a scalar, the estimator is ...
Luis Mendo's user avatar
2 votes
0 answers
51 views

MLE and CRLB with mismatched likelihoods

Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...
Greg's user avatar
  • 21
4 votes
2 answers
519 views

Cramér–Rao type bound for absolute estimation error

Let $\{X_1, X_2, \dotsc, X_n\}$ be independent and identically distributed (i.i.d.) random variables sampled from a common distribution with density $f_{\theta}(x)$, where $\theta$ is an unknown ...
Michael's user avatar
  • 544
1 vote
0 answers
46 views

Equivalence of Graphical model selection algorithms

Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...
rajatsen91's user avatar
1 vote
0 answers
244 views

Distribution of a signal covariance matrix

A common estimation problem in signal processing assumes the following signal model \begin{equation} \mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n} \end{equation} where $\...
mermeladeK's user avatar
2 votes
1 answer
101 views

Estimating mean and variance of a distribution based on error-prone estimates of its cdf

Suppose I have some random variable $X$ taking values in $[a, b]$ with unknown distribution (I am happy to assume the distribution is smooth, though it would be nice to not have to). I have a ...
David R. MacIver's user avatar
4 votes
1 answer
288 views

Equivalent method for maximum likelihood estimation of covariance parameters

My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function: $$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...
Ruben van Bergen's user avatar
3 votes
2 answers
566 views

Moments of Matrix Gamma distribution

Matrix gamma distribution (defined for example in http://en.wikipedia.org/wiki/Matrix_gamma_distribution) is one way to generalize Wishart distribution. In our course work that distribution was used ...
o.p.'s user avatar
  • 31
1 vote
0 answers
186 views

Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts

I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...
user39080's user avatar
  • 203
1 vote
1 answer
282 views

Is an unbiased estimator with arbitrarily small variance necessarily consistent?

Given an unbiased estimator $\hat \theta_n$ of a parameter $\theta$, if the estimator has small variance (approaching $0$ as $n\to\infty$), it seems reasonable to expect that the estimator is ...
Federico's user avatar
  • 133
3 votes
1 answer
578 views

Why doesn't Stein effect happen for multinomial distributions?

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an ...
Yaroslav Bulatov's user avatar
20 votes
1 answer
4k views

Using Fisher Information to bound KL divergence

Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)? KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...
Yaroslav Bulatov's user avatar
3 votes
3 answers
2k views

Is the min function ever an unbiased estimator for the mean?

Given $n$ i.i.d. variables $X_1$ to $X_n$ with an unknown probability distribution, the sample average is an unbiased estimator for the mean of the distribution. Is there some non-trivial probability ...
Claudiu's user avatar
  • 597