# Questions tagged [estimation-theory]

The estimation-theory tag has no usage guidance.

**5**

votes

**1**answer

58 views

### Estimating the size of the remainder in a random partition

Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...

**-2**

votes

**1**answer

74 views

### Existence or impossibility of Gaussian factory

Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.

**0**

votes

**0**answers

60 views

### Nonparametric Bayesian estimation of several black-box functions of different variables from their noisy sums

In order to introduce my problem, let’s start with the nonparametric estimation of a single unknown/black-box function $f:{\Omega _f} \to \mathbb{R}$ of a discrete variable $x$ in a finite domain ${\...

**0**

votes

**1**answer

45 views

### Strict positive type function on hypersurface also of positive type in neighborhood?

Let $u\in C^\infty(\mathbb{R}^n\times\mathbb{R}^n)$ be symmetric and of strictly positive type on some hypersurface $S \subset \mathbb{R}^n$ diffeomorphic to $\{0\}\times\mathbb{R}^{n-1}$. This means ...

**0**

votes

**0**answers

26 views

### Efficiency of importance sampling in terms of the size of the the support of sampling distribution

In importance sampling, one proposes to compute an integral $I:=\mathbb E_{x \sim P}[h(x)]$ by rewritting it as
$$
I=\mathbb E_{x \sim Q}\left[w(x)h(x)\right],\text{ with }w(x):=\frac{p(x)}{q(x)},
$$
...

**3**

votes

**1**answer

70 views

### Optimal linear measurement operator

Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...

**1**

vote

**0**answers

142 views

### Functional Taylor expansion for differential entropy

Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...

**2**

votes

**1**answer

141 views

### Finding a similarities and differences of sent of matrices

Suppose we have a set of rank deficient covariance matrices. How can I know the similarities and differences between those set of matrices?
Regards,

**2**

votes

**1**answer

171 views

### Distribution of ratio between complex Gaussian and Chi-square R.V.s

What would be the distribution (p.d.f.) of the following ratio?
$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can be ...

**3**

votes

**1**answer

122 views

### Proving bounds on analytic functions using only the Taylor expansion

I wonder if there is a general method for obtaining bounds on an analytic function using only its Taylor expansion (not using its special properties such as satisfying a good differential equation, ...

**3**

votes

**1**answer

99 views

### maximum likelihood estimation of X is better than that of f(X)?

Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...

**2**

votes

**2**answers

273 views

### An alternative proof of Bayesian Cramer-Rao

My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...

**1**

vote

**0**answers

67 views

### Extended Kalman filter for initial values estimation

I try to make extended Kalman filter for estimation of initial values of small celestial body. I have:
$(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values.
$z = ...

**2**

votes

**0**answers

45 views

### Rate of $L_1$ loss in estmating density on $[0,1]$

Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e.
$$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...

**1**

vote

**1**answer

93 views

### Fisher information with vanishing probability

I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as:
$$...

**1**

vote

**0**answers

78 views

### A different objective function in liner regression analysis

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...

**1**

vote

**1**answer

113 views

### Reconstructing the number of distinct elements from a random projection

Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$.
We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...

**1**

vote

**0**answers

70 views

### sufficient statistics that are irrelevant

I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one.
If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...

**1**

vote

**0**answers

43 views

### A question about the prediction error

I am reading about the prediction error estimation and I found the following:
Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...

**2**

votes

**0**answers

40 views

### Calculate sample mean confidence interval of noisy logistical distribution

I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean.
I would like to estimate its average $u$ with a confidence interval (...

**1**

vote

**0**answers

52 views

### Cramer Rao bound for relative estimation

I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$).
A feature of our estimation problem is that the involved ...

**2**

votes

**0**answers

38 views

### Perturbation results for statistical estimators

Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...

**2**

votes

**0**answers

70 views

### Uniform mean-square-error estimates

Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...

**5**

votes

**1**answer

140 views

### Can samples be compressed?

The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is:
$\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...

**1**

vote

**0**answers

54 views

### Robust weighted estimator of location

Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator.
Now let $W = (w_1, \ldots, ...

**1**

vote

**0**answers

34 views

### Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...

**2**

votes

**0**answers

62 views

### Elliptic Equation with Wentzell boundary condition

I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space.
The ...

**2**

votes

**1**answer

144 views

### Literature question on the convergence rate of the empirical distribution

Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...

**2**

votes

**0**answers

135 views

### Maximum-likelihood estimation for univariate responses from multivariate data

I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...

**1**

vote

**1**answer

209 views

### KL divergence Inequality

I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading.
Consider two probability measures $P$ and $Q$ both ...

**2**

votes

**0**answers

119 views

### How to find moment condition for generalized method of moments?

Consider a scalar system with $2K$ outputs and $K+2$ unknowns:
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $...

**4**

votes

**1**answer

169 views

### Cramér-Rao bound for randomized estimator

As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter.
Consider the case when the parameter is a scalar, the estimator is ...

**2**

votes

**0**answers

45 views

### MLE and CRLB with mismatched likelihoods

Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...

**4**

votes

**1**answer

290 views

### Cramer-Rao type bound for absolute estimation error

Let $\{X_1, X_2, \ldots, X_n\}$ be independent and identically distributed (i.i.d.) random variables sampled from a common distribution with density $f_{\theta}(x)$, where $\theta$ is an unknown ...

**1**

vote

**0**answers

41 views

### Equivalence of Graphical model selection algorithms

Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...

**1**

vote

**0**answers

210 views

### Distribution of a signal covariance matrix

A common estimation problem in signal processing assumes the following signal model
\begin{equation}
\mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n}
\end{equation}
where $\...

**2**

votes

**1**answer

82 views

### Estimating mean and variance of a distribution based on error-prone estimates of its cdf

Suppose I have some random variable $X$ taking values in $[a, b]$ with unknown distribution (I am happy to assume the distribution is smooth, though it would be nice to not have to).
I have a ...

**2**

votes

**1**answer

79 views

### What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...

**2**

votes

**3**answers

222 views

### How to estimate the entropy of a distribution on a power set?

Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$.
Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...

**2**

votes

**2**answers

169 views

### estimating variance of dependent normal distributed data

Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that
$
\text{corr}(X_{ij},X_{rs})=\rho
...

**3**

votes

**2**answers

115 views

### Markov-type functions

I'd like to have some informations about Markov-type functions (or Cauchy-type):
\[ f(z)=\int_{\Gamma} \frac{\mathrm{d}\gamma(\xi)}{\xi-z}.\]
$\gamma$ is a positive measure with compact support $\...

**1**

vote

**0**answers

72 views

### Stochastic process inference from partial observations

Consider a set $U$. My signal is a piece-wise constant "function"
$Sig: t \mapsto s$, i.e. the signal at time $t$ equals to some subset
$s \subset U$. One can see $Sig(t)$ as a stochastic process.
...

**3**

votes

**1**answer

220 views

### Equivalent method for maximum likelihood estimation of covariance parameters

My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function:
$$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...

**0**

votes

**1**answer

74 views

### About the suboptimality of linear estimators

Let $X$ be a random variable and $N$ a Gaussian noise independent from $X$. We observe $Y=X+N$ and want to estimate $X$ based on $Y$ to minimize the mean square error $mmse(X|Y):=E(\hat X(Y)-X)^2$.
...

**3**

votes

**2**answers

375 views

### Moments of Matrix Gamma distribution

Matrix gamma distribution (defined for example in http://en.wikipedia.org/wiki/Matrix_gamma_distribution) is one way to generalize Wishart distribution. In our course work that distribution was used ...

**1**

vote

**0**answers

147 views

### Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts

I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...

**4**

votes

**0**answers

177 views

### Pair of two-variable polynomial equations of high order

I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N \...

**0**

votes

**1**answer

109 views

### Signal model classification between two possbile candidates

How to decide the most possible signal model between two model candidates besed on the received signal vector?
Assume the received signal vector is $y$, the possible signal model candidates could be:
...

**2**

votes

**0**answers

202 views

### Worst-case error and Cramer-Rao Lower Bound - is there any mathematical relation between them?

I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...

**2**

votes

**0**answers

789 views

### Definition and Convergence of Iteratively Reweighted Least Squares

I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form,
$J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$
where $N$ is the number of ...