The estimation-theory tag has no usage guidance.

**2**

votes

**0**answers

44 views

### Maximum-likelihood estimation for univariate responses from multivariate data

I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n ...

**1**

vote

**1**answer

82 views

### KL divergence Inequality

I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading.
Consider two probability measures $P$ and $Q$ both ...

**1**

vote

**0**answers

36 views

### Find Moment condition for generalized method of moments

Consider a scalar system with 2K outputs and K+2 unknowns
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $a_2$, ...

**3**

votes

**1**answer

61 views

### Cramer-Rao bound for randomized estimator

As is well known, the Cramer-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter.
Consider the case when the parameter is a scalar, the estimator is ...

**2**

votes

**0**answers

32 views

### MLE and CRLB with mismatched likelihoods

Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...

**4**

votes

**1**answer

108 views

### Cramer-Rao type bound for absolute estimation error

Let $\{X_1, X_2, \ldots, X_n\}$ be independent and identically distributed (i.i.d.) random variables sampled from a common distribution with density $f_{\theta}(x)$, where $\theta$ is an unknown ...

**1**

vote

**0**answers

33 views

### Equivalence of Graphical model selection algorithms

Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...

**0**

votes

**0**answers

32 views

### Non-coherent estimation problem

I have the following signals
$$\left[\begin{array}{c} y_{mn} \\ y_{nm}\end{array}\right] =\left[\begin{array}{c} x_{n} \\ x_{m}\end{array}\right]h_{nm} +\left[\begin{array}{c} e_{mn} \\ ...

**1**

vote

**0**answers

82 views

### Distribution of a signal covariance matrix

A common estimation problem in signal processing assumes the following signal model
\begin{equation}
\mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n}
\end{equation}
where ...

**2**

votes

**1**answer

73 views

### Estimating mean and variance of a distribution based on error-prone estimates of its cdf

Suppose I have some random variable $X$ taking values in $[a, b]$ with unknown distribution (I am happy to assume the distribution is smooth, though it would be nice to not have to).
I have a ...

**2**

votes

**1**answer

70 views

### What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...

**1**

vote

**3**answers

147 views

### How to estimate the entropy of a distribution on a power set?

Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$.
Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...

**2**

votes

**2**answers

118 views

### estimating variance of dependent normal distributed data

Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that
$
\text{corr}(X_{ij},X_{rs})=\rho
...

**2**

votes

**2**answers

76 views

### Markov-type functions

I'd like to have some informations about Markov-type functions (or Cauchy-type):
\[ f(z)=\int_{\Gamma} \frac{\mathrm{d}\gamma(\xi)}{\xi-z}.\]
$\gamma$ is a positive measure with compact support ...

**1**

vote

**0**answers

64 views

### Stochastic process inference from partial observations

Consider a set $U$. My signal is a piece-wise constant "function"
$Sig: t \mapsto s$, i.e. the signal at time $t$ equals to some subset
$s \subset U$. One can see $Sig(t)$ as a stochastic process.
...

**3**

votes

**1**answer

160 views

### Equivalent method for maximum likelihood estimation of covariance parameters

My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function:
$$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...

**0**

votes

**1**answer

68 views

### About the suboptimality of linear estimators

Let $X$ be a random variable and $N$ a Gaussian noise independent from $X$. We observe $Y=X+N$ and want to estimate $X$ based on $Y$ to minimize the mean square error $mmse(X|Y):=E(\hat X(Y)-X)^2$.
...

**1**

vote

**0**answers

108 views

### Moments of Matrix Gamma distribution

Matrix gamma distribution (defined for example in http://en.wikipedia.org/wiki/Matrix_gamma_distribution) is one way to generalize Wishart distribution. In our course work that distribution was used ...

**1**

vote

**0**answers

114 views

### Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts

I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...

**4**

votes

**0**answers

169 views

### Pair of two-variable polynomial equations of high order

I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N ...

**0**

votes

**1**answer

107 views

### Signal model classification between two possbile candidates

How to decide the most possible signal model between two model candidates besed on the received signal vector?
Assume the received signal vector is $y$, the possible signal model candidates could be:
...

**2**

votes

**0**answers

158 views

### Worst-case error and Cramer-Rao Lower Bound - is there any mathematical relation between them?

I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...

**2**

votes

**0**answers

543 views

### Definition and Convergence of Iteratively Reweighted Least Squares

I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form,
$J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$
where $N$ is the number of ...

**0**

votes

**1**answer

314 views

### Estimation of Temporal Correlation of Signal

I have a signal and i'd like to estimate its temporal correlation.
My limited understanding is i should compute the PSD by estimation using a parametric model such as AR.
However, i'm not quite ...

**1**

vote

**1**answer

200 views

### Is an unbiased estimator with arbitrarily small variance necessarily consistent?

Given an unbiased estimator $\hat \theta_n$ of a parameter $\theta$, if the estimator has small variance (approaching $0$ as $n\to\infty$), it seems reasonable to expect that the estimator is ...

**0**

votes

**2**answers

438 views

### Estimating a sum

Sorry for the vague title but I couldn't find a better one.
I want to compute the sum $S = \frac{1}{N}\sum_{i=1}^N c_i x_i$ where $c_i$s are known positive constants. The problem is that computing ...

**0**

votes

**0**answers

79 views

### Estimation of X in Gaussian noise

Given
$\textbf{x}=[x_1 x_2 ... x_n]^T$ where $\textbf{x} \in \{ 0, a_1, a_2, a_3\}^n, a_i \in \mathbb{C}$ and $\textbf{z} = \{z_1 z_2,...,z_n \}$ where $z_i \textbf{~} N(0,\sigma^2)$ is a Complex ...

**0**

votes

**0**answers

133 views

### Why does the OLS estimator simplify as follows for the single regressor case?

I was reading in "A Guide to Econometrics" that given $Y = X \beta + \epsilon$, the variance covariance matrix of $\beta^\text{OLS}$ is given by $\sigma^2 (X' X)^{-1}$ where $\sigma^2$ is the variance ...

**2**

votes

**2**answers

4k views

### Maximum likelihood estimator for Power-law with Exponential cutoff

Hi,
for fitting empirical data to power-law I am aware of the work by Clauset et al. (http://arxiv.org/abs/0706.1062) and how to use maximum likelihood estimation. There exists also a simple maximum ...

**3**

votes

**0**answers

710 views

### Interpolating Wavelet Coefficients

Hi! I was instructed via reddit that this place would be the best place to post this question. Fingers cross you can help...
Ive been writing some code to get rid of noise "spikes" in a signal. I'm ...

**2**

votes

**1**answer

407 views

### Why doesn't Stein effect happen for multinomial distributions?

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an ...

**6**

votes

**0**answers

1k views

### Using Fisher Information to bound KL divergence

Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...

**1**

vote

**1**answer

269 views

### Is there a text on estimation theory online?

Where can I find graduate level, thorough, parameter estimation/ estimation theory material on the web?