All Questions
Tagged with estimate or estimation-theory
111 questions
1
vote
0
answers
62
views
Distances between up and down crosses in Gaussian Processes
Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$,
where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...
- 91
-2
votes
2
answers
477
views
Lower bound of q pochhammer symbol [closed]
How one could prove, that q pochhammer symbol $(1,1/n) = \prod_{k = 1}^{\infty}(1-\frac{1}{n^k}) \geq 1 - \frac{1}{n-1}$
1
vote
1
answer
170
views
Error metric for joint estimation of mean and variance
Background:
Let $\mu:\mathbb{R}^n\to\mathbb{R}$ and $\sigma:\mathbb{R}^n\to\mathbb{R}_+$ be two unknown functions, and consider a stochastic model of the form
$$
\mathbb{E}[Y\mid\mathbf{x}] = \mu(\...
- 617
5
votes
1
answer
107
views
Estimating the size of the remainder in a random partition
Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...
-2
votes
1
answer
92
views
Existence or impossibility of Gaussian factory
Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
0
votes
1
answer
51
views
Strict positive type function on hypersurface also of positive type in neighborhood?
Let $u\in C^\infty(\mathbb{R}^n\times\mathbb{R}^n)$ be symmetric and of strictly positive type on some hypersurface $S \subset \mathbb{R}^n$ diffeomorphic to $\{0\}\times\mathbb{R}^{n-1}$. This means ...
- 13
3
votes
1
answer
87
views
Optimal linear measurement operator
Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
- 6,541
4
votes
0
answers
715
views
Functional Taylor expansion for differential entropy
Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...
- 155
2
votes
1
answer
154
views
Finding a similarities and differences of sent of matrices
Suppose we have a set of rank deficient covariance matrices. How can I know the similarities and differences between those set of matrices?
Regards,
- 21
2
votes
1
answer
676
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
4
votes
1
answer
296
views
Proving bounds on analytic functions using only the Taylor expansion
I wonder if there is a general method for obtaining bounds on an analytic function using only its Taylor expansion (not using its special properties such as satisfying a good differential equation, ...
- 4,484
3
votes
1
answer
113
views
maximum likelihood estimation of X is better than that of f(X)?
Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...
- 482
2
votes
2
answers
632
views
An alternative proof of Bayesian Cramer-Rao
My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...
- 671
1
vote
0
answers
101
views
Extended Kalman filter for initial values estimation
I try to make extended Kalman filter for estimation of initial values of small celestial body. I have:
$(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values.
$z = ...
- 11
2
votes
0
answers
56
views
Rate of $L_1$ loss in estmating density on $[0,1]$
Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e.
$$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...
- 121
1
vote
1
answer
193
views
Fisher information with vanishing probability
I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as:
$$...
- 237
1
vote
0
answers
93
views
A different objective function in liner regression analysis
I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...
- 11
1
vote
1
answer
124
views
Reconstructing the number of distinct elements from a random projection
Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$.
We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...
- 618
1
vote
0
answers
79
views
sufficient statistics that are irrelevant
I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one.
If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...
- 129
1
vote
0
answers
49
views
A question about the prediction error
I am reading about the prediction error estimation and I found the following:
Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...
- 11
2
votes
0
answers
119
views
Calculate sample mean confidence interval of noisy logistical distribution
I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean.
I would like to estimate its average $u$ with a confidence interval (...
- 21
1
vote
0
answers
62
views
Cramer Rao bound for relative estimation
I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$).
A feature of our estimation problem is that the involved ...
- 21
3
votes
1
answer
96
views
Perturbation results for statistical estimators
Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...
- 4,049
3
votes
0
answers
82
views
Uniform mean-square-error estimates
Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...
- 111
4
votes
1
answer
203
views
Can samples be compressed?
The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is:
$\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...
- 22.1k
2
votes
0
answers
72
views
Robust weighted estimator of location
Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator.
Now let $W = (w_1, \ldots, ...
- 317
1
vote
0
answers
47
views
Bounding Hidden Markov model Bayesian filter error with inexact models
In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...
- 171
2
votes
0
answers
92
views
Elliptic Equation with Wentzell boundary condition
I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space.
The ...
- 133
2
votes
1
answer
443
views
Literature question on the convergence rate of the empirical distribution
Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...
user83947
2
votes
0
answers
187
views
Maximum-likelihood estimation for univariate responses from multivariate data
I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...
- 617
1
vote
1
answer
256
views
KL divergence Inequality
I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading.
Consider two probability measures $P$ and $Q$ both ...
- 275
2
votes
0
answers
193
views
How to find moment condition for generalized method of moments?
Consider a scalar system with $2K$ outputs and $K+2$ unknowns:
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $...
- 21
4
votes
1
answer
479
views
Cramér-Rao bound for randomized estimator
As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter.
Consider the case when the parameter is a scalar, the estimator is ...
- 337
2
votes
0
answers
51
views
MLE and CRLB with mismatched likelihoods
Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...
- 21
4
votes
2
answers
519
views
Cramér–Rao type bound for absolute estimation error
Let $\{X_1, X_2, \dotsc, X_n\}$ be independent and identically distributed (i.i.d.) random variables sampled from a common distribution with density $f_{\theta}(x)$, where $\theta$ is an unknown ...
- 544
1
vote
0
answers
46
views
Equivalence of Graphical model selection algorithms
Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...
- 275
1
vote
0
answers
244
views
Distribution of a signal covariance matrix
A common estimation problem in signal processing assumes the following signal model
\begin{equation}
\mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n}
\end{equation}
where $\...
- 345
2
votes
1
answer
101
views
Estimating mean and variance of a distribution based on error-prone estimates of its cdf
Suppose I have some random variable $X$ taking values in $[a, b]$ with unknown distribution (I am happy to assume the distribution is smooth, though it would be nice to not have to).
I have a ...
- 1,331
2
votes
1
answer
88
views
What is the problem with this model parameter estimation algorithm?
In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...
- 65
2
votes
3
answers
409
views
How to estimate the entropy of a distribution on a power set?
Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$.
Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...
- 165
2
votes
2
answers
174
views
estimating variance of dependent normal distributed data
Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that
$
\text{corr}(X_{ij},X_{rs})=\rho
...
- 133
4
votes
2
answers
135
views
Markov-type functions
I'd like to have some informations about Markov-type functions (or Cauchy-type):
\[ f(z)=\int_{\Gamma} \frac{\mathrm{d}\gamma(\xi)}{\xi-z}.\]
$\gamma$ is a positive measure with compact support $\...
- 41
4
votes
1
answer
288
views
Equivalent method for maximum likelihood estimation of covariance parameters
My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function:
$$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...
- 141
0
votes
1
answer
85
views
About the suboptimality of linear estimators
Let $X$ be a random variable and $N$ a Gaussian noise independent from $X$. We observe $Y=X+N$ and want to estimate $X$ based on $Y$ to minimize the mean square error $mmse(X|Y):=E(\hat X(Y)-X)^2$.
...
- 29
3
votes
2
answers
566
views
Moments of Matrix Gamma distribution
Matrix gamma distribution (defined for example in http://en.wikipedia.org/wiki/Matrix_gamma_distribution) is one way to generalize Wishart distribution. In our course work that distribution was used ...
- 31
1
vote
0
answers
186
views
Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts
I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...
- 203
5
votes
0
answers
190
views
Pair of two-variable polynomial equations of high order
I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N \...
- 548
0
votes
1
answer
112
views
Signal model classification between two possbile candidates
How to decide the most possible signal model between two model candidates besed on the received signal vector?
Assume the received signal vector is $y$, the possible signal model candidates could be:
...
- 31
2
votes
0
answers
243
views
Worst-case error and Cramer-Rao Lower Bound - is there any mathematical relation between them?
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...
- 959
2
votes
0
answers
1k
views
Definition and Convergence of Iteratively Reweighted Least Squares
I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form,
$J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$
where $N$ is the number of ...
- 121