All Questions
Tagged with estimation-theory estimation-theory or
53 questions with no upvoted or accepted answers
5
votes
0
answers
190
views
Pair of two-variable polynomial equations of high order
I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N \...
- 548
4
votes
0
answers
715
views
Functional Taylor expansion for differential entropy
Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...
- 155
3
votes
0
answers
113
views
Is the Kalman Filter computationally optimal for Kalman filtering?
Kalman filtering is known to be a recursive process that minimizes mean square error in linear problems.
My question is: has anybody shown that this algorithm is computationally optimal, i.e. that you ...
- 33
3
votes
0
answers
113
views
Image restoration quality general lower bounds
A typical image restoration model posits that, starting from a true image $f = f(x,y)$, we observe
$$
\tilde f = f \star h + n
$$
where $\star$ is convolution, $h$ is the point spread function (caused,...
- 1,069
3
votes
0
answers
265
views
Proving the exponential decay of Green's function for the lattice $-\Delta+p$
The Green function $G(x,y) =G(x-y)$ of the discrete Klein-Gordon operator $-\Delta+p$ on $\mathbb{Z}^{d}$ is given by:
\begin{eqnarray}
G(x-y) = \int_{[-\pi,\pi]^{d}}\frac{d^{d}k}{(2\pi)^{d}}\frac{e^{...
- 1,305
3
votes
0
answers
82
views
Uniform mean-square-error estimates
Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...
- 111
3
votes
0
answers
975
views
Interpolating Wavelet Coefficients
Hi! I was instructed via reddit that this place would be the best place to post this question. Fingers cross you can help...
Ive been writing some code to get rid of noise "spikes" in a signal. I'm ...
- 31
2
votes
0
answers
87
views
A complex problem involving densities (likelihood functions) and optimization
Consider the following autoregressive process with normal errors:
\begin{equation}\label{7YlUV4i8nuO}\tag{I}
y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2)
\end{equation}
We ...
- 13
2
votes
0
answers
122
views
Comparing the truncated $\ell^{1}$-norm of polynomial coefficients with the supremum norm on the unit disc
Let $p=a_{0}+a_{1}z+\ldots+a_{n}z^{n}$ be a polynomial. Consider the following truncated $\ell^{1}$-seminorm of the coefficients of $p$:
$$\|p\|_{\ell^{1},\text{trun.}}:=\sum_{k=1}^{n}|a_{k}|=\|p-a_{0}...
- 321
2
votes
0
answers
225
views
Sobolev (Triebel-Lizorkin) norm estimate for $F \circ u - F \circ v$
Let $F \in C^1(\mathbb R^d;\mathbb R)$ be such that $F(0) = 0$ and
$$|F'(\tau v + (1 - \tau)w)| \leq \mu(\tau)(G(v) + G(w))$$
for some $\mu \in L^1([0,1])$ and some non-negative $G \in C^0(\mathbb R^d;...
- 233
2
votes
0
answers
78
views
Distribution of unbiased estimator of covariance matrix with missing values
Initial setup
Assuming $X_1, ..., X_n \in \mathbb{R}^m$ are iid, sampled from $\mathcal{N}(\mu, V)$, one can define the estimators for the sample mean $\hat{\mu} = \frac{1}{n} := X^T 1_n$, and sample ...
- 255
2
votes
0
answers
130
views
L1 error of estimators
I came across the following problem and I have no clue how to approach it. I am looking for help with directions or references.
Consider the $\alpha$-stable distribution with unknown true mean $\mu$, ...
- 173
2
votes
0
answers
139
views
'Contraction-like' inequality: how to deal with the boundary term?
I am interested in the following problem.
Let $D = \mathrm{diag}(d_1, d_2, \ldots, d_n) \in \mathbb{R}^n$ be positive definite, let $B, K \in \mathbb{R}^n$, and let $G\in L^\infty((0, T)\times (0, L);...
- 115
2
votes
0
answers
90
views
The optimality of Kalman filtering
It is known that the Kalman filter estimates the state of the following system recursively.
$$x_{k+1}=Ax_k+w_k, \ \ w_k \sim \mathcal{N}(0,Q)$$
$$y_k=Cx_k+v_k, \ \ v_k \sim \mathcal{N}(0,W)$$
In the ...
- 61
2
votes
0
answers
385
views
Extended Kalman Filter and its State Transition Matrix
Sorry for what might be a long post, I want to give background.
Initially I had regular Kalman filter, and the state model was defined by Newtonian kinematics, with initial position 0 and speed of 2. ...
2
votes
0
answers
56
views
Rate of $L_1$ loss in estmating density on $[0,1]$
Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e.
$$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...
- 121
2
votes
0
answers
119
views
Calculate sample mean confidence interval of noisy logistical distribution
I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean.
I would like to estimate its average $u$ with a confidence interval (...
- 21
2
votes
0
answers
72
views
Robust weighted estimator of location
Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator.
Now let $W = (w_1, \ldots, ...
- 317
2
votes
0
answers
92
views
Elliptic Equation with Wentzell boundary condition
I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space.
The ...
- 133
2
votes
0
answers
187
views
Maximum-likelihood estimation for univariate responses from multivariate data
I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...
- 617
2
votes
0
answers
193
views
How to find moment condition for generalized method of moments?
Consider a scalar system with $2K$ outputs and $K+2$ unknowns:
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $...
- 21
2
votes
0
answers
51
views
MLE and CRLB with mismatched likelihoods
Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...
- 21
2
votes
0
answers
243
views
Worst-case error and Cramer-Rao Lower Bound - is there any mathematical relation between them?
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...
- 959
2
votes
0
answers
1k
views
Definition and Convergence of Iteratively Reweighted Least Squares
I've been using iteratively reweighted least squares (IRLS) to minimize functions of the following form,
$J(m) = \sum_{i=1}^{N} \rho \left(\left| x_i - m \right|\right)$
where $N$ is the number of ...
- 121
1
vote
0
answers
148
views
conjecture for general form of minimax estimator
I had previously posed an overly ambitious version of this conjecture here,
Form of minimax estimator,
which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
- 6,541
1
vote
0
answers
105
views
Estimate for the gradient of solutions in an elliptic differential equation in a Sobolev space
Let $\Omega$ be a bounded or unbounded domain in $\mathbf R^{3}$ with a smooth boundary $S$ and a normal vector given by $n$. Now, we consider the following second-order elliptic problem with Neumann ...
1
vote
0
answers
66
views
Parameter estimation of a Taylor expansion
Let $a,b$ two real numbers, $\theta$ a real parameter and suppose that you have an analytic function of the form:
$$
f_\theta(x)\triangleq \sum_{k\in\mathbb{N}}a_k(\theta)x^k \quad\forall x\in[a,b],
$$...
- 393
1
vote
0
answers
59
views
Functional approximation with derivatives
I am trying to solve a functional approximation problem.
Consider a set of measurements of a d-dimensional state $\mathrm x \in \mathbb{R}^d$, together with velocities $\dot{\mathrm x}$ and ...
1
vote
0
answers
34
views
Correlating two matrices $A,B$ with stochastic dependency structure imposed by cross-validation
Consider a labelled data set
$$D = \{(x_1, y_1),...,(x_n, y_n)\} $$
on which we want to evaluate a machine learning algorithm using $k$-fold cross validation with $m$ different random seeds. This ...
- 153
1
vote
0
answers
81
views
Calculating the mean squared error for an estimate of a large sum
Consider the set of all Boolean function $f: \{0, 1\}^{n} \rightarrow \{-1, 1\}$. Now, let's pick a function uniformly at random from this set. Let $F$ be the random variable corresponding to the ...
1
vote
0
answers
75
views
Percentile interval Lemma
Let $\theta$ be a parameter and $\hat{\theta}$ the plug-in estimate, I need a proof of the following lemma, as given in [1], p. 173, in the form of a reference or of a direct argument:
Percentile ...
1
vote
0
answers
108
views
Bootstrap-$t$ confidence intervals
I'm writing a dissertation about bootstrap methods and the main book I'm using is Efron, B., & Tibshirani, R.J. (1994), An Introduction to the Bootstrap (1st ed.), Chapman and Hall/CRC. Now I need ...
1
vote
0
answers
154
views
How to retrieve back the input using Bussgang theorem?
If we have a non-linear function $f$, that is applied to input $x$,
we have then the output $y=f(x)$
Using Bussgang decomposition we can linearize this nonlinearity and express $y$ as
$y=Bx+ η$,
...
- 39
1
vote
0
answers
56
views
Estimation of parameters through multivariate Taylor expansion?
I do have a function $$f(t) = \prod\limits_{j=1}^{n} \left(1 + \sum\limits_{i=1}^{n} M_{i,j} t_i\right)^{-\alpha_{j}}$$ defined by parameters:
$M_{i,j} \in \mathbb{R}_{+}, \;\forall i \in 1,...,d,\; ...
- 686
1
vote
0
answers
62
views
Distances between up and down crosses in Gaussian Processes
Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$,
where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...
- 91
1
vote
0
answers
101
views
Extended Kalman filter for initial values estimation
I try to make extended Kalman filter for estimation of initial values of small celestial body. I have:
$(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values.
$z = ...
- 11
1
vote
0
answers
93
views
A different objective function in liner regression analysis
I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...
- 11
1
vote
0
answers
79
views
sufficient statistics that are irrelevant
I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one.
If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...
- 129
1
vote
0
answers
49
views
A question about the prediction error
I am reading about the prediction error estimation and I found the following:
Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...
- 11
1
vote
0
answers
62
views
Cramer Rao bound for relative estimation
I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$).
A feature of our estimation problem is that the involved ...
- 21
1
vote
0
answers
47
views
Bounding Hidden Markov model Bayesian filter error with inexact models
In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...
- 171
1
vote
0
answers
46
views
Equivalence of Graphical model selection algorithms
Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...
- 275
1
vote
0
answers
244
views
Distribution of a signal covariance matrix
A common estimation problem in signal processing assumes the following signal model
\begin{equation}
\mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n}
\end{equation}
where $\...
- 345
1
vote
0
answers
186
views
Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts
I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...
- 203
1
vote
0
answers
98
views
Estimation of X in Gaussian noise
Given
$\textbf{x}=[x_1 x_2 ... x_n]^T$ where $\textbf{x} \in \{ 0, a_1, a_2, a_3\}^n, a_i \in \mathbb{C}$ and $\textbf{z} = \{z_1 z_2,...,z_n \}$ where $z_i \textbf{~} N(0,\sigma^2)$ is a Complex ...
- 11
0
votes
0
answers
42
views
An integral estimate involving Bergman kernel
Let $V$ be the normalized volume measure on $\mathbb D^2$ and $k : \mathbb D \times \mathbb D \longrightarrow \mathbb C$ be the Bergman kernel on $\mathbb D^2$ given by $$k(z,w) = \frac {1} {\left (1 -...
- 41
0
votes
0
answers
31
views
What is the Fisher information matrix of the von Mises-Fisher distribution?
Assuming the von Mises-Fisher distribution as
$$f_{p}(\mathbf{x}; \boldsymbol{\mu}, \kappa) = C_{p}(\kappa) \exp \left( {\kappa \boldsymbol{\mu}^\mathsf{T} \mathbf{x} } \right),$$
where $\kappa \ge 0$,...
- 287
0
votes
0
answers
51
views
Estimation of Nonzero Coefficients of Binary Cylotomic Polynomials
I am reading Fouvry's paper https://msp.org/ant/2013/7-5/ant-v7-n5-p07-p.pdf . I am still confused on section 4.2 why $P\leq x^{\frac{20}{9}\gamma -\frac{2}{3}}\mathcal{L}^{-16}$ leads to estimate in ...
- 271
0
votes
0
answers
21
views
Estimatives for elliptic systems involving the laplacian
Considering the problem
\begin{equation}
\left\{
\begin{array}[c]{11}
\Delta(\Delta \chi -\chi) = 0 & \text{in } \Omega, \\
\Delta \chi -\chi = h_2 - h_1, & \text{on } \partial\Omega \\
\end{...
- 101
0
votes
0
answers
106
views
Maximum likelihood estimator for power law with negative exponent
Background
I have data that roughly follows a power law with a negative exponent (up to a point; also, the parameters of the "fit" were just guesstimated by eye as a demonstration):
Now I ...
- 101