# Questions tagged [estimation-theory]

The estimation-theory tag has no usage guidance.

78
questions

**2**

votes

**1**answer

58 views

### Spline Interpolation error of higher degree

It is well-known that the interpolation error of a cubic spline has at best order $O(h^4)$, which results from polynomials of degree $3$.
Can I assume that, if one uses polynomials of degree $p$ and ...

**1**

vote

**0**answers

37 views

### Percentile interval Lemma

Let $\theta$ be a parameter and $\hat{\theta}$ the plug-in estimate, I need a proof of the following lemma, as given in [1], p. 173, in the form of a reference or of a direct argument:
Percentile ...

**1**

vote

**0**answers

41 views

### Bootstrap-$t$ confidence intervals

I'm writing a dissertation about bootstrap methods and the main book I'm using is Efron, B., & Tibshirani, R.J. (1994), An Introduction to the Bootstrap (1st ed.), Chapman and Hall/CRC. Now I need ...

**0**

votes

**0**answers

26 views

### Non-parametric estimator of continuous process's law from one sample path?

Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...

**2**

votes

**1**answer

123 views

### Stability estimates on quotients of the form $ \frac{\prod_{j=1}^n a_j}{\prod_{j=1}^n b_j} $

Suppose that $a_j,b_j \in \mathbb C$ are complex numbers, $j=1,\dots,n$, with the property that $|a_j|,|b_j| \geq c > d >0$ where $c,d$ are positive real numbers. I'm interested in the stability ...

**1**

vote

**0**answers

33 views

### The optimality of Kalman filtering

It is known that the Kalman filter estimates the state of the following system recursively.
$$x_{k+1}=Ax_k+w_k, \ \ w_k \sim \mathcal{N}(0,Q)$$
$$y_k=Cx_k+v_k, \ \ v_k \sim \mathcal{N}(0,W)$$
In the ...

**3**

votes

**1**answer

111 views

### Design a random variable which has the maximal correlation with another random variable

$Y$ is a Gaussian distributed random variable with zero mean and known variance: $Y\sim N(0,\sigma_y)$. We measure $Y$ with a sensor, which is corrupted by white Gaussian noise: $Z=Y+V$; $V\sim N(0,\...

**6**

votes

**3**answers

458 views

### How to estimate the integral involving the distance function

Let $\Omega\subset\mathbb{R}^n$ be an open bounded domain with smooth boundary. Consider the following integral:
$$I(t)=\int_{\Omega}e^{-\frac{d^2(y,\partial\Omega)}{t}}{\rm d}y.$$
My problem is how ...

**-1**

votes

**1**answer

68 views

### How to combine estimator with different variances?

Consider independent random variables $X_1,X_2,\ldots,$ that have the same expectation $\mathbb x=\mathbb E[X_1]=\mathbb E[X_2]=\ldots$
Further, assume that we know that $Var[X_i]=\sigma_i^2$.
In the ...

**0**

votes

**0**answers

104 views

### A method for extracting a condition to check whether a feature is related to an object

Let we have the object $\bf S$. This object has some properties such as length, temperature and other features. Assume that for the object $\bf S$ we selected $n$ features.For example the vector
${\bf ...

**0**

votes

**0**answers

60 views

### Anti-concentration of measure: Slud's inequality for finite populations

Suppose that I have Bernoulli trials with unknown bias $p$. I need $\Omega(\frac{\log 1/\delta}{\epsilon^2})$ samples for the average of the samples to estimate $p$ within $\epsilon$ error with ...

**1**

vote

**0**answers

49 views

### How to retrieve back the input using Bussgang theorem?

If we have a non-linear function $f$, that is applied to input $x$,
we have then the output $y=f(x)$
Using Bussgang decomposition we can linearize this nonlinearity and express $y$ as
$y=Bx+ η$,
...

**6**

votes

**1**answer

287 views

### Probability of complex eigenvalues

I find this is the best site to post this question, even though I considered cs.
It is a Monte Carlo experiment over the set of 10.000 n×n matrices.
If a single matrix eigenvalue is complex then ...

**3**

votes

**0**answers

80 views

### Proving the exponential decay of Green's function for the lattice $-\Delta+p$

The Green function $G(x,y) =G(x-y)$ of the discrete Klein-Gordon operator $-\Delta+p$ on $\mathbb{Z}^{d}$ is given by:
\begin{eqnarray}
G(x-y) = \int_{[-\pi,\pi]^{d}}\frac{d^{d}k}{(2\pi)^{d}}\frac{e^{...

**1**

vote

**0**answers

33 views

### Estimation of parameters through multivariate Taylor expansion?

I do have a function $$f(t) = \prod\limits_{j=1}^{n} \left(1 + \sum\limits_{i=1}^{n} M_{i,j} t_i\right)^{-\alpha_{j}}$$ defined by parameters:
$M_{i,j} \in \mathbb{R}_{+}, \;\forall i \in 1,...,d,\; ...

**-1**

votes

**1**answer

116 views

### Sufficient conditions on $ a_i,b_i$ for $a_1\phi(n)+b_1, \cdots, a_k\phi(n)+b_k$ to be simultaneously prime infinitely often?

I am really interested in sufficient conditions on $a_i, b_i$ guaranteeing that the linear forms $a_1\phi(n)+b_1,\dots, a_k\phi(n)+b_k$ become simultaneously prime for infinitely many positive ...

**2**

votes

**0**answers

129 views

### Extended Kalman Filter and its State Transition Matrix

Sorry for what might be a long post, I want to give background.
Initially I had regular Kalman filter, and the state model was defined by Newtonian kinematics, with initial position 0 and speed of 2. ...

**1**

vote

**0**answers

56 views

### Distances between up and down crosses in Gaussian Processes

Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$,
where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...

**-2**

votes

**2**answers

186 views

### Lower bound of q pochhammer symbol [closed]

How one could prove, that q pochhammer symbol $(1,1/n) = \prod_{k = 1}^{\infty}(1-\frac{1}{n^k}) \geq 1 - \frac{1}{n-1}$

**0**

votes

**1**answer

84 views

### Error metric for joint estimation of mean and variance

Background:
Let $\mu:\mathbb{R}^n\to\mathbb{R}$ and $\sigma:\mathbb{R}^n\to\mathbb{R}_+$ be two unknown functions, and consider a stochastic model of the form
$$
\mathbb{E}[Y|\mathbf{x}] = \mu(\...

**5**

votes

**1**answer

80 views

### Estimating the size of the remainder in a random partition

Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...

**-2**

votes

**1**answer

79 views

### Existence or impossibility of Gaussian factory

Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.

**0**

votes

**1**answer

46 views

### Strict positive type function on hypersurface also of positive type in neighborhood?

Let $u\in C^\infty(\mathbb{R}^n\times\mathbb{R}^n)$ be symmetric and of strictly positive type on some hypersurface $S \subset \mathbb{R}^n$ diffeomorphic to $\{0\}\times\mathbb{R}^{n-1}$. This means ...

**3**

votes

**1**answer

74 views

### Optimal linear measurement operator

Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...

**2**

votes

**0**answers

369 views

### Functional Taylor expansion for differential entropy

Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...

**2**

votes

**1**answer

144 views

### Finding a similarities and differences of sent of matrices

Suppose we have a set of rank deficient covariance matrices. How can I know the similarities and differences between those set of matrices?
Regards,

**2**

votes

**1**answer

393 views

### Distribution of ratio between complex Gaussian and Chi-square R.V.s

What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...

**3**

votes

**1**answer

150 views

### Proving bounds on analytic functions using only the Taylor expansion

I wonder if there is a general method for obtaining bounds on an analytic function using only its Taylor expansion (not using its special properties such as satisfying a good differential equation, ...

**3**

votes

**1**answer

103 views

### maximum likelihood estimation of X is better than that of f(X)?

Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...

**2**

votes

**2**answers

419 views

### An alternative proof of Bayesian Cramer-Rao

My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...

**1**

vote

**0**answers

85 views

### Extended Kalman filter for initial values estimation

I try to make extended Kalman filter for estimation of initial values of small celestial body. I have:
$(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values.
$z = ...

**2**

votes

**0**answers

51 views

### Rate of $L_1$ loss in estmating density on $[0,1]$

Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e.
$$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...

**1**

vote

**1**answer

129 views

### Fisher information with vanishing probability

I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as:
$$...

**1**

vote

**0**answers

88 views

### A different objective function in liner regression analysis

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...

**1**

vote

**1**answer

116 views

### Reconstructing the number of distinct elements from a random projection

Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$.
We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...

**1**

vote

**0**answers

73 views

### sufficient statistics that are irrelevant

I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one.
If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...

**1**

vote

**0**answers

44 views

### A question about the prediction error

I am reading about the prediction error estimation and I found the following:
Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...

**2**

votes

**0**answers

86 views

### Calculate sample mean confidence interval of noisy logistical distribution

I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean.
I would like to estimate its average $u$ with a confidence interval (...

**1**

vote

**0**answers

53 views

### Cramer Rao bound for relative estimation

I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$).
A feature of our estimation problem is that the involved ...

**2**

votes

**0**answers

44 views

### Perturbation results for statistical estimators

Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...

**3**

votes

**0**answers

77 views

### Uniform mean-square-error estimates

Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...

**5**

votes

**1**answer

170 views

### Can samples be compressed?

The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is:
$\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...

**1**

vote

**0**answers

58 views

### Robust weighted estimator of location

Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator.
Now let $W = (w_1, \ldots, ...

**1**

vote

**0**answers

35 views

### Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...

**2**

votes

**0**answers

66 views

### Elliptic Equation with Wentzell boundary condition

I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space.
The ...

**2**

votes

**1**answer

241 views

### Literature question on the convergence rate of the empirical distribution

Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...

**2**

votes

**0**answers

171 views

### Maximum-likelihood estimation for univariate responses from multivariate data

I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...

**1**

vote

**1**answer

234 views

### KL divergence Inequality

I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading.
Consider two probability measures $P$ and $Q$ both ...

**2**

votes

**0**answers

140 views

### How to find moment condition for generalized method of moments?

Consider a scalar system with $2K$ outputs and $K+2$ unknowns:
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $...

**4**

votes

**1**answer

271 views

### Cramér-Rao bound for randomized estimator

As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter.
Consider the case when the parameter is a scalar, the estimator is ...