Questions tagged [estimation-theory]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
1answer
58 views

Spline Interpolation error of higher degree

It is well-known that the interpolation error of a cubic spline has at best order $O(h^4)$, which results from polynomials of degree $3$. Can I assume that, if one uses polynomials of degree $p$ and ...
1
vote
0answers
37 views

Percentile interval Lemma

Let $\theta$ be a parameter and $\hat{\theta}$ the plug-in estimate, I need a proof of the following lemma, as given in [1], p. 173, in the form of a reference or of a direct argument: Percentile ...
1
vote
0answers
41 views

Bootstrap-$t$ confidence intervals

I'm writing a dissertation about bootstrap methods and the main book I'm using is Efron, B., & Tibshirani, R.J. (1994), An Introduction to the Bootstrap (1st ed.), Chapman and Hall/CRC. Now I need ...
0
votes
0answers
26 views

Non-parametric estimator of continuous process's law from one sample path?

Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
2
votes
1answer
123 views

Stability estimates on quotients of the form $ \frac{\prod_{j=1}^n a_j}{\prod_{j=1}^n b_j} $

Suppose that $a_j,b_j \in \mathbb C$ are complex numbers, $j=1,\dots,n$, with the property that $|a_j|,|b_j| \geq c > d >0$ where $c,d$ are positive real numbers. I'm interested in the stability ...
1
vote
0answers
33 views

The optimality of Kalman filtering

It is known that the Kalman filter estimates the state of the following system recursively. $$x_{k+1}=Ax_k+w_k, \ \ w_k \sim \mathcal{N}(0,Q)$$ $$y_k=Cx_k+v_k, \ \ v_k \sim \mathcal{N}(0,W)$$ In the ...
3
votes
1answer
111 views

Design a random variable which has the maximal correlation with another random variable

$Y$ is a Gaussian distributed random variable with zero mean and known variance: $Y\sim N(0,\sigma_y)$. We measure $Y$ with a sensor, which is corrupted by white Gaussian noise: $Z=Y+V$; $V\sim N(0,\...
6
votes
3answers
458 views

How to estimate the integral involving the distance function

Let $\Omega\subset\mathbb{R}^n$ be an open bounded domain with smooth boundary. Consider the following integral: $$I(t)=\int_{\Omega}e^{-\frac{d^2(y,\partial\Omega)}{t}}{\rm d}y.$$ My problem is how ...
-1
votes
1answer
68 views

How to combine estimator with different variances?

Consider independent random variables $X_1,X_2,\ldots,$ that have the same expectation $\mathbb x=\mathbb E[X_1]=\mathbb E[X_2]=\ldots$ Further, assume that we know that $Var[X_i]=\sigma_i^2$. In the ...
0
votes
0answers
104 views

A method for extracting a condition to check whether a feature is related to an object

Let we have the object $\bf S$. This object has some properties such as length, temperature and other features. Assume that for the object $\bf S$ we selected $n$ features.For example the vector ${\bf ...
0
votes
0answers
60 views

Anti-concentration of measure: Slud's inequality for finite populations

Suppose that I have Bernoulli trials with unknown bias $p$. I need $\Omega(\frac{\log 1/\delta}{\epsilon^2})$ samples for the average of the samples to estimate $p$ within $\epsilon$ error with ...
1
vote
0answers
49 views

How to retrieve back the input using Bussgang theorem?

If we have a non-linear function $f$, that is applied to input $x$, we have then the output $y=f(x)$ Using Bussgang decomposition we can linearize this nonlinearity and express $y$ as $y=Bx+ η$, ...
6
votes
1answer
287 views

Probability of complex eigenvalues

I find this is the best site to post this question, even though I considered cs. It is a Monte Carlo experiment over the set of 10.000 n×n matrices. If a single matrix eigenvalue is complex then ...
3
votes
0answers
80 views

Proving the exponential decay of Green's function for the lattice $-\Delta+p$

The Green function $G(x,y) =G(x-y)$ of the discrete Klein-Gordon operator $-\Delta+p$ on $\mathbb{Z}^{d}$ is given by: \begin{eqnarray} G(x-y) = \int_{[-\pi,\pi]^{d}}\frac{d^{d}k}{(2\pi)^{d}}\frac{e^{...
1
vote
0answers
33 views

Estimation of parameters through multivariate Taylor expansion?

I do have a function $$f(t) = \prod\limits_{j=1}^{n} \left(1 + \sum\limits_{i=1}^{n} M_{i,j} t_i\right)^{-\alpha_{j}}$$ defined by parameters: $M_{i,j} \in \mathbb{R}_{+}, \;\forall i \in 1,...,d,\; ...
-1
votes
1answer
116 views

Sufficient conditions on $ a_i,b_i$ for $a_1\phi(n)+b_1, \cdots, a_k\phi(n)+b_k$ to be simultaneously prime infinitely often?

I am really interested in sufficient conditions on $a_i, b_i$ guaranteeing that the linear forms $a_1\phi(n)+b_1,\dots, a_k\phi(n)+b_k$ become simultaneously prime for infinitely many positive ...
2
votes
0answers
129 views

Extended Kalman Filter and its State Transition Matrix

Sorry for what might be a long post, I want to give background. Initially I had regular Kalman filter, and the state model was defined by Newtonian kinematics, with initial position 0 and speed of 2. ...
1
vote
0answers
56 views

Distances between up and down crosses in Gaussian Processes

Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$, where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...
-2
votes
2answers
186 views

Lower bound of q pochhammer symbol [closed]

How one could prove, that q pochhammer symbol $(1,1/n) = \prod_{k = 1}^{\infty}(1-\frac{1}{n^k}) \geq 1 - \frac{1}{n-1}$
0
votes
1answer
84 views

Error metric for joint estimation of mean and variance

Background: Let $\mu:\mathbb{R}^n\to\mathbb{R}$ and $\sigma:\mathbb{R}^n\to\mathbb{R}_+$ be two unknown functions, and consider a stochastic model of the form $$ \mathbb{E}[Y|\mathbf{x}] = \mu(\...
5
votes
1answer
80 views

Estimating the size of the remainder in a random partition

Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...
-2
votes
1answer
79 views

Existence or impossibility of Gaussian factory

Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
0
votes
1answer
46 views

Strict positive type function on hypersurface also of positive type in neighborhood?

Let $u\in C^\infty(\mathbb{R}^n\times\mathbb{R}^n)$ be symmetric and of strictly positive type on some hypersurface $S \subset \mathbb{R}^n$ diffeomorphic to $\{0\}\times\mathbb{R}^{n-1}$. This means ...
3
votes
1answer
74 views

Optimal linear measurement operator

Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
2
votes
0answers
369 views

Functional Taylor expansion for differential entropy

Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is $h(f)=-\int f(x)\log f(x) dx$. In the literature of differential entropy estimation, ...
2
votes
1answer
144 views

Finding a similarities and differences of sent of matrices

Suppose we have a set of rank deficient covariance matrices. How can I know the similarities and differences between those set of matrices? Regards,
2
votes
1answer
393 views

Distribution of ratio between complex Gaussian and Chi-square R.V.s

What would be the distribution (p.d.f.) of the following ratio? $$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$ where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
3
votes
1answer
150 views

Proving bounds on analytic functions using only the Taylor expansion

I wonder if there is a general method for obtaining bounds on an analytic function using only its Taylor expansion (not using its special properties such as satisfying a good differential equation, ...
3
votes
1answer
103 views

maximum likelihood estimation of X is better than that of f(X)?

Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...
2
votes
2answers
419 views

An alternative proof of Bayesian Cramer-Rao

My question is: Are there an alternative proof of Cramer-Rao lower bound that does not use Cauchy-Swartz inequality? Let me outline the classical proof and explain why I am interested in this ...
1
vote
0answers
85 views

Extended Kalman filter for initial values estimation

I try to make extended Kalman filter for estimation of initial values of small celestial body. I have: $(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values. $z = ...
2
votes
0answers
51 views

Rate of $L_1$ loss in estmating density on $[0,1]$

Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e. $$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...
1
vote
1answer
129 views

Fisher information with vanishing probability

I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as: $$...
1
vote
0answers
88 views

A different objective function in liner regression analysis

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters. Let $Y = \beta^TX + \epsilon $ be the standard liner ...
1
vote
1answer
116 views

Reconstructing the number of distinct elements from a random projection

Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$. We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...
1
vote
0answers
73 views

sufficient statistics that are irrelevant

I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one. If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...
1
vote
0answers
44 views

A question about the prediction error

I am reading about the prediction error estimation and I found the following: Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...
2
votes
0answers
86 views

Calculate sample mean confidence interval of noisy logistical distribution

I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean. I would like to estimate its average $u$ with a confidence interval (...
1
vote
0answers
53 views

Cramer Rao bound for relative estimation

I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$). A feature of our estimation problem is that the involved ...
2
votes
0answers
44 views

Perturbation results for statistical estimators

Suppose I have a continuous random variable whose distribution $f$ is some parametric form (normal, exponential, etc.) that is known to me. If I draw many independent samples $x_i$ from $f$, I can ...
3
votes
0answers
77 views

Uniform mean-square-error estimates

Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...
5
votes
1answer
170 views

Can samples be compressed?

The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is: $\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...
1
vote
0answers
58 views

Robust weighted estimator of location

Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator. Now let $W = (w_1, \ldots, ...
1
vote
0answers
35 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
2
votes
0answers
66 views

Elliptic Equation with Wentzell boundary condition

I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space. The ...
2
votes
1answer
241 views

Literature question on the convergence rate of the empirical distribution

Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...
2
votes
0answers
171 views

Maximum-likelihood estimation for univariate responses from multivariate data

I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way... I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...
1
vote
1answer
234 views

KL divergence Inequality

I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading. Consider two probability measures $P$ and $Q$ both ...
2
votes
0answers
140 views

How to find moment condition for generalized method of moments?

Consider a scalar system with $2K$ outputs and $K+2$ unknowns: $y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$. The variables $n_{k,\ell}$ are zero mean noise variables. To estimate $a_1$ and $...
4
votes
1answer
271 views

Cramér-Rao bound for randomized estimator

As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter. Consider the case when the parameter is a scalar, the estimator is ...