All Questions
Tagged with estimate or estimation-theory
111 questions
6
votes
1
answer
434
views
Probability of complex eigenvalues
I find this is the best site to post this question, even though I considered cs.
It is a Monte Carlo experiment over the set of 10.000 n×n matrices.
If a single matrix eigenvalue is complex then ...
- 2,552
3
votes
0
answers
266
views
Proving the exponential decay of Green's function for the lattice $-\Delta+p$
The Green function $G(x,y) =G(x-y)$ of the discrete Klein-Gordon operator $-\Delta+p$ on $\mathbb{Z}^{d}$ is given by:
\begin{eqnarray}
G(x-y) = \int_{[-\pi,\pi]^{d}}\frac{d^{d}k}{(2\pi)^{d}}\frac{e^{...
- 1,305
1
vote
0
answers
56
views
Estimation of parameters through multivariate Taylor expansion?
I do have a function $$f(t) = \prod\limits_{j=1}^{n} \left(1 + \sum\limits_{i=1}^{n} M_{i,j} t_i\right)^{-\alpha_{j}}$$ defined by parameters:
$M_{i,j} \in \mathbb{R}_{+}, \;\forall i \in 1,...,d,\; ...
- 4,713
-1
votes
1
answer
144
views
Sufficient conditions on $ a_i,b_i$ for $a_1\phi(n)+b_1, \cdots, a_k\phi(n)+b_k$ to be simultaneously prime infinitely often?
I am really interested in sufficient conditions on $a_i, b_i$ guaranteeing that the linear forms $a_1\phi(n)+b_1,\dots, a_k\phi(n)+b_k$ become simultaneously prime for infinitely many positive ...
CommunityBot
- 1
2
votes
0
answers
385
views
Extended Kalman Filter and its State Transition Matrix
Sorry for what might be a long post, I want to give background.
Initially I had regular Kalman filter, and the state model was defined by Newtonian kinematics, with initial position 0 and speed of 2. ...
1
vote
0
answers
62
views
Distances between up and down crosses in Gaussian Processes
Given a gaussian process $g := \mathcal{GP}\left(\mu, \Sigma \right)$,
where $\mu$ is the mean and $\Sigma$ is the covariance function, I am interested in estimating the mean value $L_m$ of the ...
- 63.9k
-2
votes
2
answers
477
views
Lower bound of q pochhammer symbol [closed]
How one could prove, that q pochhammer symbol $(1,1/n) = \prod_{k = 1}^{\infty}(1-\frac{1}{n^k}) \geq 1 - \frac{1}{n-1}$
- 37.7k
2
votes
1
answer
676
views
Distribution of ratio between complex Gaussian and Chi-square R.V.s
What would be the distribution (p.d.f.) of the following ratio?
$$z = \frac{x_{1}}{|x_{1}|^2 + |x_{2}|^2 + ... + |x_{M}|^2}$$
where $x_{i} \sim \mathcal{CN}(0,a), \forall i$ and $a > 1$. As can ...
5
votes
1
answer
107
views
Estimating the size of the remainder in a random partition
Pick a sequence of real numbers $x_i$ as follows. Put $x_0=1$. If $x_i$ is chosen, then pick $x_{i+1}\in[0, x_i]$ according to the uniform distribution. Obviously we have $x_i\rightarrow 0$ with ...
- 23.2k
-2
votes
1
answer
92
views
Existence or impossibility of Gaussian factory
Gaussian factory problem: given an iid sequence $x_i \sim \mathcal{N}(\mu,\sigma^2)$, $i=1,2,\dots$, with $\mu$ and $\sigma^2$ both unknown, construct a realization $y \sim \mathcal{N}(0,1)$.
0
votes
1
answer
51
views
Strict positive type function on hypersurface also of positive type in neighborhood?
Let $u\in C^\infty(\mathbb{R}^n\times\mathbb{R}^n)$ be symmetric and of strictly positive type on some hypersurface $S \subset \mathbb{R}^n$ diffeomorphic to $\{0\}\times\mathbb{R}^{n-1}$. This means ...
- 13
2
votes
1
answer
154
views
Finding a similarities and differences of sent of matrices
Suppose we have a set of rank deficient covariance matrices. How can I know the similarities and differences between those set of matrices?
Regards,
- 21
20
votes
1
answer
4k
views
Using Fisher Information to bound KL divergence
Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...
CommunityBot
- 1
3
votes
1
answer
87
views
Optimal linear measurement operator
Let $x\in R^n$ be an unknown vector. Suppose I am allowed to choose any $A\in R^{m\times n}$, under the constraint that each row of $A$ has $\ell_2$ norm at most $1$. Then I carry out a "measurement", ...
- 12.7k
4
votes
0
answers
716
views
Functional Taylor expansion for differential entropy
Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...
- 155
3
votes
2
answers
566
views
Moments of Matrix Gamma distribution
Matrix gamma distribution (defined for example in http://en.wikipedia.org/wiki/Matrix_gamma_distribution) is one way to generalize Wishart distribution. In our course work that distribution was used ...
- 2,600
4
votes
1
answer
297
views
Proving bounds on analytic functions using only the Taylor expansion
I wonder if there is a general method for obtaining bounds on an analytic function using only its Taylor expansion (not using its special properties such as satisfying a good differential equation, ...
- 91.8k
3
votes
1
answer
113
views
maximum likelihood estimation of X is better than that of f(X)?
Consider a binary variable $C$ with $\Pr(C=0)=\Pr(C=1)=0.5$. Consider a random vector $X \in \mathbb{R}^d$, characterized by $C$, with PDF $p_m(x)$, $m\in\{0,1\}$. Define a maximum likelihood (ML) ...
- 128k
1
vote
0
answers
101
views
Extended Kalman filter for initial values estimation
I try to make extended Kalman filter for estimation of initial values of small celestial body. I have:
$(x_1^0, x_2^0, x_3^0, v_1^0, v_2^0, v_3^0) = (x^0, v^0)$ -- inaccurate initial values.
$z = ...
- 11
2
votes
0
answers
56
views
Rate of $L_1$ loss in estmating density on $[0,1]$
Let $f$ be a density on $[0,1]$ and let $X_1,X_2,\ldots$ be $\textit{iid}$ $f$-distributed. Also, let $f_n$ denote the kernel density estimator, i.e.
$$f_n(x) = \frac{1}{nh_n} \sum_{i=1}^n K\left(\...
- 2,545
1
vote
1
answer
193
views
Fisher information with vanishing probability
I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as:
$$...
- 188k
4
votes
2
answers
135
views
Markov-type functions
I'd like to have some informations about Markov-type functions (or Cauchy-type):
\[ f(z)=\int_{\Gamma} \frac{\mathrm{d}\gamma(\xi)}{\xi-z}.\]
$\gamma$ is a positive measure with compact support $\...
- 2,600
1
vote
0
answers
93
views
A different objective function in liner regression analysis
I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...
- 11
1
vote
1
answer
124
views
Reconstructing the number of distinct elements from a random projection
Assume we have an unknown sequence $x_1,\ldots, x_n\in \mathcal U$.
We get to observe the sequence $h(x_1),h(x_2),\ldots, h(x_n)$, where $h:\mathcal U\to \{1,\ldots, k\}$ is a random function such ...
- 618
1
vote
0
answers
79
views
sufficient statistics that are irrelevant
I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one.
If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...
- 129
1
vote
0
answers
49
views
A question about the prediction error
I am reading about the prediction error estimation and I found the following:
Suppose we have ${\mathbf{Y}}=\mathbf{x}_0+ \epsilon$, where, $\epsilon$ is normally distributed as $\sim \mathcal{N}(0, \...
- 11
1
vote
0
answers
47
views
Bounding Hidden Markov model Bayesian filter error with inexact models
In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...
4
votes
1
answer
479
views
Cramér-Rao bound for randomized estimator
As is well known, the Cramér-Rao bound (or information inequality) sets a lower bound on the variance of estimators of a parameter.
Consider the case when the parameter is a scalar, the estimator is ...
- 337
2
votes
0
answers
119
views
Calculate sample mean confidence interval of noisy logistical distribution
I have $n$ samples which follow a logistic distribution with unknown $u$ and $s$; it is affected by a Gaussian noise with 0 mean.
I would like to estimate its average $u$ with a confidence interval (...
- 21
1
vote
0
answers
62
views
Cramer Rao bound for relative estimation
I have an observed vector ${\bf y}$ from which I would like to estimate a parameter vector ${\bf c}$ (denote the estimate $\hat{{\bf c}}$).
A feature of our estimation problem is that the involved ...
- 21
3
votes
0
answers
82
views
Uniform mean-square-error estimates
Consider a standard statistical estimation problem with iid real observations $\{X_i\}_{i=1}^N$. For a collection of real functions $\mathcal{F}$, I want to get an estimate of the uniform rate of ...
- 111
2
votes
0
answers
193
views
How to find moment condition for generalized method of moments?
Consider a scalar system with $2K$ outputs and $K+2$ unknowns:
$y_{k,1}=x_ka_1+n_{k,1} \quad y_{k,2}=x_ka_2+n_{k,1}$.
The variables $n_{k,\ell}$ are zero mean noise variables.
To estimate $a_1$ and $...
- 8,529
4
votes
1
answer
203
views
Can samples be compressed?
The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is:
$\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...
- 5,971
2
votes
0
answers
72
views
Robust weighted estimator of location
Let $X = (x_1, \ldots, x_n)$ be a sample of i.i.d values. There are several robust estimators of sample location, most notably sample median and Hodges-Lehmann estimator.
Now let $W = (w_1, \ldots, ...
- 19.6k
2
votes
0
answers
92
views
Elliptic Equation with Wentzell boundary condition
I'm looking for a reference showing how to obtain a priori estimate for solutions to a linear second-order elliptic equation with Wentzell boundary condition in a bounded domain in $H^1$ space.
The ...
- 133
2
votes
1
answer
444
views
Literature question on the convergence rate of the empirical distribution
Assume that given $n$ i.i.d samples $(X_1, X_2, ..., X_n)$ drawn from $p_X$, an unknown probability mass function defined over a finite alphabet $\mathcal{X}$, one wants to estimate $p_X(x)$ for each $...
- 6,541
2
votes
0
answers
187
views
Maximum-likelihood estimation for univariate responses from multivariate data
I am new in the field of machine learning, so I hope I will be able to formulate my question in a clear way...
I have some data represented by vectors $\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_n \...
- 617
1
vote
1
answer
256
views
KL divergence Inequality
I am trying to find a proof for the following inequality, but I did not get anywhere following the references from the paper I was reading.
Consider two probability measures $P$ and $Q$ both ...
- 23.7k
2
votes
0
answers
51
views
MLE and CRLB with mismatched likelihoods
Suppose that I can do a Karhunen-Loeve expansion of a log-likelihood function $p(\bf{x};\theta)$ into N terms and that these accounts for a fraction $1-\delta$ of the total energy. Now consider ...
- 21
1
vote
0
answers
46
views
Equivalence of Graphical model selection algorithms
Suppose, a jointly Gaussian random vector is denoted by $X \in \mathbb{R}^{p}$ and $X$ has a distribution given by $\mathcal{N}(\mu,\Sigma)$. It is known that estimating the graphical model that ...
- 275
1
vote
0
answers
244
views
Distribution of a signal covariance matrix
A common estimation problem in signal processing assumes the following signal model
\begin{equation}
\mathbf{r} = \sum_{i=1}^{Q}\alpha_i\mathbf{s}\left(w_i\right)+\mathbf{n}
\end{equation}
where $\...
- 345
2
votes
1
answer
101
views
Estimating mean and variance of a distribution based on error-prone estimates of its cdf
Suppose I have some random variable $X$ taking values in $[a, b]$ with unknown distribution (I am happy to assume the distribution is smooth, though it would be nice to not have to).
I have a ...
- 1,331
2
votes
1
answer
88
views
What is the problem with this model parameter estimation algorithm?
In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...
- 53
3
votes
0
answers
975
views
Interpolating Wavelet Coefficients
Hi! I was instructed via reddit that this place would be the best place to post this question. Fingers cross you can help...
Ive been writing some code to get rid of noise "spikes" in a signal. I'm ...
CommunityBot
- 1
2
votes
3
answers
409
views
How to estimate the entropy of a distribution on a power set?
Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$.
Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...
- 165
2
votes
2
answers
174
views
estimating variance of dependent normal distributed data
Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that
$
\text{corr}(X_{ij},X_{rs})=\rho
...
- 145
4
votes
1
answer
288
views
Equivalent method for maximum likelihood estimation of covariance parameters
My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function:
$$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...
0
votes
1
answer
85
views
About the suboptimality of linear estimators
Let $X$ be a random variable and $N$ a Gaussian noise independent from $X$. We observe $Y=X+N$ and want to estimate $X$ based on $Y$ to minimize the mean square error $mmse(X|Y):=E(\hat X(Y)-X)^2$.
...
- 7,514
1
vote
0
answers
186
views
Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts
I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...
- 203
5
votes
0
answers
190
views
Pair of two-variable polynomial equations of high order
I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N \...
- 548