# Tagged Questions

**0**

votes

**0**answers

9 views

### How to estimate the covariance matrix if the unnormalized pdf is known but integral is intractable? [migrated]

Assume a $d$-dimensional random vector $x$, whose unnormalized pdf is known as the product of N multivariate t-distribution:
$$Pr(x)\propto\prod_{i=1}^nt_{\nu_i,\mu_i,\Sigma_i}(x)$$
Is there any ...

**2**

votes

**1**answer

51 views

### What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...

**0**

votes

**0**answers

48 views

### Linear Bounds on estimation error

Consider a markov chain on discrete state space $\mathbb{S} = \left\{1,2,..,S \right\}$, with transition probability matrix defined as $A = [a_{ij}]_{S \times S}$ where $a_{ij} = ...

**0**

votes

**0**answers

18 views

### How to generalize uncertainty coefficient to set-valued classes?

This question is the reason I asked How to estimate the entropy of a distribution on a power set?
Proficiency (AKA uncertainty coefficient) is an information-theoretic measure of predictor quality, ...

**1**

vote

**3**answers

107 views

### How to estimate the entropy of a distribution on a power set?

Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$.
Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...

**2**

votes

**2**answers

76 views

### estimating variance of dependent normal distributed data

Let $X_{ij}$ with $1\leq i<j\leq n$ (that are $X_{12},\dots, X_{1n},\dots,X_{(n-1)n}$) be ${n \choose 2}$ identically normal distributed $N(0,\sigma^2)$ such that
$
\text{corr}(X_{ij},X_{rs})=\rho
...

**1**

vote

**0**answers

56 views

### Stochastic process inference from partial observations

Consider a set $U$. My signal is a piece-wise constant "function"
$Sig: t \mapsto s$, i.e. the signal at time $t$ equals to some subset
$s \subset U$. One can see $Sig(t)$ as a stochastic process.
...

**3**

votes

**1**answer

139 views

### Equivalent method for maximum likelihood estimation of covariance parameters

My goal is to estimate the parameters of a covariance matrix $\Omega$, by maximizing the following log-likelihood function:
$$\log L(\vec\tau, \rho, \sigma \mid W, X) = -m\ln(\left | \Omega \right |) ...

**1**

vote

**0**answers

97 views

### Shrinkage (or Stein's phenomenon) in low dimensions, discrete contexts

I am trying to understand shrinkage, or the Stein phenomenon. As someone without a statistics background, the focus in most introductory presentations on normal distributions and squared error loss ...

**4**

votes

**0**answers

159 views

### Pair of two-variable polynomial equations of high order

I have the following pair of equations to be solved for two variables $\rho$ and $D$ resulting from a certain Maximum Likelihood Estimation for a time series $X_n > 0$, $n=0, \ldots, N+1$ with $N ...

**0**

votes

**0**answers

99 views

### What are Effective Regression Techniques for Linguistic Analysis of Linked Data?

I am in the early stages of a problem that involves parsing a large number ($\approx 5 \times 10^9$) of documents (web pages) and estimating values from them. In particular I need to identify pages ...

**1**

vote

**1**answer

185 views

### Is an unbiased estimator with arbitrarily small variance necessarily consistent?

Given an unbiased estimator $\hat \theta_n$ of a parameter $\theta$, if the estimator has small variance (approaching $0$ as $n\to\infty$), it seems reasonable to expect that the estimator is ...

**0**

votes

**0**answers

120 views

### Why does the OLS estimator simplify as follows for the single regressor case?

I was reading in "A Guide to Econometrics" that given $Y = X \beta + \epsilon$, the variance covariance matrix of $\beta^\text{OLS}$ is given by $\sigma^2 (X' X)^{-1}$ where $\sigma^2$ is the variance ...

**2**

votes

**1**answer

359 views

### Why doesn't Stein effect happen for multinomial distributions?

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an ...

**4**

votes

**0**answers

1k views

### Using Fisher Information to bound KL divergence

Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)?
KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...

**1**

vote

**1**answer

261 views

### Is there a text on estimation theory online?

Where can I find graduate level, thorough, parameter estimation/ estimation theory material on the web?