# Tag Info

10

The probability that a $n\times n$ real matrix (with elements that are independent random variables with standard normal distributions) has only real eigenvalues is given by $$2^{-n(n-1)/4}$$ Reference: A. Edelman, The Probability that a Random Real Gaussian Matrix has $k$ Real Eigenvalues, Related Distributions, and the Circular Law. Journal of ...

6

No, the inequality is not guaranteed. One example with $k=2$ is for $g(x_1,x_2) = OR(x_1,x_2)$. Then $f_g(1;\theta) = 1-(1-\theta)^2$, and $\mathcal{I}_g(\theta) = 4/(2\theta-\theta^2)$. We have $\mathcal{I}_g(\theta)>\mathcal{I}_{x_1}(\theta)$ when $\theta<\tfrac23$ and the reverse inequality when $\theta>\tfrac23$. The reason this should not be ...

4

Let $n:=M$. Since you say "the denominator follows a Chi-square distribution with $2M$ degrees of freedom", it appears that you assume the $x_i$'s to be iid standard complex Gaussian; anyway, if you need $a>1$, then you can do a simple rescaling. Thus, let us assume indeed that the $x_i$'s are iid standard complex Gaussian. It is then clear that the ...

4

There is no such general method. You cannot see directly from the Taylor series that $\sin x$ is bounded on the real line, or that $\exp z$ is bounded on the negative ray. Of course what I stated is not a theorem, but just think how this boundedness criterion could possibly look: ANY change in ONE coefficient of the series destroys the property.

4

In line with Deane's comment, this is an "answer" that also uses the Cauchy-Schwarz inequality but does so in a way that you might find more natural. I'll use different notation than yours (sorry; pushed for time and I'll probably mess it up if I attempt to translate quickly). Take a family of probability density functions $f(-; \theta)$ parametrized by ...

3

$$\prod (1-x_i)=1-x_1-x_2(1-x_1)-x_3(1-x_1)(1-x_2)-\ldots \geqslant 1-\sum x_i, \forall x_i\in [0,1],$$ use this for $x_i=1/n^i$.

3

Here is a heuristic that I am sure can be made rigorous. The $x_i$'s can be written recursively as $x_{i+1}=U_{i+1}x_i$, where the $U_i$ are independent Unif[0,1] random variables. In particular, $x_n=U_n\cdots U_1$, so that $\log x_n=\log U_n+\ldots+\log U_1$. By the strong law of large numbers, $(1/n)\log x_n\to \int_0^1\log t\,dt=-1$, so that $x_n\approx ... 3$\def\KL{\mathsf{KL}}$I'm not an expert (sorry), but it is intuitively obvious (and should follow from the standard properties) that$\KL(P,Q)$would decrease if we replace$P$and$Q$by$\bar P$and$\bar Q$which are proportional on$A$and$A^c$, and$\bar P(A)=P(A)$,$\bar Q(A)=Q(A)$. If so, then the required inequality is reduced to $$p+q \geq \... 2 Rational approximations of Markov functions are expressed in terms of orthogonal polynomials with respect to \gamma and related functions, and as Alexandre Eremenko pointed out you'll find more details by looking at Padé approximation; the error terms are somehow explicit. An excellent book for the topic is "Rational approximations and orthogonality" of ... 2 I don't think the power-set structure helps you estimate entropy better. The following extreme cases all give the same entropy for p_B, but entropy of the power set observation ranges from 0 to N (the full range). Extreme case 1, if you only observe B, your entropy H(X) is zero, while entropy of p_B is maximum, \log_2(N). Extreme case 2, if you ... 2 The minimax rate (in \ell_1) for estimating the empirical distribution on an alphabet of size d is \Theta(\sqrt{d/n}), where n is the number of samples. See here for more details: http://arxiv.org/abs/1411.1467 2 A good source on this topic is chapter 7 of Roob Muirhead's book, "Aspects of Multivariate Statistical Theory", or chapter 3 of Peter Forrester's "Log-Gases and Random Matrices". In particular, there are some special functions called zonal polynomials which satisfy$$ \int e^{-{\rm tr}(XZ)}\det(X)^{a-(N+1)/2}P_\lambda(X)dX\propto \det(Z)^{-a}P_\lambda(Z^{-1}... 2 The matrix Gamma distribution (as defined in Wikipedia)$MG_p(\alpha,\beta,\Sigma)$is nothing but the Wishart distribution$W_p\left(2\alpha, \frac{\beta}{2}\Sigma\right)$. The moments and the variances of the Wishart distribution are well-known. 2 Note first, that we may assume$n=D$and all elements of the sequence distinct. To obtain a lower bound, we can start by considering the expectation. Note that we can write$Z = \sum_{i=1}^{k} Z_i$, where$Z_i$is the indicator random variable of whether$i$lies in${h(x_1),\ldots ,h(x_{n})}$or not. Then$\mathbb{E}Z = \sum_{i=1}^{k} \mathbb{E}Z_i$. ... 2 I'm not sure if this is still of interest to you, but I think it is possible to get some reasonable bounds if you are okay with dropping the factor of$\frac{1}{2}$. Here's my work, which can be strengthened and refined. We start by taking two probability mass functions$p$and$q$which we denote as$p_i$and$q_i$. We define the function$f$as$f_i= q_i-...

2

I interpret the question as follows: Given are two pdf's, $p_0$ and $p_1$, on $\mathbb{R}^d$ and a function $f\colon \mathbb{R}^d \to \mathbb{R}^k$. For any $x\in\mathbb{R}^d$, let \hat c(x)\;\begin{cases} =1&\text{ if } p_1(x)>p_0(x),\\ =0&\text{ if } p_1(x)<p_0(x),\\ \in\{0,1\}&\text{ if } p_1(x)=p_0(x), \end{...

1

Let $z_1 = x_1 - x_2$, and $z_2 = x_3 - x_4$, then $z_1, z_2 \sim \mathcal{N}(0,2\sigma^2)$. Let $z = z_1 / z_2$. By results about ratios of Normal variates $z \sim \textrm{Cauchy}()$, i.e. is a standard Cauchy random variable. Let $u = \frac{1}{\pi} \textrm{arctan}(z) + \frac{1}{2}$, the CDF of the standard Cauchy. Therefore $u \sim \textrm{U}([0,1])$. ...

1

There is the theory of "optimal experimental design" or "optimal design of experiments". The main question there is "if I want to measure a certain quantity, how should do it in an optimal way?" "Optimal" can mean many different things here, e.g. that some estimator of the quantity has minimal variance (and this is still not precise, as "variance" is a ...

1

Per the comments, the objective function has real(trace(.)), not trace(.). I will not bother with dividing by N (also mentioned in comments), since that doesn't affect the argmin. This can be formulated as a non-convex Quadratically-Constrained Quadratic Programming (non-convex QCQP) problem, which is much more difficult to solve than a convex QCQP. I will ...

1

Perhaps, the following re-arrangement of the argument will help remove the mystery of the choice of $g(x,y)=\frac{\partial}{\partial x} \ln f(x,y)=\frac{f'_x(x,y)}{f(x,y)}$, where $f:=f_{X,Y}$ and ${}'_x$ denotes the partial derivative in $x$. Assume appropriate regularity conditions, whatever are needed for the manipulations below. Integrating by parts, ...

1

An infinite Fisher information $F(\theta)$ means that the lower bound $1/F(\theta)$ in the error of an unbiased estimator of $\theta$ is zero. A simple example is the Bernoulli trial, success or failure with probabilities $\theta$ and $1-\theta$. The Fisher information is $$F(\theta)= \frac{1}{\theta}\frac{1}{1-\theta}$$ So $F(\theta)$ diverges for $\theta=... 1 A good reference book could be Hero, A. “Signal Detection and Classification” Digital Signal Processing Handbook Ed. Vijay K. Madisetti and Douglas B. Williams Boca Raton: CRC Press LLC, 1999 in which different scenarios were discussed. Besides, some reference books are also helpful. 1 I think I got an affirmative answer for the fixed-size (i.e. non-sequential) case. Let$\hat \theta$be an estimator of the parameter$\theta$. The estimate is obtained as a (deterministic) function of$n$observations$x_1, \ldots, x_n$and$m$auxiliary random variables$y_1, \ldots, y_m$.$\hat \theta$is a randomized estimator, in the sense that the ... 1 The derivation of the Cramer-Rao lower bound in Kay uses the weighted Cauchy-Schwarz inequality: $$\left[ \int w(\mathbf{x})g(\mathbf{x})h(\mathbf{x})d\mathbf{x} \right]^2 \leq \int w(\mathbf{x})g^2(\mathbf{x})d\mathbf{x} \int w(\mathbf{x}) h^2(\mathbf{x}) d\mathbf{x}$$ where$g$and$h$are arbitrary scalar functions, and$w(\mathbf{x}) \geq 0$for all$...

1

The problem with that approach is that every time you add an observation of $X$, you also add an "observation" of $Z$. The space in which you want to find the optimal values of $Z$ and $\theta$, thus grows very large. Instead of making the problem easier, I think you make it more complicated by adding $Z$. If the optimization problem is intractable, you ...

1

If you'd rather use a frequentist method (instead of the Bayesian approach described above), then a recent paper (that also reviews the literature on this) is this one.

1

This paper explains entropy estimation without distribution estimation in the undersampled regime.

1

Your problem is not meaningful in the currently given formulation because the covariance matrix you described is not a valid covariance matrix in general. Covariance matrices of multivariate Gaussian variables are always positive semidefinite. Let us consider the example of $n=4$. The covariance matrix $C$ of the variables \$X_{1,2}, X_{1,3}, X_{1,4}, X_{2,3}...

1

Not an answer, but no comments for me. I don't think the problem is fully specified. You've given marginal distributions and second moments, but no joint distribution. For example, if 0 < rho < 1/2, there is a multivariate normal with these properties; X_ij = Y_i + Y_j + e_ij, where the Y's and e's are mutually independent mean 0 normals, with ...

Only top voted, non community-wiki answers of a minimum length are eligible