All Questions
Tagged with linear-algebra probability-distributions
44 questions
1
vote
0
answers
91
views
How to optimize parametric information-theoretic bounds?
I am faced with an information-theoretic upper bound, such as
\begin{align}
\sqrt{\alpha'}2^{I_\alpha(X;Y)},
\end{align}
where $I_\alpha(X;Y)$ is the Rényi mutual information with parameter $\alpha>...
- 287
1
vote
0
answers
40
views
Asymptotic unitary invariance of rank-one spiked Gaussian matrix
I'm working on some Random Matrix Theory related stuff for my thesis, and i've come across the following problem:
Consider a (normalized) spiked Wigner matrix $\mathbf{A}$
$$ \mathbf{A} = \frac{\beta}{...
- 11
0
votes
0
answers
31
views
What is the Fisher information matrix of the von Mises-Fisher distribution?
Assuming the von Mises-Fisher distribution as
$$f_{p}(\mathbf{x}; \boldsymbol{\mu}, \kappa) = C_{p}(\kappa) \exp \left( {\kappa \boldsymbol{\mu}^\mathsf{T} \mathbf{x} } \right),$$
where $\kappa \ge 0$,...
- 287
1
vote
0
answers
39
views
Constructing a centered distribution absolutely continuous with respect to uniform measure on the sphere with a pre-specified covariance?
Let $\mathcal{K}_n$ denote the space of $n \times n$, real symmetric positive semidefinite matrices $K$ having unit trace.
It is easy to verify that for each $K \in \mathcal{K}_n$, there exists a ...
- 420
1
vote
1
answer
99
views
Maximum column norm of random $A^{-1}B$
Suppose that $A$ is an $n$ by $n$ Gaussian matrix (each component i.i.d. normal distributed with mean 0 and variance 1). Let $b$ be a $n$-Gaussian vector. Then it could be easily proven that the ...
- 33
2
votes
2
answers
215
views
How to analyze the value of convergence of functions of random matrices?
Consider a random i.i.d matrix $\mathbf{A}_{m\times n}$ with entries generated from a complex Gaussian distribution with zero mean and unit variance. I am interested in the large dimension analysis of ...
- 287
4
votes
1
answer
250
views
Does a subset with small cardinality represent the whole set?
Assume that we have heavy-tailed distribution $F(x)$ such that
\begin{align}
F(x)=\mathbb{P}[X\geq x]=x^{-0.5}.
\end{align}
Then, we produce $N$ independent samples $X_1,X_2,\ldots,X_N$ from this ...
- 287
-1
votes
1
answer
77
views
Variance of the logarithm of the mixed Rademacher and complex Gaussian distribution
Consider the scenario where $X$ is a Rademacher random variable taking values $\{−1,+1\}$ with equal probability, and $Z$ is a complex Gaussian random variable with a mean of $0$ and a variance of $\...
- 287
0
votes
1
answer
103
views
Is it reasonable to consider the subgaussian property of the logarithm of the Gaussian pdf?
Let $Y$ denote a Gaussian random variable characterized by a mean $\mu$ and a variance $\sigma^2$. Consider $N$ independent and identically distributed (i.i.d.) copies of $Y$, denoted as $Y_1, Y_2, \...
- 287
2
votes
1
answer
239
views
Hoeffding's Lemma for bounded complex random variables?
If we have a real random variable $X$ such that $a\leq X\leq b$ almost surely, we can establish the following inequality:
\begin{align}
\mathbb{E}\left[\exp\Big(t(X-\mathbb{E}[X])\Big)\right]\leq\exp\...
- 287
14
votes
1
answer
417
views
Lipschitz property of the determinant
$\newcommand{\A}{\mathcal A}\newcommand{\Tr}{\operatorname{tr}}$For $c$ and $C$ such that $0<c<C<\infty$, let $\A_{d;c,C}$ denote the set of all symmetric positive-definite real $d\times d$ ...
- 128k
1
vote
1
answer
115
views
The effect of a small change of the probability distribution on the output of the function
Suppose $X$, $Y$, $X'$ and $Y'$ are random variables whose probability density follows the following relations.
\begin{align}
\|p_X-p_{X'}\|_{\mathrm{TV}}&\leq\epsilon_1,\\
\|p_Y-p_{Y'}\|_{\mathrm{...
- 287
0
votes
1
answer
317
views
Positive definiteness and inverse of a symmetric matrix
Let $L_{\boldsymbol{\Delta}}\in \mathbb{R}^{n \times n}$ be the following symmetric matrix:
$L_{\boldsymbol{\Delta}} = \boldsymbol{\Delta} - \frac{1}{\operatorname{tr}(\boldsymbol{\Delta})} \...
- 11
1
vote
1
answer
82
views
Does the following expectation-based inequality hold?
Let $\mathcal{F}$ be the space of all functions that uniformly and independently map the alphabet $\mathcal{X}$ to the set $\{1,2,\ldots,A\}$. Let $p(x|y)$ be an arbitrary conditional probability ...
- 287
1
vote
1
answer
675
views
KL divergence between gaussian with uniform prior
I have 2 normal distributions $\mathcal{N}(\mu_1, \mathbb{I}_d)$ where $\mu_1$ is a fixed vector in $\mathbb{R}^d$ and $\mathcal{N}(\mu_2, \mathbb{I}_d)$, where $\mu_2$ is $\mu_1 + V$, where $V$ is ...
- 215
1
vote
1
answer
158
views
Generating iid random vectors such that the distribution of their dot product is $\mathit{Uniform}[a, b]$
Take two independent and identically distributed random vectors $X_i$, $X_j$.
I want to find a multivariate distribution for these vectors such that the dot product $X_i^\top X_j \sim U[a, b]$.
This ...
- 119
2
votes
1
answer
1k
views
Components of a Gram matrix and its eigenvalues
The Gram Matrix is defined as $$\sum_{i=1}^n X_iX_i^T,$$ where $X_i$ is drawn from the unit sphere based according to some continuous distribution (Relation between eigenvalues and the gram matrix for ...
- 215
1
vote
0
answers
176
views
Maximum mutual information of random unitary transformation
Let $\mathbf{U}$ and $\mathbf{V}$ be random unitary matrices independent of random input vector $\mathbf{x}$. Moreover, $\mathbf{z}$ be random iid complex Gaussian vector with zero mean and identity ...
- 287
3
votes
0
answers
225
views
Eigenvalues of Hadamard product of two Wishart-type matrices
Given two independent Gaussian matrices with i.i.d. entries: $A\in\mathbb{R}^{n\times p}$ and $B\in\mathbb{R}^{n\times q}$, where and $A_{i,j},B_{i,j}\sim\mathcal{N}(0,1)$. Assume that $\max(p,q)<n....
- 31
-1
votes
1
answer
60
views
Linear operator over a simplex space in a multinomial distribution parameter estimation problem
This is actually a variant of a well-known problem of how the parameters of a multinomial distribution can be estimated by maximum likelihood, and this arises from a final year project I undertook ...
- 1
0
votes
0
answers
92
views
Linear independence of Wishart matrices
Let $W\sim W_n(I,d)$ be a real Wishart matrix of an identity covariance matrix and $d$ degrees of freedom, i.e., $W=XX^T$ for $X$ being an $n\times d$ matrix whose entries are i.i.d sampled from a ...
- 123
2
votes
1
answer
905
views
Diagonalizability of Gaussian random matrices
Let $X$ be an $n\times n$ matrix whose elements are i.i.d. sampled from a normal distribution of zero mean and unit variance. Is $X$ diagonalizable over $\mathbb{C}$ with probability 1? Is there a ...
- 123
0
votes
0
answers
132
views
Upper bound on the condition number of the product of a random sparse matrix and a semi-orthogonal matrix
Let $G \in \mathbb{R}^{n \times m}$ (m > n, m = O(n)) whose all entries are i.i.d. distributed as $\mathcal{N}(0, 1) * \text{Ber}(p)$. Let $V \in \mathbb{R}^{m \times n}$ be a fixed semi-orthogonal ...
- 109
5
votes
1
answer
3k
views
Eigenvalues and eigenvectors of Gaussian random matrices
Let us assume we have a square matrix $A$ whose entries are sampled from a standard Gaussian distribution of mean $0$. Do we have any information about the distribution of its eigenvalues?
...
- 899
1
vote
1
answer
131
views
Large scale analysis of matrix multiplications
Let $\mathbf{A}_{m\times n}$ and $\mathbf{B}_{m\times n}$ be two random i.i.d matrices with zero mean and unit variance. Then, are the following large-scale analysis true (m,n go to infinity with ...
- 287
0
votes
1
answer
181
views
expectation of a quadratic function of a matrix variate normal distribution
I want to compute the following expectation term:
$E[{\bf{XA}}{{\bf{X}}^T}]$
where ${\bf X} \in R^{M \times M}$ and its elements are normal random variables such that
$vec\left( {\bf{X}} \right)\...
- 275
1
vote
1
answer
186
views
Expected norm of linear maps
I want to compute the expected norm of a vector-matrix multiplication. I have a vector $x \in \mathbb{R}^n$ with norm one and a matrix $M \in \mathbb{R}^{n \times n}$, whose entries are iid taken from ...
- 899
1
vote
0
answers
137
views
Choice of residual function for least squares error minimization
Good morning,
I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data.
I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as
$K_{IC} = \sigma \sqrt{D} k_0(\...
- 11
3
votes
0
answers
116
views
Trace of Symmetric matrices in fixed rank
I am solving some problem related to symmetric matrices over a finite field $\mathbb{F}_q$ and I am stuck at the following problem:
For every $a\in\mathbb{F}_q $, let $S_a(t,m)$ be the set of all $m\...
- 179
1
vote
1
answer
480
views
Ratio of perfectly correlated gaussian distributions
Let $M$ be a positive definite matrix and let $w \in S^{d-1}$ be a unit vector uniformly distributed over the sphere. I want to understand the distribution of the quadratic form $\frac{w^T M^3 w}{w^T ...
- 71
2
votes
0
answers
100
views
On a random matrix construction
Given a symmetric matrix $M\in\Bbb Z^{n\times n}$ or rank $r$ with absolute value of any entry bound by $2^{b^2-1}-1$ and maximum eigenvalue at most $\lambda$.
We consider the set $\mathcal T_b$ of $\...
- 13.9k
0
votes
0
answers
93
views
Changing Couplings of Discrete Random Variables
Let $X,Y$ be two discrete random variables. Two joint mass distributions (couplings) with marginals $X$ and $Y$ and with entries $p_{i,j}=\mathbb{P}_1(X=i,Y=j)$ and $p_{i,j}'=\mathbb{P}_2({X=i,Y=j})$ ...
- 329
3
votes
0
answers
88
views
Statistical properties from the projection on Hadamard matrix
I am using Hadamard matrix to generate the measurement matrix in compressive sensing for signal acquisition. Consider all the rows except the first row, which consists of all 1's, as a candidate for ...
- 31
4
votes
2
answers
239
views
Distribution of $0$-$1$ matrices
Consider $n\times n$ matrices with entries in $\{0,1\}$. The determinants of these ranges from $0$ to the Hadamard bound $\frac{(n+1)^{\frac{n+1}2}}{2^n}$. Assume $n$ is large enough.
What does the ...
- 309
3
votes
1
answer
248
views
Log-convexity of conditional variances
Let $K$ be a positive integer and $C$ be any $K \times K$ non-singular matrix. For positive real numbers $q_1, \dots, q_K$, define
$$\Sigma(q_1, \dots, q_K) = CC' + diag(\frac{1}{q_1}, \dots, \frac{1}...
- 765
3
votes
1
answer
187
views
Moment matching on the standard simplex
Let $\vec{\mu}_1, \vec{\mu}_2,\ldots, \vec{\mu}_k \in \Delta^{d-1}$ be $k\ (k\geq 2)$ distinct vectors on the standard simplex, where
$$\Delta^{d-1} = \{\vec{\mu}\in R^{d}:\| \vec{\mu}\|_1 = 1,\mu_j \...
- 1,127
2
votes
1
answer
155
views
A graph assignment problem
Consider bipartite graph with vertex set $V_1\cup V_2$ where $|V_1|=\frac{n(n-1)}2$ and $|V_2|=n$. The vertices in $V_1$ all have degree $2$ and connected to two vertices in $V_2$. The vertices in $...
user76479
1
vote
1
answer
225
views
About expectation norms on graphs
Let $S \subseteq V$ of a $d-$regular graph $G$ such that $\mu = \frac{\vert S \vert }{\vert V \vert } $. Let $A$ be the adjacency matrix of the graph. Then define the quantity $\phi(S)= \frac{E(S,\bar{...
- 1,893
3
votes
1
answer
326
views
Two matrix Fisher distributions on SO(3)?
After the uniform distribution (normalized Haar measure), the matrix Fisher distribution seems to be the most popular probability distribution on the Lie group SO(3). The density is proportional to ...
- 41
1
vote
0
answers
46
views
the 3th and 4th order statistics of Circularly Symmetric Complex Normal random vector?
Assume that ${\bf{z}} \in {\mathbb{C}}^{n \times 1}$ is a CSCG random vector denoted with $\mathcal{C} ~ (\bf{\mu} _0,\bf \Sigma _0)$ where $\mu _0$ and $\bf \Sigma _0$ are mean and contrivance matrix,...
- 275
1
vote
1
answer
141
views
Numerical optimisation for multivariate Gaussians
Hi,
I want to calculate
$
f_{\mathbf x}(x_1,\ldots,x_k)\, =
\frac{1}{(2\pi)^{k/2}|\boldsymbol\Sigma|^{1/2}}
\exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu})^T{\boldsymbol\Sigma}^{-1}({\mathbf x}...
- 619
4
votes
2
answers
3k
views
Monotonicity of the hard EM algorithm.
Consider the problem where we want to find a maximum likelihood estimate of $\theta$, given $X$ and $$P_\theta(Y) = \sum_z P_\theta(Y,x)$$ where $x$ is a latent variable.
I know that the soft EM ...
- 1,902
2
votes
0
answers
279
views
Eigenvectors of convolution with a normal distribution over a restricted interval
Suppose I have a random variable $X_0$ with a p.d.f $f_0$ supported on the real interval $[a_0, b_0]$. $X_1$ is the restriction to $[a_1, b_1]$ of the sum $X_0 + g$, where $g$ is normally distributed $...
- 1,902
2
votes
0
answers
292
views
Seeking the normalizing constant (or any references) for a distribution over a subset of positive definite martrices
I'm interested in a probability distribution over the set of positive definite matrices with unit diagonal elements. That is, and $X$ such that:
$X \in S^{n+}, \forall_{i}X_{ii} = 1$ where $S^{n+}$ ...
- 379