0
votes
0answers
15 views

Significance of Eigenvectors of a Covariance matrix [migrated]

In PCA and in many other problems of machine learning we use Eigenvectors of covariance matrix of the data. How do we visualize Eigenvectors of Covariance matrix? The Principal Eigenvector ...
2
votes
2answers
220 views

Gaussian expectation of an exponentiated outer product

Given a normal random column vector $\mathbf{x} \sim N(\mu, \Sigma)$, I need the expectation, $$ E\left[ \exp(\mathbf{xx}^\top)\right]$$ where $\exp(\cdot)$ is element-wise exponential function (not ...
0
votes
1answer
81 views

Understanding the derivation of a ML-estimator (statistics)

I'm trying to understand the derivation of a ML-estimator and more specifically the rewriting of the covariance matrix $\Sigma$. In this rewriting, a lemma is used to show that: $$ \tag{1} ...
1
vote
0answers
167 views

Incoherence of the row/column span

Due to V.Chandrasekaran., et al‎ (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that: $$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$ where the lower bound is achieved (for ...
4
votes
1answer
116 views

Weak ergodicity of nonhomogenous products of 0-1 matrices

Here is a question which probably has a negative answer, but I couldn't find any literature directly on it. Let $(A_n)$ be a sequence of rectangular 0-1 matrices (that is, the entries are restricted ...
8
votes
2answers
369 views

Rescaling positive definite matrices to force a unit eigenvector

Hello, Let $X'X$ be a positive definite matrix and let $\mathbf{1}$ denote the vector of ones. I'm hoping to construct a positive, diagonal matrix $W$ such that $$(W X'X W) \mathbf{1} = ...
0
votes
0answers
162 views

Singular Value Decomposition Question

Hi: I'm new to this list so apologies if I do anything wrong with my question. Suppose I have a matrix $Y$ whose SVD can be decomposed as $Y = U_{0}\Sigma_{0}V_{0}^{*} + U_{1}\Sigma_{1} ...
3
votes
1answer
480 views

Determinant of an updated Covariance matrix

I am faced with the following problem : Originally (at time 0) I have a number of data samples $x^0_{1...n}$ (normalised : $E[x] = 0, Var[x] = 1$) from which I have calculated the covariance matrix ...
4
votes
5answers
923 views

median of matrices

I have $n$ positive definite Hermitian matrices $M_n$ and I want to define and compute their median. These matrices correspond to independent estimations of a covariance matrix in the presence of ...
2
votes
1answer
294 views

Derivation of Iteration Rules

Suppose we are given a matrix $V$ and our goal is to find non-negative matrices $W$ and $H$ such that $V \approx WH$. So we want to minimize $K(V || WH)$ (Kullback-Leibler Divergence) where $$K(V||WH) ...
0
votes
0answers
122 views

Why does the OLS estimator simplify as follows for the single regressor case?

I was reading in "A Guide to Econometrics" that given $Y = X \beta + \epsilon$, the variance covariance matrix of $\beta^\text{OLS}$ is given by $\sigma^2 (X' X)^{-1}$ where $\sigma^2$ is the variance ...
4
votes
1answer
282 views

Explanation of a proof of an determinant bound?

Let $A\in \mathbb{R}^{n\times n}$ be a positive definite symmetric matrix with eigenvalues $\lambda_1\ge \cdots\ge \lambda_n$, $X\in \mathbb{R}^{n\times k}$ such that $X'X=I_k$ ($X'$ means the ...
5
votes
2answers
837 views

Solving for Moore Penrose pseudo inverse

I have a system to solve, set up as : $$Ax = b$$ with a square rank deficient matrix $A$. The paper suggests to use a Moore Penrose pseudo inverse, which in my case can be computed using the ...
2
votes
1answer
496 views

What are the origin and applications of this result?

In a course taught by Morris Eaton on multivariate statistics that dealt mostly with the Wishart distribution, I learned this proposition: Suppose $$ M = \begin{bmatrix} A & B \\\\ B^T & C ...
2
votes
1answer
391 views

Multinomial transformation for matrices

Suppose we have a vector of probabilities $\mathbf{p}=(p_1,...,p_n)$, where $p_i>0$ for $i=1,...n$ and $\sum p_i=1$. Define new vector $\mathbf{r}=(r_1,...,r_{n-1})$ in a following way: ...
3
votes
2answers
1k views

“Main” diagonal of a matrix

Hello! I'm in search of some (possibly statistical) measure for matrices. I want to classify a square matrix as having the largest numbers running along the main diagonal or along the anitdiagonal. ...