# Tagged Questions

**0**

votes

**0**answers

15 views

### Significance of Eigenvectors of a Covariance matrix [migrated]

In PCA and in many other problems of machine learning we use Eigenvectors of covariance matrix of the data. How do we visualize Eigenvectors of Covariance matrix? The Principal Eigenvector ...

**3**

votes

**1**answer

111 views

### Two matrix Fisher distributions on SO(3)?

After the uniform distribution (normalized Haar measure), the matrix Fisher distribution seems to be the most popular probability distribution on the Lie group SO(3). The density is proportional to ...

**0**

votes

**0**answers

89 views

### Cholesky decomposition of a large covariance matrix

I have a tricky problem concerning a covariance matrix cholesky decomposition.
What I need is to obtain the cholesky decomposition of the estimated variance matrix of the set of samples stored in a ...

**2**

votes

**0**answers

105 views

### Hilbert Schmidt Operators and the Conditional Expectation Operator

Consider the function $\text{E}_W: L_2(\mathbb{R},P_X) \mapsto L_2(\mathbb{R},P_W)$ where $P_X$ and $P_W$ are two different probability measures. They are related in such a way that if $f_X$, $f_W$ ...

**0**

votes

**0**answers

50 views

### Linear Bounds on estimation error

Consider a markov chain on discrete state space $\mathbb{S} = \left\{1,2,..,S \right\}$, with transition probability matrix defined as $A = [a_{ij}]_{S \times S}$ where $a_{ij} = ...

**1**

vote

**0**answers

167 views

### Incoherence of the row/column span

Due to V.Chandrasekaran., et al (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that:
$$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$
where the lower bound is achieved (for ...

**4**

votes

**1**answer

116 views

### Weak ergodicity of nonhomogenous products of 0-1 matrices

Here is a question which probably has a negative answer, but I couldn't find any literature directly on it.
Let $(A_n)$ be a sequence of rectangular 0-1 matrices (that is, the entries are restricted ...

**4**

votes

**3**answers

310 views

### Online estimation of covariance matrix

I am trying to dynamically estimate the (low-dimensional) covariance matrix ${\mathbb E}[{\bf x}_t{\bf x}_t^\top]$ of a stream of data points ${\bf x}_t\in{\mathbb R}^N$ online, without any memory. ...

**4**

votes

**0**answers

225 views

### Probability distribution function for singular value sum of Gaussian random matrix

Let $\mathbf{X}$ be an $N \times N$ random matrix with IID Gaussian entries. They can be standard normal, but $N$ is not large: that is $N$ $<$ 6, typically. Call its singular value decomposition ...

**1**

vote

**1**answer

71 views

### Estimating the relation between the covariance of a vector and a monotone function of the same vector

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as
$$ ...

**-1**

votes

**1**answer

279 views

### Rank of covariance matrix whose diagonal elements are same [closed]

Suppose A is a covariance matrix whose diagonal elements are same, i.e. $A_{1,1}=A_{2,2}=\cdots=A_{N,N}$, can we conclude that A is full rank?
Suppose the absolute values of the off-diagonal elements ...

**8**

votes

**2**answers

369 views

### Rescaling positive definite matrices to force a unit eigenvector

Hello,
Let $X'X$ be a positive definite matrix and let $\mathbf{1}$ denote the vector of ones.
I'm hoping to construct a positive, diagonal matrix $W$ such that
$$(W X'X W) \mathbf{1} = ...

**2**

votes

**0**answers

166 views

### Expectation of a multivariate Gaussian over a plane

For a vector $X$ which follows a multinomial Gaussian distribution $N(\vec{0},\Sigma)$, a given vector $b$, and a known scalar value $c$, I would like to calculate the expectation :
$E[X|X^Tb = c]$
...

**5**

votes

**1**answer

323 views

### Computing the correlation between two vectors without divulging them

Alice and Bob respectively know a vector of $N$ real numbers $u$ and $v$. They would both like to know $\rho = \langle u,v \rangle/N$ but Alice does not want Bob to gain anymore information about $u$ ...

**-1**

votes

**2**answers

126 views

### Multiple Linear Regression Estimation without full recalc [closed]

Ok, so I am running a classic linear regression where betahat = (X'X)^-1X'y
Due to performance issues, I would like to estimate betahat with an additional data point (x1,x2,x3,x4,...,y) without ...

**3**

votes

**1**answer

480 views

### Determinant of an updated Covariance matrix

I am faced with the following problem :
Originally (at time 0) I have a number of data samples $x^0_{1...n}$ (normalised : $E[x] = 0, Var[x] = 1$) from which I have calculated the covariance matrix ...

**4**

votes

**5**answers

922 views

### median of matrices

I have $n$ positive definite Hermitian matrices $M_n$ and I want to define and compute their median.
These matrices correspond to independent estimations of a covariance matrix in the presence of ...

**1**

vote

**1**answer

259 views

### How many zero-constraints can be added to a subspace-restricted matrix before no solution exists?

I'm trying to develop an estimator for the concentration matrix of a Gaussian Graphical Model. I've become stuck in trying to find conditions for the estimator to exist. I have a sufficient ...

**1**

vote

**1**answer

585 views

### Shortcut for variance of vectors

I have a dataset of vectors and need to find the sum of squares or variance based on the euclidean distance between the vectors.
I can do this by finding the "average" vector (by calculating the ...

**2**

votes

**1**answer

294 views

### Derivation of Iteration Rules

Suppose we are given a matrix $V$ and our goal is to find non-negative matrices $W$ and $H$ such that $V \approx WH$. So we want to minimize $K(V || WH)$ (Kullback-Leibler Divergence) where $$K(V||WH) ...

**0**

votes

**1**answer

952 views

### Convergence of Eigenvalues

Suppose we have a matrix $A_n = \frac{1}{n}\sum_{i=1}^nX_i X_i^T$, where $X_i$ is a $p$-dimensional random-vector. We also know that $E(XX^T) = \Sigma_{p \times p}$. Let us denote the $j$-th largest ...

**1**

vote

**0**answers

174 views

### What is the MP pseudoinverse's role in statistical learning and Self-Organizing Maps?

During a discussion in our lab last month, a professor mentioned to me that the behavior of Self-Organizing Maps can be described in terms of repeated applications of the Moore-Penrose psuedoinverse, ...

**4**

votes

**1**answer

882 views

### Monotonicity of the hard EM algorithm.

Consider the problem where we want to find a maximum likelihood estimate of $\theta$, given $X$ and $$P_\theta(Y) = \sum_z P_\theta(Y,x)$$ where $x$ is a latent variable.
I know that the soft EM ...

**0**

votes

**2**answers

252 views

### Nonexistence of projection

Following is a doubt I got when I was reading Csiszar's Annals of Statistics paper "Why least squares and maximum entropy: An axiomatic approach to inference for linear inverse problems".
He comes up ...

**1**

vote

**0**answers

340 views

### Bounding point-wise maximum of the absolute difference of two convex functions

Let $\Delta: R \times R \rightarrow R_{+}$ be a positive and convex function (convex in, say, both the arguments) called the loss function.
Let $x \in R^d$. Moreover, let $H_1,...,H_r$ be sets of ...

**1**

vote

**1**answer

512 views

### Sequential sampling of Gaussian and von Mises-Fisher Random Variable

I don't find any article discussing this problem, so I dare to ask it.
Suppose we are dealing with a data $x_0 \in \mathbb{R}$ and a function $f:\mathbb{R} \to \mathbb{R}$. Say we repeatedly apply ...

**1**

vote

**2**answers

820 views

### Inequality-constrained linear-regression, what is the covariance of the estimator?

If you do a linear regression: $||Ax - e ||^2$, where e is iid Gaussian, mean 0 and variance 1, then your answer is $x_{hat} = (A' A)^{-1} (A' * e)$ and the covariance of $x_{hat}$ is $(A' A)^{-1}$
...

**0**

votes

**1**answer

178 views

### Data Averging method

I have a series of 2 columns called Amount & Percentage. Now the amount is incrementing with no linear value eg 1659,3167,4495 and I want to have a data for every 1000 that is ...

**1**

vote

**2**answers

887 views

### Assessing measurement accuracy and precision

I have been asked to assess the accuracy and precision of a new measurement method (Let's call it method B). This new method is compared to an existing one (A) that has its own specifications in ...

**4**

votes

**1**answer

493 views

### O(n^2) algorithm to approximate the sum of the log of the singular values of a matrix

Given an $M \times N$ matrix of rank $N$ ($M \ge N$) with $i^{th}$ singular value $\sigma_i$, does their exist an $O(M^2)$ algorithm for approximating the sum $ H =\sum_{i=1}^N \log(\sigma_i)$ with ...

**3**

votes

**2**answers

1k views

### “Main” diagonal of a matrix

Hello!
I'm in search of some (possibly statistical) measure for matrices. I want to classify a square matrix as having the largest numbers running along the main diagonal or along the anitdiagonal. ...

**1**

vote

**2**answers

1k views

### Multiple outliers for two variable linear regression

Problem
Visually, the "extreme" outliers in the following graph are somewhat obvious:
http://i.imgur.com/tiSbS.png
Question
Given:
T - Set of all temperatures
Y - Set of all years
ΣT - Sum of ...

**3**

votes

**3**answers

2k views

### Linear Regression Coefficients W/ X, Y swapped

Let's say I have a linear regression model of the form $ y = B_x x + I_x + \epsilon $, where $B_x$ is the beta coefficient of the $x$ term, $I_x$ is the intercept term and $\epsilon$ is additive, ...

**0**

votes

**2**answers

237 views

### Corruption and Recovery

Suppose we want to recover an input vector $f \in \textbf{R}^n$ from some measurements $y = Af + \varepsilon$. Now $A$ is an $m \times n$ matrix and $\varepsilon$ are some unknown errors. Is this ...

**1**

vote

**1**answer

572 views

### Quantifying Aggregate Vector Strength/Vector Arithmatic

Say I have 5 vectors and I measure the similarity of each one to a fixed reference vector using cosine similarity. But now what I want to do is understand the aggregate or collective strength of these ...

**2**

votes

**1**answer

785 views

### Question about orthogonal matching pursuit

Let y be a n-vector, X a n-by-p matrix of full rank (p < n) and b a p-vector, so that y = Xb + e, for some noise vector e. I am not sure how to show reduction of error in orthogonal matching ...