0
votes
0answers
15 views

Significance of Eigenvectors of a Covariance matrix [migrated]

In PCA and in many other problems of machine learning we use Eigenvectors of covariance matrix of the data. How do we visualize Eigenvectors of Covariance matrix? The Principal Eigenvector ...
3
votes
1answer
112 views

Two matrix Fisher distributions on SO(3)?

After the uniform distribution (normalized Haar measure), the matrix Fisher distribution seems to be the most popular probability distribution on the Lie group SO(3). The density is proportional to ...
0
votes
0answers
89 views

Cholesky decomposition of a large covariance matrix

I have a tricky problem concerning a covariance matrix cholesky decomposition. What I need is to obtain the cholesky decomposition of the estimated variance matrix of the set of samples stored in a ...
2
votes
0answers
105 views

Hilbert Schmidt Operators and the Conditional Expectation Operator

Consider the function $\text{E}_W: L_2(\mathbb{R},P_X) \mapsto L_2(\mathbb{R},P_W)$ where $P_X$ and $P_W$ are two different probability measures. They are related in such a way that if $f_X$, $f_W$ ...
0
votes
0answers
50 views

Linear Bounds on estimation error

Consider a markov chain on discrete state space $\mathbb{S} = \left\{1,2,..,S \right\}$, with transition probability matrix defined as $A = [a_{ij}]_{S \times S}$ where $a_{ij} = ...
1
vote
0answers
167 views

Incoherence of the row/column span

Due to V.Chandrasekaran., et al‎ (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that: $$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$ where the lower bound is achieved (for ...
4
votes
1answer
116 views

Weak ergodicity of nonhomogenous products of 0-1 matrices

Here is a question which probably has a negative answer, but I couldn't find any literature directly on it. Let $(A_n)$ be a sequence of rectangular 0-1 matrices (that is, the entries are restricted ...
4
votes
3answers
312 views

Online estimation of covariance matrix

I am trying to dynamically estimate the (low-dimensional) covariance matrix ${\mathbb E}[{\bf x}_t{\bf x}_t^\top]$ of a stream of data points ${\bf x}_t\in{\mathbb R}^N$ online, without any memory. ...
4
votes
0answers
225 views

Probability distribution function for singular value sum of Gaussian random matrix

Let $\mathbf{X}$ be an $N \times N$ random matrix with IID Gaussian entries. They can be standard normal, but $N$ is not large: that is $N$ $<$ 6, typically. Call its singular value decomposition ...
1
vote
1answer
71 views

Estimating the relation between the covariance of a vector and a monotone function of the same vector

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as $$ ...
-1
votes
1answer
279 views

Rank of covariance matrix whose diagonal elements are same [closed]

Suppose A is a covariance matrix whose diagonal elements are same, i.e. $A_{1,1}=A_{2,2}=\cdots=A_{N,N}$, can we conclude that A is full rank? Suppose the absolute values of the off-diagonal elements ...
8
votes
2answers
369 views

Rescaling positive definite matrices to force a unit eigenvector

Hello, Let $X'X$ be a positive definite matrix and let $\mathbf{1}$ denote the vector of ones. I'm hoping to construct a positive, diagonal matrix $W$ such that $$(W X'X W) \mathbf{1} = ...
2
votes
0answers
166 views

Expectation of a multivariate Gaussian over a plane

For a vector $X$ which follows a multinomial Gaussian distribution $N(\vec{0},\Sigma)$, a given vector $b$, and a known scalar value $c$, I would like to calculate the expectation : $E[X|X^Tb = c]$ ...
5
votes
1answer
323 views

Computing the correlation between two vectors without divulging them

Alice and Bob respectively know a vector of $N$ real numbers $u$ and $v$. They would both like to know $\rho = \langle u,v \rangle/N$ but Alice does not want Bob to gain anymore information about $u$ ...
-1
votes
2answers
126 views

Multiple Linear Regression Estimation without full recalc [closed]

Ok, so I am running a classic linear regression where betahat = (X'X)^-1X'y Due to performance issues, I would like to estimate betahat with an additional data point (x1,x2,x3,x4,...,y) without ...
3
votes
1answer
480 views

Determinant of an updated Covariance matrix

I am faced with the following problem : Originally (at time 0) I have a number of data samples $x^0_{1...n}$ (normalised : $E[x] = 0, Var[x] = 1$) from which I have calculated the covariance matrix ...
4
votes
5answers
923 views

median of matrices

I have $n$ positive definite Hermitian matrices $M_n$ and I want to define and compute their median. These matrices correspond to independent estimations of a covariance matrix in the presence of ...
1
vote
1answer
259 views

How many zero-constraints can be added to a subspace-restricted matrix before no solution exists?

I'm trying to develop an estimator for the concentration matrix of a Gaussian Graphical Model. I've become stuck in trying to find conditions for the estimator to exist. I have a sufficient ...
1
vote
1answer
585 views

Shortcut for variance of vectors

I have a dataset of vectors and need to find the sum of squares or variance based on the euclidean distance between the vectors. I can do this by finding the "average" vector (by calculating the ...
2
votes
1answer
294 views

Derivation of Iteration Rules

Suppose we are given a matrix $V$ and our goal is to find non-negative matrices $W$ and $H$ such that $V \approx WH$. So we want to minimize $K(V || WH)$ (Kullback-Leibler Divergence) where $$K(V||WH) ...
0
votes
1answer
952 views

Convergence of Eigenvalues

Suppose we have a matrix $A_n = \frac{1}{n}\sum_{i=1}^nX_i X_i^T$, where $X_i$ is a $p$-dimensional random-vector. We also know that $E(XX^T) = \Sigma_{p \times p}$. Let us denote the $j$-th largest ...
1
vote
0answers
174 views

What is the MP pseudoinverse's role in statistical learning and Self-Organizing Maps?

During a discussion in our lab last month, a professor mentioned to me that the behavior of Self-Organizing Maps can be described in terms of repeated applications of the Moore-Penrose psuedoinverse, ...
4
votes
1answer
882 views

Monotonicity of the hard EM algorithm.

Consider the problem where we want to find a maximum likelihood estimate of $\theta$, given $X$ and $$P_\theta(Y) = \sum_z P_\theta(Y,x)$$ where $x$ is a latent variable. I know that the soft EM ...
0
votes
2answers
252 views

Nonexistence of projection

Following is a doubt I got when I was reading Csiszar's Annals of Statistics paper "Why least squares and maximum entropy: An axiomatic approach to inference for linear inverse problems". He comes up ...
1
vote
0answers
340 views

Bounding point-wise maximum of the absolute difference of two convex functions

Let $\Delta: R \times R \rightarrow R_{+}$ be a positive and convex function (convex in, say, both the arguments) called the loss function. Let $x \in R^d$. Moreover, let $H_1,...,H_r$ be sets of ...
1
vote
1answer
512 views

Sequential sampling of Gaussian and von Mises-Fisher Random Variable

I don't find any article discussing this problem, so I dare to ask it. Suppose we are dealing with a data $x_0 \in \mathbb{R}$ and a function $f:\mathbb{R} \to \mathbb{R}$. Say we repeatedly apply ...
1
vote
2answers
820 views

Inequality-constrained linear-regression, what is the covariance of the estimator?

If you do a linear regression: $||Ax - e ||^2$, where e is iid Gaussian, mean 0 and variance 1, then your answer is $x_{hat} = (A' A)^{-1} (A' * e)$ and the covariance of $x_{hat}$ is $(A' A)^{-1}$ ...
0
votes
1answer
178 views

Data Averging method

I have a series of 2 columns called Amount & Percentage. Now the amount is incrementing with no linear value eg 1659,3167,4495 and I want to have a data for every 1000 that is ...
1
vote
2answers
887 views

Assessing measurement accuracy and precision

I have been asked to assess the accuracy and precision of a new measurement method (Let's call it method B). This new method is compared to an existing one (A) that has its own specifications in ...
4
votes
1answer
493 views

O(n^2) algorithm to approximate the sum of the log of the singular values of a matrix

Given an $M \times N$ matrix of rank $N$ ($M \ge N$) with $i^{th}$ singular value $\sigma_i$, does their exist an $O(M^2)$ algorithm for approximating the sum $ H =\sum_{i=1}^N \log(\sigma_i)$ with ...
3
votes
2answers
1k views

“Main” diagonal of a matrix

Hello! I'm in search of some (possibly statistical) measure for matrices. I want to classify a square matrix as having the largest numbers running along the main diagonal or along the anitdiagonal. ...
1
vote
2answers
1k views

Multiple outliers for two variable linear regression

Problem Visually, the "extreme" outliers in the following graph are somewhat obvious: http://i.imgur.com/tiSbS.png Question Given: T - Set of all temperatures Y - Set of all years ΣT - Sum of ...
3
votes
3answers
2k views

Linear Regression Coefficients W/ X, Y swapped

Let's say I have a linear regression model of the form $ y = B_x x + I_x + \epsilon $, where $B_x$ is the beta coefficient of the $x$ term, $I_x$ is the intercept term and $\epsilon$ is additive, ...
0
votes
2answers
237 views

Corruption and Recovery

Suppose we want to recover an input vector $f \in \textbf{R}^n$ from some measurements $y = Af + \varepsilon$. Now $A$ is an $m \times n$ matrix and $\varepsilon$ are some unknown errors. Is this ...
1
vote
1answer
572 views

Quantifying Aggregate Vector Strength/Vector Arithmatic

Say I have 5 vectors and I measure the similarity of each one to a fixed reference vector using cosine similarity. But now what I want to do is understand the aggregate or collective strength of these ...
2
votes
1answer
785 views

Question about orthogonal matching pursuit

Let y be a n-vector, X a n-by-p matrix of full rank (p < n) and b a p-vector, so that y = Xb + e, for some noise vector e. I am not sure how to show reduction of error in orthogonal matching ...