0
votes
0answers
28 views

Cholesky decomposition of a large covariance matrix

I have a tricky problem concerning a covariance matrix cholesky decomposition. What I need is to obtain the cholesky decomposition of the estimated variance matrix of the set of samples stored in a ...
2
votes
0answers
95 views

Hilbert Schmidt Operators and the Conditional Expectation Operator

Consider the function $\text{E}_W: L_2(\mathbb{R},P_X) \mapsto L_2(\mathbb{R},P_W)$ where $P_X$ and $P_W$ are two different probability measures. They are related in such a way that if $f_X$, $f_W$ ...
0
votes
0answers
48 views

Linear Bounds on estimation error

Consider a markov chain on discrete state space $\mathbb{S} = \left\{1,2,..,S \right\}$, with transition probability matrix defined as $A = [a_{ij}]_{S \times S}$ where $a_{ij} = ...
1
vote
0answers
160 views

Incoherence of the row/column span

Due to V.Chandrasekaran., et al‎ (p.11) : In general for any $k$-dimensional subspace of $A_{n×n}$ we have that: $$\sqrt{(k/n)} \leq incoherence(A)\leq 1$$ where the lower bound is achieved (for ...
4
votes
1answer
94 views

Weak ergodicity of nonhomogenous products of 0-1 matrices

Here is a question which probably has a negative answer, but I couldn't find any literature directly on it. Let $(A_n)$ be a sequence of rectangular 0-1 matrices (that is, the entries are restricted ...
4
votes
3answers
230 views

Online estimation of covariance matrix

I am trying to dynamically estimate the (low-dimensional) covariance matrix ${\mathbb E}[{\bf x}_t{\bf x}_t^\top]$ of a stream of data points ${\bf x}_t\in{\mathbb R}^N$ online, without any memory. ...
4
votes
0answers
203 views

Probability distribution function for singular value sum of Gaussian random matrix

Let $\mathbf{X}$ be an $N \times N$ random matrix with IID Gaussian entries. They can be standard normal, but $N$ is not large: that is $N$ $<$ 6, typically. Call its singular value decomposition ...
1
vote
1answer
70 views

Estimating the relation between the covariance of a vector and a monotone function of the same vector

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as $$ ...
-1
votes
1answer
264 views

Rank of covariance matrix whose diagonal elements are same [closed]

Suppose A is a covariance matrix whose diagonal elements are same, i.e. $A_{1,1}=A_{2,2}=\cdots=A_{N,N}$, can we conclude that A is full rank? Suppose the absolute values of the off-diagonal elements ...
8
votes
2answers
357 views

Rescaling positive definite matrices to force a unit eigenvector

Hello, Let $X'X$ be a positive definite matrix and let $\mathbf{1}$ denote the vector of ones. I'm hoping to construct a positive, diagonal matrix $W$ such that $$(W X'X W) \mathbf{1} = ...
2
votes
0answers
163 views

Expectation of a multivariate Gaussian over a plane

For a vector $X$ which follows a multinomial Gaussian distribution $N(\vec{0},\Sigma)$, a given vector $b$, and a known scalar value $c$, I would like to calculate the expectation : $E[X|X^Tb = c]$ ...
5
votes
1answer
319 views

Computing the correlation between two vectors without divulging them

Alice and Bob respectively know a vector of $N$ real numbers $u$ and $v$. They would both like to know $\rho = \langle u,v \rangle/N$ but Alice does not want Bob to gain anymore information about $u$ ...
-1
votes
2answers
125 views

Multiple Linear Regression Estimation without full recalc [closed]

Ok, so I am running a classic linear regression where betahat = (X'X)^-1X'y Due to performance issues, I would like to estimate betahat with an additional data point (x1,x2,x3,x4,...,y) without ...
3
votes
1answer
454 views

Determinant of an updated Covariance matrix

I am faced with the following problem : Originally (at time 0) I have a number of data samples $x^0_{1...n}$ (normalised : $E[x] = 0, Var[x] = 1$) from which I have calculated the covariance matrix ...
4
votes
5answers
885 views

median of matrices

I have $n$ positive definite Hermitian matrices $M_n$ and I want to define and compute their median. These matrices correspond to independent estimations of a covariance matrix in the presence of ...
1
vote
1answer
259 views

How many zero-constraints can be added to a subspace-restricted matrix before no solution exists?

I'm trying to develop an estimator for the concentration matrix of a Gaussian Graphical Model. I've become stuck in trying to find conditions for the estimator to exist. I have a sufficient ...
1
vote
1answer
571 views

Shortcut for variance of vectors

I have a dataset of vectors and need to find the sum of squares or variance based on the euclidean distance between the vectors. I can do this by finding the "average" vector (by calculating the ...
2
votes
1answer
293 views

Derivation of Iteration Rules

Suppose we are given a matrix $V$ and our goal is to find non-negative matrices $W$ and $H$ such that $V \approx WH$. So we want to minimize $K(V || WH)$ (Kullback-Leibler Divergence) where $$K(V||WH) ...
0
votes
1answer
900 views

Convergence of Eigenvalues

Suppose we have a matrix $A_n = \frac{1}{n}\sum_{i=1}^nX_i X_i^T$, where $X_i$ is a $p$-dimensional random-vector. We also know that $E(XX^T) = \Sigma_{p \times p}$. Let us denote the $j$-th largest ...
1
vote
0answers
173 views

What is the MP pseudoinverse's role in statistical learning and Self-Organizing Maps?

During a discussion in our lab last month, a professor mentioned to me that the behavior of Self-Organizing Maps can be described in terms of repeated applications of the Moore-Penrose psuedoinverse, ...
4
votes
1answer
824 views

Monotonicity of the hard EM algorithm.

Consider the problem where we want to find a maximum likelihood estimate of $\theta$, given $X$ and $$P_\theta(Y) = \sum_z P_\theta(Y,x)$$ where $x$ is a latent variable. I know that the soft EM ...
0
votes
2answers
252 views

Nonexistence of projection

Following is a doubt I got when I was reading Csiszar's Annals of Statistics paper "Why least squares and maximum entropy: An axiomatic approach to inference for linear inverse problems". He comes up ...
1
vote
0answers
340 views

Bounding point-wise maximum of the absolute difference of two convex functions

Let $\Delta: R \times R \rightarrow R_{+}$ be a positive and convex function (convex in, say, both the arguments) called the loss function. Let $x \in R^d$. Moreover, let $H_1,...,H_r$ be sets of ...
1
vote
1answer
506 views

Sequential sampling of Gaussian and von Mises-Fisher Random Variable

I don't find any article discussing this problem, so I dare to ask it. Suppose we are dealing with a data $x_0 \in \mathbb{R}$ and a function $f:\mathbb{R} \to \mathbb{R}$. Say we repeatedly apply ...
1
vote
2answers
791 views

Inequality-constrained linear-regression, what is the covariance of the estimator?

If you do a linear regression: $||Ax - e ||^2$, where e is iid Gaussian, mean 0 and variance 1, then your answer is $x_{hat} = (A' A)^{-1} (A' * e)$ and the covariance of $x_{hat}$ is $(A' A)^{-1}$ ...
0
votes
1answer
178 views

Data Averging method

I have a series of 2 columns called Amount & Percentage. Now the amount is incrementing with no linear value eg 1659,3167,4495 and I want to have a data for every 1000 that is ...
1
vote
2answers
869 views

Assessing measurement accuracy and precision

I have been asked to assess the accuracy and precision of a new measurement method (Let's call it method B). This new method is compared to an existing one (A) that has its own specifications in ...
4
votes
1answer
491 views

O(n^2) algorithm to approximate the sum of the log of the singular values of a matrix

Given an $M \times N$ matrix of rank $N$ ($M \ge N$) with $i^{th}$ singular value $\sigma_i$, does their exist an $O(M^2)$ algorithm for approximating the sum $ H =\sum_{i=1}^N \log(\sigma_i)$ with ...
3
votes
2answers
1k views

“Main” diagonal of a matrix

Hello! I'm in search of some (possibly statistical) measure for matrices. I want to classify a square matrix as having the largest numbers running along the main diagonal or along the anitdiagonal. ...
1
vote
2answers
1k views

Multiple outliers for two variable linear regression

Problem Visually, the "extreme" outliers in the following graph are somewhat obvious: http://i.imgur.com/tiSbS.png Question Given: T - Set of all temperatures Y - Set of all years ΣT - Sum of ...
3
votes
3answers
2k views

Linear Regression Coefficients W/ X, Y swapped

Let's say I have a linear regression model of the form $ y = B_x x + I_x + \epsilon $, where $B_x$ is the beta coefficient of the $x$ term, $I_x$ is the intercept term and $\epsilon$ is additive, ...
0
votes
2answers
237 views

Corruption and Recovery

Suppose we want to recover an input vector $f \in \textbf{R}^n$ from some measurements $y = Af + \varepsilon$. Now $A$ is an $m \times n$ matrix and $\varepsilon$ are some unknown errors. Is this ...
1
vote
1answer
570 views

Quantifying Aggregate Vector Strength/Vector Arithmatic

Say I have 5 vectors and I measure the similarity of each one to a fixed reference vector using cosine similarity. But now what I want to do is understand the aggregate or collective strength of these ...
2
votes
1answer
781 views

Question about orthogonal matching pursuit

Let y be a n-vector, X a n-by-p matrix of full rank (p < n) and b a p-vector, so that y = Xb + e, for some noise vector e. I am not sure how to show reduction of error in orthogonal matching ...