All Questions
Tagged with linear-algebra na.numerical-analysis
176 questions
0
votes
2
answers
97
views
Optimization algorithms for Kronecker approximation of high-dimensional covariance matrices
I'm working with a high-dimensional covariance matrix and exploring Kronecker product approximations to make it computationally manageable.
Here's the setup:
I have a graph $G$ represented by a $D\...
- 1
0
votes
0
answers
32
views
Finding measure representation for rank 2 moment matrices
Assuming the following equation has a solution, I'm interested in finding any concrete values of $x_{1},\dots x_{n},y_{1},\dots y_{n},c_{1},c_{2},R$ that fulfills it.
$$
\begin{bmatrix}
1 & 1 \\
...
- 275
4
votes
1
answer
342
views
rank of an integer valued matrix
I make some numerical experiments, involving rank of integer valued matrices of the size about $14\times 24$. As the matrix is integer valued, theoretically there should be no room for errors. However ...
- 728
1
vote
0
answers
95
views
Vandermonde-type factorization of moment matrix?
Consider $n,d \in \mathbb{N}_{>0}$, there are many functions $y:\mathbb{N}^{n} \to \mathbb{R}$. Now for simplicity, we denote $y(\alpha)$ to be $y_{\alpha}$. Let $|\alpha| = \sum_{i=1}^{n}\alpha_{i}...
- 275
0
votes
0
answers
67
views
Concentration of bilinear forms
This is a bit vague so I'll begin by indicating the motivation. I am looking for ways to [do something interesting or useful] with the self-attention in transformer models. Ultimately the self-...
- 6,962
1
vote
1
answer
184
views
Average distance between points of lower dimensional simplices in $\mathbb R^n$
Notation: By a simplex, we mean the convex hull of a finite set of distinct points in $\mathbb R^n$, which are called the vertices of the simplex. $\mathcal H^n$ will denote the $n$-dimensional ...
- 6,215
1
vote
0
answers
61
views
Discrete-to-continuum convergence of principal Fokker-Planck eigenvalues
I am looking for a reference justifying the following statement.
Let $L^n$ be any "reasonably consistent" finite-difference approximation of the Fokker-Planck operator in dimension $d=1$
$$
...
- 5,380
3
votes
1
answer
273
views
Inflection point calculation for cubic Bézier curve encounters division by zero
I've been working on finding the inflection points of a cubic Bezier curve using the method described in a paper Hain, Venkat, Racherla, and Langan - Fast, Precise Flattening of Cubic Bézier Segment ...
- 133
2
votes
1
answer
217
views
How to do LU factorization efficiently based on the factorized result added with a low-rank matrix?
Suppose a square $n\times n$, dense matrix $A^{\text{old}}$ has been factorized into $L^{\text{old}}$ and $U^{\text{old}}$ components by performing a LU decomposition $A^{\text{old}} = L^{\text{old}}U^...
- 21
0
votes
2
answers
131
views
Reshaping data vector into a matrix for deconvolution using a circulant matrix
Suppose we have a circulant matrix S made from pseudorandom binary sequence of length $N$ consisting of $0$'s or/and $1$'s. $1$ means that we can inject something for chemical analysis and $0$ means ...
- 879
2
votes
1
answer
299
views
Product of a vector by an inverse of Toeplitz matrix
It is well known that using fast Fourier transform it's possible to multiply a vector by a Toeplitz matrix $A \cdot v = w$ in $n\cdot\log(n)$ operations.
I read somewhere that also the product of a ...
- 21
1
vote
0
answers
77
views
Find a vector in the null space of a large dense matrix, where elements in the matrix are not directly accessible
I am working with Conjugate Gradient method to solve for 𝐴𝑥=𝑏, where 𝐴 is an extremely large PSD and Singular matrix. I cannot directly access the elements of 𝐴. The only thing I can do is ...
3
votes
1
answer
368
views
Linear system with sum of Kronecker products
Here and here, specific ways to address the equation in $x$, for $N=2$, are given:
$$\sum_{i=1}^N (A_i\otimes B_i)x=c$$
Is anything know about the case $N>2$?
I am looking in fact for an efficient ...
- 235
0
votes
0
answers
108
views
Solving a nonlinear equation maybe with Lambert W function
Can you please help me solve the following nonlinear equation?
\begin{equation}
\boldsymbol{z} \odot\left(\boldsymbol{\Gamma}^{\top} \boldsymbol{y}\right)=(\beta)^{\frac{1}{m-1}}\left(\frac{m-1}{...
- 11
6
votes
1
answer
913
views
Resultant of linear combinations of Chebyshev polynomials of the second kind
The Chebyshev polynomial $U_n(x)$ of the second kind is characterized by
$$
U_n(\cos\theta)=\frac{\sin(n+1)\theta}{\sin(\theta)}.
$$
It seems that
$$\operatorname*{Res}_x \left( U_n(x)+tU_{n-1}(x),\...
- 437
3
votes
1
answer
175
views
Is there a classical textbook/reference on numerical discretization schemes?
I found that it is relatively easy to find a book that discusses Euler discretization or Runge-Kutta discretization, but I am not aware of one that is well-known and/or common knowledge (i.e., field-...
- 479
3
votes
0
answers
122
views
Preconditioners for $Ax=y$ that rely on hierarchical statistical modeling
Solving $Ax=y$ exactly can be done as:
fit a linear autoregressive model by treating rows of $A$ as data
apply this model to $A^T y$
Imperfect predictive model corresponds to an approximate inverse ...
- 2,252
1
vote
1
answer
59
views
Does norm of discrepancy decrease monotonously in CGLS/CGNR
I am the author of the package for tomographic reconstruction https://github.com/kulvait/KCT_cbct I have implemented CGLS/CGNR , algorithm which applies conjugate gradients on normal equation
$$
A^\...
- 151
0
votes
1
answer
123
views
Explicit expression of Padé–Hermite approximant of type I
It is well known that the Padé approximants $(P,Q)$ of an analytic function in the neighborhood of $0$ can be expressed as a quotient of Hankel determinants built on the coefficients of the function $...
- 3,998
0
votes
0
answers
114
views
Degeneracies in linear combination of tensor product of Pauli matrices
Let $P_i \in \{I,X,Y,Z\}^{\otimes n} $, that is $P_i = \bigotimes_{i =1 }^n \sigma_i$ with $\sigma_i \in \{I,X,Y,Z\}$, where
$$
I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \hspace{1cm} X =...
- 131
1
vote
0
answers
198
views
Complexity of singular value decomposition using matrix multiplication oracles
Suppose I have an $n\times m$ real matrix $A$, $n\ll m$ with full row rank $(\mathrm{rank}(A) = n)$. I have an oracle that can compute $Ax$ or $A^T y$ for any $x\in \mathbb{R}^m, y\in \mathbb{R}^n$. ...
- 662
4
votes
1
answer
720
views
Singular value decomposition of truncated discrete Fourier transform matrix
Let $\mathbf{F}$ be a discrete Fourier transform (DFT) matrix such that
\begin{align}
F_{m,n}=e^{-j2\pi(m-1)(n-1)/N},\quad m,n=1,\ldots,N.
\end{align}
What we can say about the singular value ...
- 287
4
votes
1
answer
408
views
Proof of Levinson-Durbin algorithm
Is there any article or reference book with a full proof of the Levinson-Durbin algorithm used for solving linear system with a Toeplitz matrix ?
- 41
2
votes
0
answers
71
views
What are desirable properties that data should satisfy to reasonably use the dynamic mode decomposition?
In the dynamic mode decomposition, we consider a sequence of data vectors $\{z_0, \dots, z_m\}$ where $z_k \in \mathbb{R}^n$ for all $n$. We assume that the data satisfies the linear relationship $z_{...
- 263
0
votes
0
answers
135
views
Is there an efficient algorithm to project a vector onto the eigenbasis of a symmetric matrix?
Let $H$ be a symmetric matrix over $\mathbb R^n$. Given some vector $u$, I would like to express $u$ in the eigenbasis for $H$. Can this be done efficiently, perhaps using some kind of iterative ...
- 623
0
votes
0
answers
227
views
Decomposition of symmetric block matrix
I came across this question and got really interested about it. There, the OP asks whether is possible to decompose a $2n \times 2n$ block matrix:
$$ \begin{pmatrix}
X & I \\
I & Y
\end{...
0
votes
1
answer
266
views
Using QR or SVD to sum up finite number of matrices
Problem
I was wondering if there are any theoretical results that tackle the following problem:
Construct the following matrices $\mathbf{\mathcal{S}_{1}},\mathbf{\mathcal{S}_{2}},\ldots,\mathbf{\...
0
votes
0
answers
57
views
Numerically finding matrix approximation by lower-dimensional "pseudo-similar" matrix
Consider an $N\times N$ (real or complex) matrix $A$, and some $n<N$. Is there a good numerical algorithm that finds the set consisting of an $n\times n$ matrix $B$, an $n\times N$ matrix $I$, and ...
- 3,001
0
votes
1
answer
230
views
Solution of complex linear system
In Brubeck, Nakatsukasa, and Trefethen - Vandermonde with Arnoldi (example 3) they solve the following linear system:
$$\operatorname{Re}\left(\begin{array}{ccc}1 & \cdots & z_{1}^{n} \\ 1 &...
- 45
2
votes
1
answer
241
views
How to solve this set of equations as efficiently as possible (with "efficiently" measured in FLOPS)?
The system of equations is the following:
$$
\Gamma_i^{\ -1} = \sum_{i=1}^nA_{ij}\Gamma_j,
$$
where $\Gamma = (\Gamma_i)$ is a vector of size $n$ and $A$ is a matrix of size $n\times n$, with $n \gt ...
- 21
1
vote
1
answer
146
views
Norm of a matrix with clustered eigenvalues
On page 271 of Trefethen and Bau's Numerical Linear Algebra, it is constructed a matrix
$$A=2I_{m\times m}+0.5\cdot\frac{\text{rand}(m)}{\sqrt{m}}$$
for $m=200$, where rand(m) is an array with $m\...
- 13
1
vote
2
answers
202
views
Robust estimation of $Ax=b$
Problem setting :
$ \underset{x}{\text{min}} \|Ax-b\|$, where $A \in \mathcal{R}^{m \times n}, m\gg n $, full rank.
L1 loss is used for robust estimation using IRLS. The corresponding equation to ...
- 21
6
votes
1
answer
222
views
Computing $(AA\otimes BB + AB \otimes BA)^{-1}$
Can anyone suggest a way to numerically compute the following matrix vector product?
$$u=A^{-1}b=(AA\otimes BB + AB \otimes BA)^{-1}\operatorname{vec}(C)$$
Here $AA,BB,AB,BA$ and $C$ are $d\times d$ ...
- 2,252
11
votes
2
answers
1k
views
Existence of sparse LU decomposition of sparse matrix
Let $A$ be a sparse matrix over some field. I would like to know about the existence of LU decompositions so that $L,U$ are both sparse.
More precisely, let $A$ be an $N$-by-$N$ matrix. Suppose each ...
- 547
2
votes
0
answers
618
views
block diagonal approximation of (SPD) matrix
I am interested in approximating a symmetric matrix in a block diagonal form, i.e. compute just some entries of the matrix located in blocks around the diagonal. Are there any theoretical guarantees ...
- 335
3
votes
1
answer
836
views
Solving multilinear equations
Let $N=\{1,2,\ldots,n\}$. Suppose we are given $n$ equations, with each equation taking the form $\sum_{A\subseteq N}\left(c_A \prod_{i\in A}x_i \right) = 0$, where each $c_A$ is a real number ...
- 239
6
votes
2
answers
1k
views
Is the matrix positive definite given the Gauss-Seidel method converges?
I know that the Gauss-Seidel method converges given that the matrix you want to solve is symmetric positive definite. However, I'm wondering if the "converse" of the statement is true. Namely, if $A$ ...
- 205
1
vote
1
answer
323
views
How can I find minimum and maximum eigenvalue of non-positive define matrix [closed]
There is a power iteration method, but it only returns the greatest(in absolute value) eigenvalue of matrix. So when we have negative eigenvalues it'll give wrong results.
Is there any method, which ...
- 11
0
votes
0
answers
159
views
How to solve a non-local self-consistent equation
I have been struggling lately with solving numerically an equation of the form:
$$ g(x\pm x_{0}) = F[ g(x) ] $$
where $g(x)$ is a matrix satisfying the condition $g(x\to\pm\infty)=0$. My question is ...
- 123
1
vote
0
answers
126
views
Fastest way to calculate the eigenvalues of a product of two Toeplitz matrices
I have the following problem:
I need to find the fastest way to calculate the eigenvalues of a matrix that is the product of two Toeplitz matrices. $B = A U$.
The first is a regular Toeplitz matrix $A$...
- 11
1
vote
0
answers
448
views
Smallest eigenvalue for large kernel matrix
I am interested in the the asymptotics of the minimum eigenvalue $\lambda_n^n$ of a class of kernel matrix $P = [ K(x_i - x_j) ]_{i,j}$, with $x_i$ equally spaced in the unit cube of $\mathbb{R}^d$.
...
- 151
5
votes
1
answer
644
views
A conjecture about the submatrix of orthogonal matrix
Let $U$ be an $n\times n$ orthogonal matrix, i.e. $U\in\mathbb{R}^{n \times n}$. For any non-empty ordered sets $S_1,S_2\subset\{1,2,...,n\}$, define $U_{S_1S_2}$ to be an $|S_1|\times|S_2|$ submatrix ...
- 1,495
4
votes
1
answer
413
views
Lipschitz property of matrix function only depending on singular values
Let $f$ be a function from $\mathbb{R}^{n\times n}$ to $\mathbb{R}$ such that there exists another symmetric function $g$ (invariant under permutation of coordinates) from $\mathbb{R}^{n}$ to $\mathbb{...
- 1,495
4
votes
2
answers
3k
views
Methods of solving linear system of equations, how to select the appropriate method
A linear system of equations Ax=b can be solved using various methods, namely, inverse method, Gauss/Gauss-Jordan elimination, LU factorization, EVD (Eigenvalue Decomposition), and SVD (Singular Value ...
- 141
2
votes
1
answer
90
views
Can a Multilayer Perceptron fit any binary function?
Consider a perceptron $F(x) = \phi(x * w - b), \ x \in \mathbb{R}^n,$ (with Heaviside activation function $\phi$) and a dataset consisting of a finite subset $\Omega \subseteq \mathbb{R}^n$ with ...
- 23
6
votes
1
answer
218
views
Any convergence rule for ${\mathbf X}_k={\mathbf A}{\mathbf X}_{k-1}{\mathbf B}$?
We know iteration ${\mathbf X}_k=\mathbf{A}{\mathbf X}_{k-1}$ converges if the spectral radius of $\mathbf A$ is smaller than 1 (see here). Is there any known rule for iteration ${\mathbf X}_k={\...
- 272
0
votes
1
answer
540
views
Computing spectrum of convex combination of SPD matrices given individual spectral decompositions
Given the spectral decompositions of a non-commuting collection of symmetric positive definite $N\times N$ matrices $$\left\{ K_{i}\right\} _{i=1}^{M}, U_{i}D_{i}U_{i}^{T}=K_{i},\quad i=1,\dots,M,$$ ...
- 133
7
votes
1
answer
356
views
Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?
(Posted this on math.stackexchange and cross.correlated over more than a week ago, but didn't get an answer, and this is a question in my research so this seems like it might have been the better ...
- 141
1
vote
1
answer
102
views
Polynomial Eigenvalue Problem with few non-zero coefficients
Let us define a diagonal matrix $\mathbf{D}(\lambda) = diag(\lambda^{m_1}, \dots, \lambda^{m_n})$ with $\lambda\in\mathbb{C}$ and positive integers $m_1, \dots, m_n$. The generalized characteristic ...
- 909
4
votes
1
answer
1k
views
Inverse of matrix with blocks of ones
It seems that there is a nice inverse for matrices that can be written as a diagonal matrix plus a symmetric matrix consisting of scaled blocks of ones.
Consider a real matrix of the form:
$$\begin{...
- 1,068