All Questions
Tagged with linear-algebra na.numerical-analysis
176 questions
0
votes
2
answers
97
views
Optimization algorithms for Kronecker approximation of high-dimensional covariance matrices
I'm working with a high-dimensional covariance matrix and exploring Kronecker product approximations to make it computationally manageable.
Here's the setup:
I have a graph $G$ represented by a $D\...
- 20.2k
0
votes
1
answer
123
views
Explicit expression of Padé–Hermite approximant of type I
It is well known that the Padé approximants $(P,Q)$ of an analytic function in the neighborhood of $0$ can be expressed as a quotient of Hankel determinants built on the coefficients of the function $...
CommunityBot
- 1
0
votes
0
answers
32
views
Finding measure representation for rank 2 moment matrices
Assuming the following equation has a solution, I'm interested in finding any concrete values of $x_{1},\dots x_{n},y_{1},\dots y_{n},c_{1},c_{2},R$ that fulfills it.
$$
\begin{bmatrix}
1 & 1 \\
...
- 275
4
votes
1
answer
342
views
rank of an integer valued matrix
I make some numerical experiments, involving rank of integer valued matrices of the size about $14\times 24$. As the matrix is integer valued, theoretically there should be no room for errors. However ...
- 3,319
1
vote
0
answers
95
views
Vandermonde-type factorization of moment matrix?
Consider $n,d \in \mathbb{N}_{>0}$, there are many functions $y:\mathbb{N}^{n} \to \mathbb{R}$. Now for simplicity, we denote $y(\alpha)$ to be $y_{\alpha}$. Let $|\alpha| = \sum_{i=1}^{n}\alpha_{i}...
- 275
2
votes
1
answer
241
views
How to solve this set of equations as efficiently as possible (with "efficiently" measured in FLOPS)?
The system of equations is the following:
$$
\Gamma_i^{\ -1} = \sum_{i=1}^nA_{ij}\Gamma_j,
$$
where $\Gamma = (\Gamma_i)$ is a vector of size $n$ and $A$ is a matrix of size $n\times n$, with $n \gt ...
- 3,065
0
votes
0
answers
67
views
Concentration of bilinear forms
This is a bit vague so I'll begin by indicating the motivation. I am looking for ways to [do something interesting or useful] with the self-attention in transformer models. Ultimately the self-...
- 6,962
8
votes
1
answer
7k
views
Upper bound on largest eigenvalue of a real symmetric $n \times n$ matrix with all main diagonal entries positive, everywhere else nonpositive
Is there a good analytic upper bound on the largest eigenvalue of a real symmetric n*n matrix with all main diagonal entries strictly positive, all other entries <=0 with typically many of them ...
1
vote
1
answer
184
views
Average distance between points of lower dimensional simplices in $\mathbb R^n$
Notation: By a simplex, we mean the convex hull of a finite set of distinct points in $\mathbb R^n$, which are called the vertices of the simplex. $\mathcal H^n$ will denote the $n$-dimensional ...
- 128k
1
vote
0
answers
61
views
Discrete-to-continuum convergence of principal Fokker-Planck eigenvalues
I am looking for a reference justifying the following statement.
Let $L^n$ be any "reasonably consistent" finite-difference approximation of the Fokker-Planck operator in dimension $d=1$
$$
...
- 5,380
22
votes
9
answers
17k
views
Fast evaluation of polynomials
Hello everybody !
I was reading a book on geometry which taught me that one could compute the volume of a simplex through the determinant of a matrix, and I thought (I'm becoming a worse computer ...
3
votes
1
answer
273
views
Inflection point calculation for cubic Bézier curve encounters division by zero
I've been working on finding the inflection points of a cubic Bezier curve using the method described in a paper Hain, Venkat, Racherla, and Langan - Fast, Precise Flattening of Cubic Bézier Segment ...
- 7,226
2
votes
1
answer
217
views
How to do LU factorization efficiently based on the factorized result added with a low-rank matrix?
Suppose a square $n\times n$, dense matrix $A^{\text{old}}$ has been factorized into $L^{\text{old}}$ and $U^{\text{old}}$ components by performing a LU decomposition $A^{\text{old}} = L^{\text{old}}U^...
- 4,219
10
votes
1
answer
4k
views
Special considerations when using the Woodbury matrix identity numerically
Are there any special considerations when using the Woodbury matrix identity numerically? What is the best metric for numerical stability in this case? Can anyone point me to a good reference?
The ...
- 103
0
votes
2
answers
131
views
Reshaping data vector into a matrix for deconvolution using a circulant matrix
Suppose we have a circulant matrix S made from pseudorandom binary sequence of length $N$ consisting of $0$'s or/and $1$'s. $1$ means that we can inject something for chemical analysis and $0$ means ...
- 1,245
2
votes
1
answer
299
views
Product of a vector by an inverse of Toeplitz matrix
It is well known that using fast Fourier transform it's possible to multiply a vector by a Toeplitz matrix $A \cdot v = w$ in $n\cdot\log(n)$ operations.
I read somewhere that also the product of a ...
4
votes
1
answer
408
views
Proof of Levinson-Durbin algorithm
Is there any article or reference book with a full proof of the Levinson-Durbin algorithm used for solving linear system with a Toeplitz matrix ?
CommunityBot
- 1
2
votes
1
answer
276
views
Possible pathological properties of positive definite matrix
Suppose $A$ is a positive definite matrix such that$$ I \preceq A \preceq 1.01I.$$ Is it possible that $\sum\limits_{i=1}^n A_{1i}$ can be arbitrarily large?
0
votes
2
answers
321
views
Solving sparse linear least squares or a positive definite 5-band matrix system fast
I want to quickly solve the following linear least-squares problem
$$\min_{x \in \mathbb{R}^n} \left\| A x - b \right\|_2^2$$
with a special sparse structure where each row in $A$ has only up to $4$ ...
1
vote
0
answers
77
views
Find a vector in the null space of a large dense matrix, where elements in the matrix are not directly accessible
I am working with Conjugate Gradient method to solve for 𝐴𝑥=𝑏, where 𝐴 is an extremely large PSD and Singular matrix. I cannot directly access the elements of 𝐴. The only thing I can do is ...
- 66.3k
3
votes
1
answer
369
views
Linear system with sum of Kronecker products
Here and here, specific ways to address the equation in $x$, for $N=2$, are given:
$$\sum_{i=1}^N (A_i\otimes B_i)x=c$$
Is anything know about the case $N>2$?
I am looking in fact for an efficient ...
- 15.4k
1
vote
1
answer
59
views
Does norm of discrepancy decrease monotonously in CGLS/CGNR
I am the author of the package for tomographic reconstruction https://github.com/kulvait/KCT_cbct I have implemented CGLS/CGNR , algorithm which applies conjugate gradients on normal equation
$$
A^\...
- 151
3
votes
4
answers
4k
views
Fast multiplication of constant symmetric positive-definite matrix and vector.
Consider the matrix $H=H^T$, $H>0$, $H \in R^{n \times n}$, and the vector $v \in R^n$. In a numerical algorithm, I need to compute the product $b = Hv$. Right now I am following the naive approach:...
- 1
6
votes
1
answer
913
views
Resultant of linear combinations of Chebyshev polynomials of the second kind
The Chebyshev polynomial $U_n(x)$ of the second kind is characterized by
$$
U_n(\cos\theta)=\frac{\sin(n+1)\theta}{\sin(\theta)}.
$$
It seems that
$$\operatorname*{Res}_x \left( U_n(x)+tU_{n-1}(x),\...
- 114k
0
votes
0
answers
108
views
Solving a nonlinear equation maybe with Lambert W function
Can you please help me solve the following nonlinear equation?
\begin{equation}
\boldsymbol{z} \odot\left(\boldsymbol{\Gamma}^{\top} \boldsymbol{y}\right)=(\beta)^{\frac{1}{m-1}}\left(\frac{m-1}{...
- 11
3
votes
1
answer
175
views
Is there a classical textbook/reference on numerical discretization schemes?
I found that it is relatively easy to find a book that discusses Euler discretization or Runge-Kutta discretization, but I am not aware of one that is well-known and/or common knowledge (i.e., field-...
- 6,045
3
votes
0
answers
122
views
Preconditioners for $Ax=y$ that rely on hierarchical statistical modeling
Solving $Ax=y$ exactly can be done as:
fit a linear autoregressive model by treating rows of $A$ as data
apply this model to $A^T y$
Imperfect predictive model corresponds to an approximate inverse ...
- 2,252
0
votes
0
answers
114
views
Degeneracies in linear combination of tensor product of Pauli matrices
Let $P_i \in \{I,X,Y,Z\}^{\otimes n} $, that is $P_i = \bigotimes_{i =1 }^n \sigma_i$ with $\sigma_i \in \{I,X,Y,Z\}$, where
$$
I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \hspace{1cm} X =...
- 63.9k
1
vote
0
answers
198
views
Complexity of singular value decomposition using matrix multiplication oracles
Suppose I have an $n\times m$ real matrix $A$, $n\ll m$ with full row rank $(\mathrm{rank}(A) = n)$. I have an oracle that can compute $Ax$ or $A^T y$ for any $x\in \mathbb{R}^m, y\in \mathbb{R}^n$. ...
- 63.9k
10
votes
5
answers
8k
views
Shifted QR algorithm—why does the shift help?
I read that a way to speed up the convergence rate of the QR algorithm is to shift the target
matrix. It is not so clear to me why this helps. The convergence rate depends on the
minimum gap between ...
- 4,943
4
votes
1
answer
721
views
Singular value decomposition of truncated discrete Fourier transform matrix
Let $\mathbf{F}$ be a discrete Fourier transform (DFT) matrix such that
\begin{align}
F_{m,n}=e^{-j2\pi(m-1)(n-1)/N},\quad m,n=1,\ldots,N.
\end{align}
What we can say about the singular value ...
- 188k
0
votes
1
answer
266
views
Using QR or SVD to sum up finite number of matrices
Problem
I was wondering if there are any theoretical results that tackle the following problem:
Construct the following matrices $\mathbf{\mathcal{S}_{1}},\mathbf{\mathcal{S}_{2}},\ldots,\mathbf{\...
CommunityBot
- 1
2
votes
0
answers
71
views
What are desirable properties that data should satisfy to reasonably use the dynamic mode decomposition?
In the dynamic mode decomposition, we consider a sequence of data vectors $\{z_0, \dots, z_m\}$ where $z_k \in \mathbb{R}^n$ for all $n$. We assume that the data satisfies the linear relationship $z_{...
- 263
0
votes
0
answers
135
views
Is there an efficient algorithm to project a vector onto the eigenbasis of a symmetric matrix?
Let $H$ be a symmetric matrix over $\mathbb R^n$. Given some vector $u$, I would like to express $u$ in the eigenbasis for $H$. Can this be done efficiently, perhaps using some kind of iterative ...
- 623
0
votes
3
answers
7k
views
Find an $N$-dimensional vector orthogonal to a given vector
I'm writing an eigensolver and I'm trying to generate a guess for the next iteration in the solve that is orthogonal to to all known eigenvectors calculated thus far. This means that if I have only ...
- 1
0
votes
0
answers
228
views
Decomposition of symmetric block matrix
I came across this question and got really interested about it. There, the OP asks whether is possible to decompose a $2n \times 2n$ block matrix:
$$ \begin{pmatrix}
X & I \\
I & Y
\end{...
11
votes
2
answers
1k
views
Existence of sparse LU decomposition of sparse matrix
Let $A$ be a sparse matrix over some field. I would like to know about the existence of LU decompositions so that $L,U$ are both sparse.
More precisely, let $A$ be an $N$-by-$N$ matrix. Suppose each ...
- 3,741
0
votes
0
answers
57
views
Numerically finding matrix approximation by lower-dimensional "pseudo-similar" matrix
Consider an $N\times N$ (real or complex) matrix $A$, and some $n<N$. Is there a good numerical algorithm that finds the set consisting of an $n\times n$ matrix $B$, an $n\times N$ matrix $I$, and ...
- 3,001
0
votes
1
answer
230
views
Solution of complex linear system
In Brubeck, Nakatsukasa, and Trefethen - Vandermonde with Arnoldi (example 3) they solve the following linear system:
$$\operatorname{Re}\left(\begin{array}{ccc}1 & \cdots & z_{1}^{n} \\ 1 &...
- 45
1
vote
1
answer
146
views
Norm of a matrix with clustered eigenvalues
On page 271 of Trefethen and Bau's Numerical Linear Algebra, it is constructed a matrix
$$A=2I_{m\times m}+0.5\cdot\frac{\text{rand}(m)}{\sqrt{m}}$$
for $m=200$, where rand(m) is an array with $m\...
1
vote
2
answers
202
views
Robust estimation of $Ax=b$
Problem setting :
$ \underset{x}{\text{min}} \|Ax-b\|$, where $A \in \mathcal{R}^{m \times n}, m\gg n $, full rank.
L1 loss is used for robust estimation using IRLS. The corresponding equation to ...
- 1,517
6
votes
1
answer
222
views
Computing $(AA\otimes BB + AB \otimes BA)^{-1}$
Can anyone suggest a way to numerically compute the following matrix vector product?
$$u=A^{-1}b=(AA\otimes BB + AB \otimes BA)^{-1}\operatorname{vec}(C)$$
Here $AA,BB,AB,BA$ and $C$ are $d\times d$ ...
- 3,065
4
votes
2
answers
948
views
Numerically solving for pseudo inverse of non-squared Vandermonde matrix
I have a linear system to solve, set up as:
$\bf{Ax}=\bf{b}$
with a non-squared matrix A,
$
\bf{A}=
\begin{bmatrix}
1 & A_{1} & A_{1}^2 & \cdots & A_{1}^n \\
1 & A_{2} & A_{...
- 384
2
votes
1
answer
728
views
Cubic spline of a two-variable function
So, I am aware of how to (both iteratively and using a linear equation) compute the cubic spline of a one-variable function with $m$ control points. However, I am not sure how to do any type of spline ...
- 3,065
1
vote
1
answer
279
views
Splines linearly independent
Let $N_1:=\chi_{[0,1]}$ be defined as this characteristic function and $N_n:=N_{n-1}*N_1$ then this leads to polynomials with support $[0,n]$. These splines are well-studied click for wikipedia My ...
- 3,065
2
votes
0
answers
618
views
block diagonal approximation of (SPD) matrix
I am interested in approximating a symmetric matrix in a block diagonal form, i.e. compute just some entries of the matrix located in blocks around the diagonal. Are there any theoretical guarantees ...
- 3,065
7
votes
2
answers
3k
views
Factorizing a block symmetric matrix
Let $X,Y\in\mathbb{R}^{n\times n}$ be symmetric matrices. You may assume that $X$ is positive semidefinite and $Y$ negative semidefinite, if needed, but not that they are invertible.
I would like to ...
3
votes
1
answer
836
views
Solving multilinear equations
Let $N=\{1,2,\ldots,n\}$. Suppose we are given $n$ equations, with each equation taking the form $\sum_{A\subseteq N}\left(c_A \prod_{i\in A}x_i \right) = 0$, where each $c_A$ is a real number ...
- 16.9k
6
votes
2
answers
1k
views
Is the matrix positive definite given the Gauss-Seidel method converges?
I know that the Gauss-Seidel method converges given that the matrix you want to solve is symmetric positive definite. However, I'm wondering if the "converse" of the statement is true. Namely, if $A$ ...
- 205
1
vote
1
answer
324
views
How can I find minimum and maximum eigenvalue of non-positive define matrix [closed]
There is a power iteration method, but it only returns the greatest(in absolute value) eigenvalue of matrix. So when we have negative eigenvalues it'll give wrong results.
Is there any method, which ...
- 96.4k