Search Results

This is a bit vague so I'll begin by indicating the motivation. I am looking for ways to [do something interesting or useful] with the self-attention in transformer models. Ultimately the self-attenti …

Consider the following two polynomials: $$ g=x^3 - x^2 - (c + 2)x + c $$ and $$ h=x^3 - x^2 - cx + c $$ The roots of $h$ are $1$ and $\pm \sqrt{c}$. I am interested in obtaining the roots of $g$, …

I am trying to get an intuitive feel for why Padé approximation works so well. Given a truncated Taylor/Maclaurin series it "extrapolates" it beyond the radius of convergence. But what I can't grasp i …

Consider this matrix: $Z=\begin{bmatrix}23.9 & -7 & -17 \\\\ -7 & 23.9 & -17 \\\\ -17 & -17 & 33.9 \end{bmatrix}$ Its inverse is entrywise negative (you can check...) and quite small in absolute val …

I have an equation of the form $\sum_{i=1}^{m}{\frac{1}{a_{i}-x}}=\sum_{j=1}^{n}{\frac{1}{b_{j}-x}}$ and would like to express $x$ as a an approximate explicit function of the $a_{i},b_{j},m,n$. Have …

Is there a practical way to compute the permanent of a large ($91 \times 91$) $(0,1)$ matrix? I have tried to use the matlab function written by Luke Winslow which works great for smaller matrices bu …

Suppose that $A$ is a Hermitian matrix and that $u,v$ are two vectors. Is there some known function $\kappa(A)$ so that $||u-v|| \leq \kappa(A) |\frac{u^{\*}Au}{u^{\*}u}-\frac{v^{\*}Av}{v^{\*}v}|$? U …

Suppose that $B$ is a Hermitian matrix with one known eigenpair $(\lambda,v)$. (assume its the smallest or largest pair, if you like). Form the rank one update $B+\rho bb^{T}$. Now I'm interested in …

Suppose that we are given an equation $Ax=b$. The minimum least-squares solution is of course $x_{m}=A^{\dagger}b$. What I want to know is whether there are known bounds on $||x-x_{m}||$. In the probl …