Skip to main content
7 of 8
added 44 characters in body

Modeling decay of a linear system with a mixing term

I'm trying to model convergence of $x\in \mathbb{R}^{+d}$ which follows the following recurrence

$$\mathbf{x}\leftarrow (\mathbf{1}-\mathbf{h})^2 \mathbf{x} + \mathbf{h}\langle \mathbf{x}, \mathbf{h}\rangle$$

Here $\mathbf{1}$ indicates a vector of $1$'s, vector multiplication, addition, squaring are applied pointwise and $\mathbf{h}\propto (1^{-p},2^{-p},\ldots ,d^{-p})$ for some $p\in (1,2)$.

$\mathbf{h}$ is small enough so that continuous approximation $\mathbf{x}_t\approx \exp(At)\mathbf{x}_0$ holds with $\mathbf{x_0}=\mathbf{h}$ and I need to know how trajectory of $\|\mathbf{x}_t\|_1$ depends on $p$ when $d\to\infty$.

Things are easy if we didn't have the the $\mathbf{h}\langle \mathbf{x}, \mathbf{h}\rangle$ term (mixing term). $A$ is diagonal, and approximating $\|\mathbf{x}_t\|_1$ with an integral I get formulas which match observed behavior very well.

enter image description here Notebook

However, keeping the mixing term makes things much harder to handle. System now evolves as $\exp(At)$ where $A$ is a diagonal+rank1 matrix. Integration approach as before gives formula which is cumbersome. There's also numeric approach which works but doesn't give insight on the role of $p$.

Any advice on the approaches to follow to get a nice upper bound on $\|\mathbf{x}_t\|_1$ in terms of $p$?

Motivation: this equation models expected value of iterated Gaussian linear system like this for non-isotropic Gaussian case. Used to model training curve of neural network training.