A matrix $A\in\textbf{Mat}_n(\mathbb{R})$ is called asymptotically nilpotent if for each vector $v$, ${\lim}_{k\to\infty}A^k(v) = 0$.
Question 1. Assume that $\mathcal{A}$ is the subset of $\textbf{Mat}_n(\mathbb{R})$ with only asymptotically nilpotent matrices and which is closed under the Lie bracket operation (i.e. $[X, Y] = XY - YX\in \mathcal{A},\forall X, Y\in\mathcal{A}$). Is it true that for some $P\in\text{GL}_n(\mathbb{R})$, $P\mathcal{A}P^{-1}$ is a subset of all matrices with coefficients of absolute value $<1$ on and below the diagonal?
A set $S\subseteq \textbf{Mat}_n(\mathbb{R})$ is called asymptotically nilpotent if for each sequence $A_1, A_2, A_3,\ldots\in S$ and each vector $v$, $\lim_{k\to\infty}A_kA_{k - 1}\ldots A_2A_1(v) = 0$.
Question 2. Assume that $S$ is an asymptotically nilpotent set of matrices. Is it true that for some $P\in\text{GL}_n(\mathbb{R})$, $PSP^{-1}$ is a subset of all matrices with coefficients of absolute value $<1$ on and below the diagonal?
Question 3. Let $\mathbf{Q}_p$ be the field of $p$-adic numbers. Assume that $S\subset \text{Mat}_n(\mathbf{Q}_p)$ is an asymptotically nilpotent finite set of matrices. Is it true that for some $P\in\text{GL}_n(\mathbf{Q}_p)$, $PSP^{-1}$ is a subset of all matrices with coefficients of absolute value $<1$ on and below the diagonal?
I will state some personal ideas.
In the conditions of the Question 2 it is possible to prove the next simpler theorem
Theorem 1. Assume that $S$ is a finite asymptotically nilpotent set of matrices. Then there is an operator $P\in\text{GL}_n(\mathbb{R})$ such that $PSP^{-1}$ is a subset of matrices $(a_{ij})_{i, j\leq n}$ where $a_{i1}<1$ for $i\leq n$.
Proof. Consider the open set of vectors $U = \{(a_1, a_2,\ldots, a_n)\in\mathbb{R}^n\:|\: a_1 < 1\}$. Since $S$ is asymptotically nilpotent there is a vector $e_1\in\mathbb{R}^n\setminus U$ such that $A(e_1)\in U$ for each $A\in \langle S\rangle$ ($\langle S\rangle$ is the semigroup generated with $S$). Now for a natural $N$ let $e_1^N = e_1\in\mathbb{R}^n$ and $e_i^N = (\underbrace{0,\ldots, 0}_{i -1 }, N,0,\ldots, 0)\in\mathbb{R}^n$ for $2\leq i\leq n$. Let $P_N\in \text{GL}_n(\mathbb{R})$ denote the operator with $P_N(e_i^N) = (\underbrace{0,\ldots, 0}_{i -1 }, 1,0,\ldots, 0)$ for $1\leq i\leq N$. Then for $N$ large enough $P_NSP_N^{-1} =(a_{ij})_{i, j\leq n}$ where $a_{i1}<1$ for $i\leq n$.
Question 3 in the case $n=2$ follows from the next Theorem 2.
Theorem 2. Let $\mathbf{Q}_p$ be the field of $p$-adic numbers. Assume that $S\subset \text{Mat}_2(\mathbf{Q}_p)$ is an asymptotically nilpotent finite set of matrices. Then for some $P\in\text{GL}_2(\mathbf{Q}_p)$, $PSP^{-1}$ is a subset of all matrices with coefficients of absolute value $<1$ on and below the diagonal.
Proof. Consider $P\in\text{GL}_2(\mathbf{Q}_p)$ as in the Theorem 1. Thus for each $A\in S$, $PAP^{-1} = (a_{ij})$ with $|a_{11}|_p, |a_{21}|_p<1$. $\text{tr}(A) = \text{tr}(PAP^{-1}) = a_{11} + a_{22} = \lambda_1^A + \lambda_2^A$ where $\lambda_i^A$ are eigenvalues of $A$. So $|a_{11} + a_{22}|_p = |\lambda_1^A + \lambda_2^A|_p < 1$ and since $|a_{11}|_p<1$, then in the case $|a_{22}|_p\not= |a_{11}|_p$, $\max(|a_{11}|_p, |a_{22}|_p) = |a_{11} + a_{22}|_p < 1$ and $|a_{22}|_p < 1$ as well.
