Let the singular values of $A$ be denoted $\lambda_1,\ldots, \lambda_n$, so that $det(xI-A)=\prod_{i=1}^n (x-\lambda_i)$. Then $tr(A^k)=\sum_{i=1}^n \lambda_i^k =p_k(\lambda_1,\ldots,\lambda_n)$$tr(A^k)=\sum_{i=1}^n \lambda_i^k =p_k(\lambda_1,\ldots,\lambda_n)=p_k$, where $p_k$ is the power symmetric polynomial of degree $k$ in $n$ variables. Now, we observe that $p_k$ is a polynomial in $p_1,\ldots, p_n$ for $k>n$. I found another mathoverflow question which computed the first few of these.
Consider $n=2$ (note, $SL(2)=Sp(2)$), and suppose $p_i(\lambda_1,\lambda_2) \leq 2, 1\leq i\leq 2$$|p_1(\lambda_1,\lambda_2)|=|\lambda_1+\lambda_2|=|\lambda_1+\lambda_1^{-1}| \leq 2$. It turns out that if $-2\leq p_1 \leq 2$, then that $|tr(A^k)|\leq 2$ for all $k$, and $A$ is either parabolic or elliptic (and similarly if $|p_2|\leq 2$). So assume $p_1,p_2 \leq -3$. Then $p_3=-p_1^3 +\frac32 p_1p_2 \geq \frac{81}{2}$. Thus, $N(2)$ exists.
Addendum: I now have a proof for general $n$.
There's a similar pattern to the general formula for $p_{n+1}$ in terms of $p_1,\ldots,p_n$, which is given in a comment by Darij Grinberg to this question. He gives the formula:
$$ \sum_{i=1}^{n+1} \frac{(-1)^i}{i!} \sum_{j_1,\ldots,j_i\geq 1; j_1+\cdots+j_i=n+1} \frac{1}{j_1 \cdots j_i} p_{j_1}\cdots p_{j_i}= 0.$$$$ p_{n+1}= (n+1)\sum_{i=2}^{n+1} \frac{(-1)^i}{i!} \sum_{j_1,\ldots,j_i\geq 1; j_1+\cdots+j_i=n+1} \frac{1}{j_1 \cdots j_i} p_{j_1}\cdots p_{j_i}= 0.$$
This gives the same leading order asymptotics $p_{n+1}=(-1)^np_1^{n+1} + O(p_1^n)$. A similar analysis might yield thereforeThis sum has the property that if $p_1,\ldots,p_n <0$,
then all the terms are positive, since each term is a boundproduct of the
form $(-1)^i p_{j_1}\cdots p_{j_i}$.
Theorem
Consider monic degree $n$ polynomials $q(x)\in \mathbb{R}[x]$, $q(x)=\prod_{i=1}^n (x-\lambda_i)$. There is a compact subset $C_\delta$ of the space of monic degree $n$ polynomials so that if $q(x)\notin C_\delta$, then $p_k(\lambda_1,\ldots,\lambda_n) > \delta$ for some $k \leq n(n+1)$.
Proof Assume $p_1,\ldots,p_n \leq \delta$. We'll assume $p_1 \leq -R$,
for some $R>>0$, to be determined. If $p_2,\ldots,p_n \leq 0$, then
$p_{n+1} \geq \frac{1}{n!} R^n > \delta$ for $R>>0$. So assume some
subset $0< p_{l_1},\ldots ,p_{l_m} \leq \delta$. Consider all the terms
of the expression of $p_{n+1}$ in which an odd number of $p_{l_1},\ldots, p_{l_m}$ appear (with multiplicity), then these terms will be $\leq 0$, and
the rest of the terms will be positive. However, these terms will be dominated
by a term replacing each $p_{l_i}$ with $p_1^{l_i}$. This correspondence
is not 1-1, and the coefficients may differ, but by making $R>>0$ large
enough, we may guarantee that all the negative terms will be bounded
above by corresponding large positive terms. Thus, in this case we may
show that $p_{n+1}>\delta$ for $R>>0$.
Now, suppose $p_k \leq -R$ for some $1\leq k\leq n$. Then by the same
reasoning, $p_{k(n+1)} >\delta$, essentially replacing $A$ with $A^k$.
So our compact set $C_\delta$ consists of $-R\leq p_i \leq \delta, 1\leq i\leq n$.
Now, consider monic degree $n$ polynomials with integer coefficients.
There are only finitely many in $C_2$, so for all these, there will be
but it might takesome uniform power $N(n)$ so that $p_k >2$ for some care to work it out$k\leq N(n)$.
In fact, we see that for all but finitely many polynomials, $k\leq n(n+1)$.