The following result follows from Tate-Honda theory
Let $A$ be an abelian variety over a finite field $k$, and let $f_A$ be the characteristic polynomial of $A$. Then $A$ is isogenous to a power of a simple abelian variety if and only if $f_A$ is a power of an irreducible polynomial.
I can't find a set of online notes which contains this statement. Kirsten Eisenträger's notes are generally very good, but they get this result wrong on the first page -- Theorem 1.1 claims that, if $f_A$ is a power of an irreducible polynomial, then $A$ is simple, ignoring the possibility that $A$ is a power of a simple variety.
Let $A$ be isogenous to $\bigoplus A_i^{n_i}$, where the $A_i$ are simple and mutually non-isogenous. Every abelian variety has such a decomposition. Then $f_A = \prod f_{A_i}^{n_i}$.
Suppose that $f_A$ is a power of an irreducible polynomial. Then all of the $f_{A_i}$ must also be powers of that polynomial. In particular, for any $i$ and $j$, either $f_{A_i}$ divides $f_{A_j}$ or vice versa; without loss of generality, suppose $f_{A_i} | f_{A_j}$. By a result of Tate, this means that $A_i$ is isogenous to a subvariety of $A_j$. Since $A_i$ and $A_j$ are simple, this means that $A_i$ and $A_j$ are isogenous. Since we assumed that the $A_i$ were mutually nonisogenous, there must in fact be only one summand in our decomposition of $A$, and $A$ is isogenous to $A_1^{n_1}$ for some simple $A_1$ and some $n_1$.
Suppose now that $A$ is isogenous to $B^{n}$ for $B$ simple. Then $f_A = f_B^n$. So our goal is to show that $f_B$ is a power of an irreducible polynomial. If not, write $f_B = gh$ where $g$ and $h$ are relatively prime of positive degree. By a result of Honda, there exist abelian varieties $C$ and $D$ with characteristic polynomials $g$ and $h$. By the result of Tate cited above, $C$ and $D$ are isogenous to subvarieties of $B$, contradicting that $B$ is simple. $\square$
The answer to the question in your title is "yes", we can decide whether $A$ is irreducible by knowing its $\zeta$ function. Fix a prime power $q$. Let $k$ be the field with $q$ elements. Let $W(q)$ be the set of irreducible monic polynomials over $\mathbb{Q}$ all of whose roots have norm $q^{1/2}$. The main result of Honda-Tate theory (Theorem 4.1 in Kirsten's notes) is that there is a bijection between isogeny classes of $k$-simple abelian varieties over $k$ and $W(q)$. For each polynomial $g$ in $W(q)$, there is some positive integer $n(g,q)$ such that the characteristic polynomial of the corresponding simple abelian variety is $f^{n(g,q)}$. The tricky point is that $n(g,q)$ is not always $1$. For example, in Denis's answer, what is going on is that $n(x-p, p^2)=2$. So it is true that $A$ is $k$-simple if and only if $f_A$ is of the form $g^{n(g,q)}$; you just need to know how to compute that $n$ function. I think you should be able to extract this from sections 4 and 5 of Kirsten's notes, but I don't know the details.