No. Minor note: I assume that your definition of irreducible includes not being divisible by an integer $>1$, so that $5x-5$ annihilating the all ones sequence is not an example.
I'm going to show that the constant term of $f$ is $\pm 1$, since that lets me use power series in positive powers of $t$. Obviously, this is equivalent to looking at the leading coefficient by the symmetry $t \mapsto t^{-1}$.
Suppose to the contrary that $f(t)$ annihilates the sequence $(a_i)$ and $f(t)$ is irreducible. Let $p$ be a prime which divides the constant term of $f$ but does not divide every term of $f$. Let $\mathbb{C}_p$ be the completion of the algebraic closure of the $p$-adics. The $p$-adic Newton polygon of $f$ has a line segment with positive slope, so $f$ has a root $\rho$ in $\mathbb{C}_p$ with $|\rho|_p<1$.
Let $a(t) = \sum_{i \geq 0} a_i t^i$. Since the $a_i$ are integers, they have $|a_i|_p \leq 1$, and thus the sums converges for any $t \in \mathbb{C}_p$ with $|t|_p<1$. The condition that $f$ annihilates $(a_i)$ means that $f(t) a(t) = g(t)$ for $g$ a polynomial in $\mathbb{Z}[t]$ of degree less than that of $f$. In particular, $g(\rho)=0$.
But $f$ is irreducible over $\mathbb{Q}$, so it must be the minimal polynomial of $\rho$, and $\deg g < \deg f$. This is a contradiction.