This question is completely answered, and the result is that the condition involving
the Moebius inversion you mention is both necessary and sufficient! See

* K. H. Kim, N. Ormes, F. Roush. _The spectra of nonnegative integer matrices via formal power series_, J. Amer. Math. Soc. **13** (2000) 773–806. https://doi.org/10.1090/S0894-0347-00-00342-8 

This is really a remarkable and
beautiful theorem.