Is there any "sufficient and necessary" condition for a comlpex polynomial P(z)=a_0+...a_n z^n, in terms of its coefficinets, to have all its roots outside the unit disc(or equivalently inside the unit disc)?
If not, is there any such conditions where the coefficinets are real or rational or integers?
Look up the Schur-Cohn algorithm. It's discussed at length in Volume I of Henrici's "Applied and Computational Complex Analysis, and its exactly what you need.
It runs as follows. If $p$ is a complex polynomial of degree $n$ with leading coefficient $a_0$ and constant term $a_n$, define $T(p)$ to be the polynomial $$ \bar{a}_0 p(z) -a_np^*(z). $$
(Here $p^*(z)$ is the reciprocal polynomial $z^np(1/z)$.) Then all zeros of $p$ lie outside the closed unit disc if and only if the numbers $T^k(p)(0)$ for $k=1,\ldots,n$ are all positive.
There is a well-known Routh-Hurwitz stability condition (condition in terms of coefficients for the roots of a polynomial to lie in the left half-plane). By composing with a fractional linear transformation which maps the left half-plane onto the exterior of the unit circle, you obtain a condition for the roots to be in the exterior. (Of course, composition is a rational function, but just take its numerator). Similarly, you can obtain necessary and sufficient conditions for the roots to be in the unit disc, or on the unit circle, or in any disc or a circle.
The stability condition is written in terms of certain determinants made of coefficients; it can be found in books and papers on polynomials and on control theory.
