If you use this definition, then $\zeta_p(k)$ is zero at negative even integers $k$, so by a $p$-adic continuity argument, it must also be zero at positive even integers.

What about the odd integers? At $k = 1$ there is a pole, unsurprisingly. At odd $k \ge 3$ the value is extremely mysterious, just as the complex zeta values $\zeta(k)$ are. There is an interpretation of the odd $p$-adic zeta values in terms of a $p$-adic regulator map in $K$-theory, but this is tough to get explicit information out of. 

As an example of how little we understand these numbers, I believe it's an open problem whether the values $\zeta_p(k)$ for odd $k \ge 3$ are always non-zero, although this is certainly expected.