If you use this definition, then $\zeta_p(k)$ is zero at negative even integers $k$, so by a $p$-adic continuity argument, it must also be zero at positive even integers.
What about the odd integers? At $k = 1$ there is a pole, unsurprisingly. At odd $k \ge 3$ the value is extremely mysterious, just as the complex zeta values $\zeta(k)$ are. There is an interpretation of the value in terms of a $p$-adic regulator map in $K$-theory but this is tough to get explicit information out of.
As an example of how little we understand these numbers, I believe it's an open problem whether the values $\zeta_p(k)$ for odd $k \ge 3$ are always non-zero, although this is certainly expected.