$p$adic zeta function is a $p$adic interpolation of the Riemann $\zeta$function for the values $\zeta(1−k)$, $k\ge 1$ (see $p$adic Numbers, $p$adic Analysis, and ZetaFunctions by Neal Koblitz) or, more precisely, it's corrected values $$\zeta_p(1−k):=(1p^{k1})\frac{B_k}{k}.$$ What is known about the values $\zeta_p(k)$ for $k\ge 1$?
If you use this definition, then $\zeta_p(k)$ is zero at negative even integers $k$, so by a $p$adic continuity argument, it must also be zero at positive even integers.
What about the odd integers? At $k = 1$ there is a pole, unsurprisingly. At odd $k \ge 3$ the value is extremely mysterious, just as the complex zeta values $\zeta(k)$ are. There is an interpretation of the odd $p$adic zeta values in terms of a $p$adic regulator map in $K$theory (see this question), but this is tough to get explicit information out of.
As an example of how little we understand these numbers, I believe it's an open problem whether the values $\zeta_p(k)$ for odd $k \ge 3$ are always nonzero, although this is certainly expected.

$\begingroup$ Thanks! Do I understand correctly that "residue" $\lim_{n\to\infty}B_{2ap^n}$ at the pole $1$ can not be defined correctly? $\endgroup$ – Alexey Ustinov Oct 22 '15 at 3:10

$\begingroup$ If $p \neq 2$, then the negative even integers are dense in $\mathbb{Z}_p$; continuity would imply $\zeta_p$ is identically zero.if it were zero at the even integers. $\endgroup$ – user13113 Oct 22 '15 at 5:33

2$\begingroup$ @Hurkyl, The zeta function is not a continuous function on $\mathbf{Z}_p$; in Koblitz' construction, for p > 2 it is a collection of (p1) continuous functions (one for each residue class modulo p1). Since p1 is even your argument does not work. (A more highpowered approach is to view $\zeta_p$ as a function on the space $Hom(\mathbf{Z}_p^\times, \mathbf{C}_p^\times)$, which is a disjoint union of p1 copies of the open unit disc). $\endgroup$ – David Loeffler Oct 22 '15 at 6:34

2$\begingroup$ @AlexeyUstinov The residue at the pole is very easy to define if you set things up correctly; it is 1  1/p. This is explained very nicely in one of the earlier chapters of Washington's book on cyclotomic fields. $\endgroup$ – David Loeffler Oct 22 '15 at 6:36

$\begingroup$ Yes, I have found the residue in H. Cohen's book (Proposition 11.3.9). $\endgroup$ – Alexey Ustinov Oct 22 '15 at 6:41
I agree nothing much is known, but there are a number of formulas linked to the values at positive integers of padic Lfunctions: see Section 11.3 of my GTM 240 book (sorry for the selfadvertisement).

1

$\begingroup$ The relation between $\zeta_{p}(k)$ and $\zeta(k)$ is also described in section 4.3.3 of Fonctions $L$ $p$adiques des représentations $p$adiques (Astérisque 229) $\endgroup$ – Olivier Oct 22 '15 at 11:15
Let me mention some irrationality results about padic zeta values:
In 2005, Frank Calegari arxiv link proved that for $p=2,3$, the padic zeta values $\zeta_p(3)$ is irrational (hence nonzero). It can be viewed as the padic analogue of Apéry's theorem. Later, Frits Beukers arxiv link gave an alternating proof of Calegari's results (and the irrationality for some other padic Lvalues).
In 2010, Pierre Bel arxiv link obtained some partial results on the irrationality of padic Hurwitz zeta values. See also his 2018 paper on $\zeta_p(4,x)$.
Very recently (in 2020), Johannes Sprang arxiv link showed that $$ \dim_{K}(K+\zeta_p(2)K+\zeta_p(3)K+\cdots+\zeta_p(s)K) \geqslant \frac{(1o(1))\log s}{2[K:\mathbb{Q}](1+\log 2)}, $$ which is the analogue of BallRivoal theorem for padic zeta values.