I know a couple of answers to the above question:
They can be used for point counting over finite fields/estimating the distribution of primes in characteristic 0.
There are various conjectures/results relating the special values of L-functions with other stuff in the vein of the class number formula/Birch Swinnerton-Dyer conjecture, Iwasawa theory on the other hand.
What else can we do (conjecturally or otherwise) with zeta functions? I am interested in connections of the zeta functions to objects that have nothing to do with zeta functions as such (but are still of interest to arithmetic geometers and other mathematicians). My interests and background are definitely very algebraic so I have almost no idea about what results on the analytic side imply.