I would argue against the OP's opinion. The definition $\Gamma(z)$ becomes very natural if you write it as $\Gamma(z) = \int_0^\infty t^{z} e^{-t} d^\times t$, where $d^\times t=dt/t$ is the Haar measure on the multiplicative group of positive numbers. Moreover, $t^z$ is a character of this group, hence the definition is an instance of the Fourier transform on locally compact abelian groups, in this case called the Mellin transform. In fact, this is why this version works well for the Riemann zeta function and in fact for any automorphic $L$-function: $\pi^{-\frac{s}{2}}\Gamma(\frac{s}{2})\zeta(s)$ is invariant under $s\to 1-s$. Of course, one might say that $\zeta(s)$ is not normalized in the right way, but in terms of the Dirichlet coefficients of $\zeta(s)$, or more generally in terms of the Langlands parameters of an automorphic $L$-function, the current normalization is the right one (cf. Ramanujan conjecture)!
EDIT: I just realized this is an elaboration of a comment Emerton made earlier.