Ok so I think I have worked out which integers are critical. Let $F$ be a finite Galois extension of $\mathbf{Q}$ and $\rho:\operatorname{Gal}(F/\mathbf{Q}) \to GL(V)$ be an irreducible Artin representation, with $\rho \neq 1$.
First I should mention that there is a functional equation relating $L(\rho,s)$ and $L(\overline{\rho},1-s)$ but it involves some power of $\pi$, so one should be careful when formulating an algebraicity conjecture. So let me restrict to the case $s=1-m$ where $m$ is an integer $\geq 1$.
As proved in Deligne's article Valeurs de fonctions L et périodes d'intégrales, if $1-m$ is critical then the associated period is trivial and Deligne's conjecture is true, so that $L(\rho,1-m)$ is a nonzero algebraic number. One can even prove, using a theorem of Siegel, that $L(\rho,1-m)$ belongs to the number field generated by the values of the character of $\rho$, and that $L(\rho,1-m)^{\sigma}=L(\rho^\sigma,1-m)$ for any $\sigma \in \operatorname{Aut}(\mathbf{C})$, for a detailed proof see Thm 1.2 in Coates-Lichtenbaum, On $\ell$-adic zeta functions, Annals of Math. 98 n°3 (1973) (I'm grateful to Junkie for pointing out to me this reference). If $1-m$ is not critical then $L(\rho,1-m)=0$ but algebraicity of the leading term is not expected (regulators are expected to be transcendental, but even their irrationality is very difficult to prove).
Thus it remains to find the critical integers. By definition $1-m$ is critical iff the gamma factor $L_\infty(\rho,s)$ has no pole at $s=1-m$. By definition $L_\infty(\rho,s)=\Gamma_{\mathbf{R}}(s)^{\dim V^+} \Gamma_{\mathbf{R}}(s+1)^{\dim V^-}$ where $\Gamma_{\mathbf{R}}(s)=\pi^{-s} \Gamma(s/2)$ and $V^{\pm}$ is the $\pm$-eigenspace for the action of $\rho(c)$, where $c \in \operatorname{Gal}(F/\mathbf{Q})$ is a choice of complex conjugation. A small computation then gives :
$$s=1-m \textrm{ is critical if and only if } V=V^{(-1)^m}.$$
Examples and remarks.
- If $\rho$ is $1$-dimensional, this is consistent with the situation for Dirichlet characters.
- If $F$ is totally real then $V=V^+$ so the result is consistent with the Klingen-Siegel theorem.
- If $\rho$ is an odd irreducible $2$-dimensional Artin representation (correponding to a weight 1 newform thanks to the proof of Serre's conjectures) then $\dim V^+ = \dim V^{-}=1$ so there is no critical integer for $L(\rho,s)$.
- On the other hand if $\rho$ is $2$-dimensional and even (corresponding conjecturally to a non-holomorphic Maass cusp form), then half of the integers are critical for $L(\rho,s)$.
- It is possible to extend the above analysis to Artin representations associated to arbitrary Galois extensions of number fields.