Second Edit: I have added a bounty to the question. I would be very happy to give it for an answer to just the first greyed question. I am not comfortable with explicit formulas, but I wonder if the conjecture stated here might be translated into conjecture about the distribution of $0$'s on the critical line (assuming GRH) for Dirichlet L-function, uniformly in the character. This in turn should be known (conjecturally I mean) to specialists...

Edit: as GH noticed, the way I tried to state Montgomery's conjecture is absurd. I still think that deep down, my question makes sense, but my formulation of it was nonsensical. Apologies to everyone. I'll try in the next few days to reformulate it a correct way (if at all possible).

Edit: as GH noticed, the way I tried to state Montgomery's conjecture is wrong. There were some mistakes in the references I used, which compounded with some mistakes of mine, gave a very poor post. Let me try again, hoping that the questions make more sense now.

The Montgomery's conjecture I am talking about is the following: Let $q \geq 1$ an integer, $a \geq 1$ an integer such that $(a,q)=1$, and let $$\psi(q,a,x) = \sum_{p^\alpha \leq x, p^\alpha \equiv a[q]} \log p.$$ Then for all $\epsilon>0$, one has $$(1) \ \ \ \ \ \ \ \ \ \ \ \ \psi(q,a,x)=\frac{x}{\phi(q)} + O(x^{1/2+\epsilon}q^{-1/2})$$ uniformly for $x>q$ (that is, the implied constant depends only of $\epsilon$).

Montgomery himself states a slightly stronger conjecture in [M], with the error term in $O((x/q)^{1/2+\epsilon})$ and forgets to assume $x>q$. Without the latter condition, the conjecture is trivially false, as noted by GH (see the comments below). Even with the restriction $x>q$, Montgomery's initial conjecture is false, as proven in [FG], who proposes the slightly weaker statement above as a replacement: see [FG, conjecture 1(b)]. The conjecture is also given in {IK,17.5] but without the restriction $x>q$.

Since Dirichlet, results on primes in arithmetic progressions are proved using Dirichlet characters. Hence let $\chi$ be a non-principal Dirichlet character of conductor $q$. Define (as usual) $$\psi(\chi,x) = \sum_{p^\alpha < x} \chi(p^\alpha) \log p.$$ One has $\psi (\chi,x)=\sum_{a \pmod{q}} \chi(a) \psi(q,a,x)$ and $\psi(q,a,x) = \frac{1}{\phi(q)} \sum_\chi \chi(a)^{-1} \psi(\chi,x)$.

Is it reasonable to conjecture the following estimate, uniformly in the non-principal character $\chi$ and the integer $q$, for $x>q$, $$(2)\ \ \ \ \ \ \ \ \ \ \ \ \ \psi(\chi,x) = O(x^{1/2+\epsilon} q^{-1/2})$$? Has this been conjectured somewhere?

Note that the Montgomery's estimate (1) for $\psi(q,a,x)$ follows immediately from (2) by summing over $a$ invertible mod $q$ and dividing by $\phi(q)$. However, it is not clear that (1) implies (2) as the error term seems to get multiplied by $\phi(q)$. Yet, I have made some numerical computation, and the estimate for Dirichlet character seems to hold.

Now assuming that the first question has a positive answer, I want to generalize it boldly to Artin's representations. Let $\rho : Gal(K/\mathbb Q) \rightarrow Gl_d(\mathbb C)$ be an irreducible, non-trivial representation of dimension $d$, Artin's conductor $q$ and let $\chi$ be its character. In this context, I define (also as usual): $$\psi(\chi,x)=\sum_{p^\alpha < x} \chi(Frob_p^\alpha) \log p.$$

What would be a reasonable conjectural estimate for $\psi(\chi,x)$, in terms of $q$, $d$, and $x$, giving back the above conjecture when $d=1$? Has anyone formulated such a conjecture?

Perhaps a starting point is the estimate one gets by usual methods assuming both the GRH for the Artin $L$-function $L(s,\rho)=L(s,\chi)$ and the holomorphy of its $L$-function everywhere, that is, Artin's conjecture for $\rho$ (cf. [IK, Theorem (5.15)]): $$\psi(\chi,x)=O(x^{1/2} \log(x) (d \log x + \log q)).$$ Can we replace the estimate by $O(x^{1/2+\epsilon} d / q^{1/2})$ for example ? I am at a loss even to guess a reasonable formula.

My motivation is trying to understand the best error terms in effective Chebotarev theorem. Any conjecture as asked will lead to a new estimate for Chebotarev, and I have a nice collection of examples to test those versions of Chebotarev against.

References:

[FG] Friedlander, Granville, Limitations of the equi-distribution of primes I, Annals of Maths vol. 129, no2, 1989

[IK] Iwaniec, Kowalski, Analytic Number Theory

[M] Montgomery, Problems about Prime Numbers, in PSPM XXVIII, AMS.

The title would say it all if I precise which Montgomery's conjecture I am talking about, which I do in the next paragraph.

Let $\chi$ be a non-principal Dirichlet character of conductor $q$. Define (as usual) $$\psi(\chi,x) = \sum_{p^\alpha < x} \chi(p^\alpha) \log p.$$ Montgomery has conjectured the following: $$\psi(\chi,x) = O(x^{1/2+\epsilon} q^{-1/2}).$$ Here $\epsilon>0$ is arbitrarily small, and the implicit constant depends only on $\epsilon$. (cf. [IK,17.5], which gives the same estimate for the error term for $\psi(q,a,x)$ defined loc. cit. But $\psi(\chi,x)=\frac{1}{\phi(q)} \sum_{a} \psi(q,a,x)$ where the sum is over $a$ invertible mod $q$, and the formulation given above follows.)

Needless to say that this is an extremely strong conjecture. If we forget about the dependence on $q$, that is if we fix $q$, this is equivalent to the Riemann hypothesis for $L(s,\chi)$. Yet the conjecture seems to be even stronger that GRH for $L(s,\chi)$ regarding the dependence on $q$, as what we get with standard method assuming GRH for $L(s,\chi)$ is (cf [IK,Theorem 5.15]) $$\psi(\chi,x) = O(x^{1/2} \log(x) (\log(x)+\log(q))$$ and with GRH for all Dirichlet characters of conductor $q$ (cf. [IK,(15.3)]) $$\psi(\chi,x) = O(x^{1/2} \log(x)^2)$$ (with absolute implied constant in both cases). There are results saying that Montgomery's conjectural estimate is essentially the best possible.

Now I want to generalize this to Artin's representations. Let $\rho : Gal(K/\mathbb Q) \rightarrow Gl_d(\mathbb C)$ be an irreducible, non-trivial representation of dimension $d$, Artin's conductor $q$ and let $\chi$ be its character. In this context, I define (also as usual): $$\psi(\chi,x)=\sum_{p^\alpha < x} \chi(Frob_p^\alpha) \log p.$$

What would be a reasonable conjectural estimate for $\psi(\chi,x)$, in terms of $q$, $d$, and $x$, giving back the Montgomery conjecture when $d=1$? Has anyone formulated such a conjecture?

Perhaps a starting point is the estimate one gets by usual methods assuming both the GRH for the Artin $L$-function $L(s,\rho)=L(s,\chi)$ and the holomorphy of its $L$-function everywhere ,that is, Artin's conjecture for $\rho$ (cf. [IK, Theorem (5.15)]): $$\psi(\chi,x)=O(x^{1/2} \log(x) (d \log x + \log q)).$$ I would not be surprised if we can get rid easily of the $\log q$ term as in the case of Dirichlet character, but can we replace the estimate by $O(x^{1/2_\epsilon} \log(x) d / q^{1/2})$ for example ? I am at a loss even to guess a reasonable formula.

My motivation is trying to understand the best error terms in effective Chebotarev theorem. Any conjecture as asked will lead to a new estimate for Chebotarev, and I have a nice collection of examples to test those versions of Chebotarev against.

Reference: [IK] Iwaniec, Kowalski, Analytic Number Theory

Edit: as GH noticed, the way I tried to state Montgomery's conjecture is absurd. I still think that deep down, my question makes sense, but my formulation of it was nonsensical. Apologies to everyone. I'll try in the next few days to reformulate it a correct way (if at all possible).