In this paper of mine (not yet published) : http://www.math.ucla.edu/~i707107/SJK_TmImprovement.pdf

I prove that 

Let $a>1$. Denote by $l_a(p)$   the multiplicative order of $a$ modulo $p$. Then we have
    $$
    \sum_{p<x} \frac{1}{l_a(p)} \leq \left(\frac{2}{\pi}\sqrt{6\log a} +o(1)\right)\frac{\sqrt{x}}{\log x}.$$

using Z. Engberg's upper bound 

$$\sum_{l_a(p)=d} 1 \leq  \frac{ \varphi(d)\log a}{2\log d} + O\left(\frac{d\log\log d}{(\log d)^2}\right).$$

It seems that breaking $\sqrt x$ on the upper bound is very difficult. 

If your assertion is true, then the upper bound would have $N \log\log x$ which is way beyond the upper bound of mine.