Yes. This is classic, see for instance Theorem 2.1 in Cone-valued Lyapunov functions by Lakshmikantham and Leela from 1977. (I apologize for the reference because there must be something more to the point of your question. To interpret the result in the paper just note that the cone in question is the cone of vectors with nonnegative entries). The conditions on the diagonal and the off-diagonal elements of your matrix mean that you are looking at what is called a Metzler matrix. The dominance condition is not needed. It is not even needed that the diagonal entries are negative.
The point is that Metzler matrices, i.e., matrices with nonnegative off-diagonal entries are precisely the matrices for which $e^{At}$ has nonnegative entries for all $t\geq 0$.
The argument for the case that $y$ is continuously differentiable goes something like this. (A bit more care also yields the result with less regularity, see the reference). Assume that $y$ is about to overtake $x$ in some entry at time $t$ and $y(s) \leq x(s)$ for all $s \in [0,t]$. Thus, by continuity, $y_i(t) = x_i(t)$ and $y(t) \leq x(t)$ componentwise. Then $$ \dot y_i(t) \leq (A y(t))_i = a_{ii} y_i(t) + \sum_{i\neq j} a_{ij} y_j(t) \leq a_{ii} x_i(t) + \sum_{i\neq j} a_{ij} x_j(t) = (A x(t))_i = \dot x_i(t)$$$$ \dot y_i(t) \leq (A y(t))_i = a_{ii} y_i(t) + \sum_{j\neq i} a_{ij} y_j(t) \leq a_{ii} x_i(t) + \sum_{j\neq i} a_{ij} x_j(t) = (A x(t))_i = \dot x_i(t)$$ so that $y_i(t+s) \not > x_i(t+s)$ for small $s>0$. Thus $y$ can never overtake $x$ in any component.
Note that for the inequality we use that $x_i(t) = y_i(t)$, as there is no information about the sign of $a_{ii}$ and the condition that the off-diagonal entries are nonnegative.