To supplement Robert's counterexample, let me mention below some interesting facts about the matrix exponential, along with what may be regarded as the "correct" way of obtaining matrix exponential like operator inequalities.

Assume throughout that $A \ge B$ (in Löwner order).

The map $X \mapsto X^r$ for $0 \le r \le 1$ is operator monotone, i.e., $A^r \ge B^r$. This result is called Löwner-Heinz inequality (it was apparently originally discovered by Löwner). Now using \begin{equation*} \lim_{r\to 0} \frac{X^r-I}{r} = \log X, \end{equation*} we can conclude monotonicity of $\log X$, so that $\log A \ge \log B$.

A quick experiment reveals that $A^2 \not\ge B^2$ in general (in fact $X^r$ for $r > 1$ is not monotone), which severely diminishes hopes of $e^X$ being monotone.

However, though $e^A \not\ge e^B$, T. Ando (On some operator inequalities, Math. Ann., 1987) showed that \begin{equation*} e^{-tA} \# e^{tB} \le I,\qquad\forall t \ge 0. \end{equation*} Here, $X \# Y := X^{1/2}(X^{-1/2}YX^{-1/2})^{1/2}X^{1/2}$ denotes the matrix geometric mean (so basically, operator inequalities go along better with the "right" notion of a geometric mean)

Additional remarks

But it turns out that a slightly modified version $(BA^2B)^{1/2} \ge B^2$ does hold, as does $A^2 \ge (AB^2A)^{1/2}$. These inequalities are special cases of a family of such results proved by T. Furuta, and these are called Furuta inequalities, for example, we have \begin{equation*} (B^rA^sB^r)^{1/q} \ge (B^{s+2r})^{1/q},\quad 0 \le s \le 1, r \ge 0, q \ge 1. \end{equation*}

To supplement Robert's counterexample, let me mention below some interesting facts about the matrix exponential, along with what may be regarded as the "correct" way of obtaining matrix exponential like operator inequalities.

Assume throughout that $A \ge B$ (in Löwner order).

The map $X \mapsto X^r$ for $0 \le r \le 1$ is operator monotone, i.e., $A^r \ge B^r$. This result is called Löwner-Heinz inequality (it was apparently originally discovered by Löwner). Now using \begin{equation*} \lim_{r\to 0} \frac{X^r-I}{r} = \log X, \end{equation*} we can conclude monotonicity of $\log X$, so that $\log A \ge \log B$.

A quick experiment reveals that $A^2 \not\ge B^2$ in general (in fact $X^r$ for $r > 1$ is not monotone), which severely diminishes hopes of $e^X$ being monotone.

However, though $e^A \not\ge e^B$, T. Ando (On some operator inequalities, Math. Ann., 1987) showed that \begin{equation*} e^{-tA} \# e^{tB} \le I,\qquad\forall t \ge 0. \end{equation*} Here, $X \# Y := X^{1/2}(X^{-1/2}YX^{-1/2})^{1/2}X^{1/2}$ denotes the matrix geometric mean (so basically, operator inequalities go along better with the "right" notion of a geometric mean)

Additional remarks

Although $A^2 \ge B^2$ does not hold, it turns out that a slightly modified version $(BA^2B)^{1/2} \ge B^2$ does hold, as does $A^2 \ge (AB^2A)^{1/2}$. These inequalities are special cases of a family of such results proved by T. Furuta, and these are called Furuta inequalities, for example, we have \begin{equation*} (B^rA^sB^r)^{1/q} \ge (B^{s+2r})^{1/q},\quad 0 \le s \le 1, r \ge 0, q \ge 1. \end{equation*}