To supplement Robert's counterexample, let me mention below some interesting facts about the matrix exponential, along with what may be regarded as the "correct" way of obtaining matrix exponential like operator inequalities.

Assume throughout that $A \ge B$ (in Löwner order).

The map $X \mapsto X^r$ for $0 \le r \le 1$ is *operator monotone*, i.e., $A^r \ge B^r$. This result is called *Löwner-Heinz* inequality (it was apparently originally discovered by Löwner). Now using
\begin{equation*}
\lim_{r\to 0} \frac{X^r-I}{r} = \log X,
\end{equation*}
we can conclude monotonicity of $\log X$, so that $\log A \ge \log B$.

A quick experiment reveals that $A^2 \not\ge B^2$ in general (in fact $X^r$ for $r > 1$ is not monotone), which severely diminishes hopes of $e^X$ being monotone.

However, though $e^A \not\ge e^B$, T. Ando (*On some operator inequalities*, Math. Ann., 1987) showed that
\begin{equation*}
e^{-tA} \# e^{tB} \le I,\qquad\forall t \ge 0.
\end{equation*}
Here, $X \# Y := X^{1/2}(X^{-1/2}YX^{-1/2})^{1/2}X^{1/2}$ denotes the *matrix geometric mean* (so basically, operator inequalities go along better with the "right" notion of a geometric mean)

**Additional remarks**

Although $A^2 \ge B^2$ *does not* hold, it turns out that a slightly modified version $(BA^2B)^{1/2} \ge B^2$ does hold, as does $A^2 \ge (AB^2A)^{1/2}$. These inequalities are special cases of a family of such results proved by T. Furuta, and these are called *Furuta inequalities*, for example, we have
\begin{equation*}
(B^rA^sB^r)^{1/q} \ge (B^{s+2r})^{1/q},\quad 0 \le s \le 1, r \ge 0, q \ge 1.
\end{equation*}

notsay "It is well-known..." is the reason for the answer; my reason was the fact that Andreas has provided themagic keyword"operator monotone" googling which the answer becomes apparent (e.g., by reading the Wikipedia entry that shows up on searching "operator monotone matrix exponential"). Since it seems that others care about this question, I'll pen down an answer with some additional useful information that people may appreciate. – Suvrit Jun 30 at 18:35