There is a precise, almost literal, sense in which $f^* : \textbf{Sh} (Y) \to \textbf{Sh} (X)$ generalises the inverse image as defined in elementary set theory.
Observe that open subspaces $V \subseteq Y$ correspond to subterminal objects in $\textbf{Sh} (Y)$: the sheaf of sections of the inclusion $V \hookrightarrow Y$ is a subterminal object, and every subterminal object is isomorphic to one of this form.
Since $f^*$ preserves finite limits, it preserves subterminal objects, and in fact it sends the sheaf corresponding to $V$ to the sheaf corresponding to $f^{-1} V$.
So we may think of $f^* : \textbf{Sh} (Y) \to \textbf{Sh} (X)$ as being an extension of the set-theoretic inverse image operation, and this justifies the name "inverse image functor".

Once you have an "inverse image functor", there is a powerful temptation to call its partner the "direct image functor".
My personal opinion is that the name "direct image functor" is unsuitable for the *right* adjoint of the inverse image functor.
For one thing, $f_* : \textbf{Sh} (X) \to \textbf{Sh} (Y)$ is not a generalisation of the set-theoretic direct image operation, as I will now explain.

Let $U$ be an open subspace of $X$.
What do you suppose $f_*$ applied to the subterminal object corresponding to $U$ yields?
It is a subterminal object, of course, but it is not (the subterminal object corresponding to) the direct image
$$\exists_f U = \{ y \in Y : \exists x \in U . y = f (x) \} = \{ y \in Y : \exists x \in X . y = f (x) \land x \in U \}$$
which is not even guaranteed to be open if we do not assume $f : X \to Y$ is an open map.
Instead, it corresponds to
$$\forall_f U = \{ y \in Y : f^{-1} \{ y \} \subseteq U \} = \{ y \in Y : \forall x \in X . y = f (x) \Rightarrow x \in U \}$$
which is always open: indeed, $\forall_f U$ is the union of all open $V \subseteq Y$ such that $f^{-1} V \subseteq U$.

Unfortunately, the reality is that $f^* : \textbf{Sh} (Y) \to \textbf{Sh} (X)$ does not always have a left adjoint that generalises $\exists_f$, so while it is $\exists_f$ that bears the name "direct image" in elementary set theory, for want of a better name it is $f_*$ that gets the name in topos theory.