This is a repost of a question on Math stackexchange. No one is biting at it there, so I guess it is harder than I thought.

Assume $X$ and $Y$ are topological spaces, $f : X \to Y$ is a continuous map. Let ${\bf Sh}(X)$, ${\bf Sh}(Y)$ be the category of sheaves on $X$ and $Y$ respectively. Modulo existence issues we can define the inverse image functor $f^{-1} : {\bf Sh}(Y) \to {\bf Sh}(X)$ to be the left adjoint to the push forward functor $f_{*} : {\bf Sh}(X) \to {\bf Sh}(Y)$ which is easily described.

My question is this: Using this definition of the inverse image functor, how can I show (without explicitly constructing the functor) that it respects stalks? i.e is there a completely categorical reason why the left adjoint to the push forward functor respects stalks?