Jakob and Peter have mentioned the category of $G$-sets for a group $G$. This category remains cartesian closed if $G$ is replaced by a monoid $M$, or for that matter by a category (so we are looking at a presheaf category, which is not only cartesian closed but a topos), but IMO it's tricky to guess what exponential objects look like even in the monoid case, since the usual way the answer for $G$ a group is presented involves inverses and it's not at all obvious how to rewrite the answer without inverses so it generalizes. I really do recommend trying to guess the answer; I'm about to spoil it!
For me it's easier to go all the way up to the generality of presheaf categories even just to understand the case of monoids, so let's consider a general presheaf category $[C^{op}, \text{Set}]$, two presheaves $F, G$, and their exponential $[F, G]$. We can probe the exponential by mapping representables into it: if $c \in C$ is an object and we denote by $c$ the corresponding representable presheaf, then we have
$$\text{Hom}(c, [F, G]) \cong \text{Hom}(c \times F, G)$$
and moreover this identification is functorial in $c$ so this is actually a complete description of $[F, G]$ as a presheaf (by the Yoneda lemma; making this remark explicit for searchability). Note that the monoidal unit here is the terminal presheaf with constant value the terminal object $1$ in $\text{Set}$, which is representable iff $C$ itself has a terminal object. Maps out of the unit can be thought of as "global points" or "global sections" of presheaves.
Let's specialize this to monoids (so $C$ has a single object $c$ with $\text{End}(c) \cong M$ for $M$ a monoid). Here $[C^{op}, \text{Set}]$ becomes the category of right $M$-sets and the unique representable presheaf is $M$ regarded as a right $M$-set via right multiplication (Cayley's theorem!), so the internal hom between two right $M$-sets $X$ and $Y$ is
$$\text{Hom}_M(M \times X, Y)$$
where $M \times X$ has the diagonal action, and one can check that the right $M$-action on this homset comes from left multiplication on $M$. Did you guess this? I didn't! To be honest I really don't have a good sense of how to think about this construction, although note that taking fixed points of the right $M$-action does produce $\text{Hom}_M(X, Y)$ as expected, since it corresponds to quotienting $M \times X$ by the left $M$-action, which produces $X$.
The further simplification that occurs when $M$ is a group $G$ comes from the fact that $G \times X$ with the diagonal action is canonically isomorphic to $G \times X$ with the action only on $G$ (equivalently, with the action on $X$ trivialized); the isomorphism is given by
$$G \times X \ni (g, x) \mapsto (g, xg^{-1}) \in G \times X_d$$
where I write $X_d$ for $X$ with the trivial $G$-action. Hence
$$\text{Hom}_G(G \times X, Y) \cong \text{Hom}_G(G \times X_d, Y) \cong \text{Hom}_{\text{Set}}(X, Y)$$
as usual, and then when we compute the $G$-action coming from the left action on the copy of $G$ that has now disappeared we get the usual conjugation formula. There's probably more to say about this isomorphism, having to do with torsors and so forth.