Here are a slightly different flavor of examples. Of course as mentioned by Peter LumsdainLumsdaine, the trick is always the same: looking at categories where $Hom(I,X)$ looks quite different from the "underlying sets of $X$"
Consider the category whose objects are setsets, and whose morphisms are relations, i.e. subsets of $X \times Y$ (with the usual composition of relation).
It has a monoidal structure given by the product of sets. The exponential for this monoidal structure is simply the product, indeed,
A relation from $X$ to $Y \times Z$ is the same as a relation from $X \times Z$ to $Y$.
Other similar examples includesinclude:
The 2-category of sets and spanspans of sets (with the products of sets as monoidal structure).
The category of small categories and profunctorprofunctors, with the products of categories as monoidal structure, here the exponential $[X,Y]$ is given by $Y \times X^{op}$
The category of polynomial functorfunctors (which is actually cartesian closed) described in The cartesian closed bicategory of generalized species and structures by Fiore,Gambino Gambino,Hyland Hyland and Winskel. Where the exponential is a little more complicated, but still has the same flavor.
But one can somehow "explain the trick" behind them: One way to think about these example is that these categories are equivalent to categories where the Hom objects corresponds more naturally to a natural structure oneon the set of morphisms, but the equivalence of categories somehow makes the "underlying set" very different.
For example, the category of setsets and relation can be seen as the category of "Free suplattice"suplattices":
A suplattice is an ordered set with all supremumsupremums, and suplattice morphisms are order preserving map, preserving the supremums. Given two suplattices $X$ and $Y$ the set of morphisms from $X$ and $Y$ is naturally a suplattice for the pointwise ordering (induce by the order of $Y$), and this corresponds to a monoidal symmetric closed structure on the category of suplattice. So this is really a "non-example" of what is asked: the hom objects are really exactly the set of morphisms with a natural structure induced on them.
Now the forgetfullforgetful functor from suplattice to sets has a left adjoint, sending any set to the suplattice $P(X)$ of subsets of $X$, sublattice morphisms from $P(X)$ to $P(Y)$ coincide with relation from $X$ and $Y$, and the monoidal structure described above is induced from the one on suplattices...
So our example become a non-example up to an equivalence of categories. Similar descriptiondescriptions can be obtained for the others exampleother examples I have mentioned.