I need to find an example of a cartesian closed category whose functor category (from some diagram $J$) is not cartesian closed. I thought of one possible solution: let $A$ be a commutative square (which is a poset, hence ccc) and let $J$ be the category $\bullet \to \bullet$. Then $A^J$ is the arrow category, and I claim it's impossible to form the product of the parallel arrows in $A$'s square. This seems like a poor example though, any other suggestions?

I think it will not be easy to find an example with $J=2$. Since a cartesian closed category has finite products, the result cited in David White's answer shows the category $A$ for such an example must not have equalizers. So it cannot be a preorder, and so (having binary products) it cannot be finite. It is not hard to find infinite cartesian closed categories without equalizers  but I have not found one with a specific pair of functors that I can show have no exponential. The best example that occurs to me has $J$ the poset with bottom element $0$ and a countable infinity of objects right above $0$, with no arrows to each other. That is the partial order on the natural numbers with $x\leq y$ if and only if $x=0$. And for $A$ the category of finite sets (hereditarily finite, if you like ZF foundations and want a small category). The result follows since the functor assigning the empty set to $0$ and the two element set to every other object of $J$ has infinitely many natural transformations to itself (uncountably many). 


Your example works, but only if $A$ is badly behaved. (EDIT: As Colin points out in his comment, the OP's idea was to take $A$ as a cartesian closed poset, and that won't work. However, keeping $J$ as $\bullet \to \bullet$ will yield an example where $A$ is cartesian closed but $A^J$ is not). In particular, this link gives the following (paraphrased) equivalence:
Here $[a,d] = d^a = Hom(a,d)$ exists because $A$ is cartesian closed. That link also says this result is a special case of the considerations in the following paper, since the category $A^J$ is obtained from $A$ by Artin gluing.
If you need a more explicit example that $A^J$ can fail to be closed, it might be found there. Note that in all applications I've ever used, this extra hypothesis about pullbacks is true and so $A^J$ has two closed cartesian structures. In the first, the product is $f\otimes g: a \otimes c \rightarrow b\otimes d$ and the closed structure is given by the projection $[a,c] \times_{[a,d]} [b,d] \rightarrow [b,d]$ In the second, the product is $f\Box g: a \otimes d \coprod_{a\otimes c} b\otimes c \rightarrow b\otimes d$ and the closed structure is given by the map induced by the pullback $[b,c] \rightarrow [a,c]\times_{[a,d]} [b,d]$ You see where the pullbacks are necessary. Of course, the terminal object is simply $1_\ast$ where $\ast$ is the terminal object in $A$. 

