This is rather a (long) comment.

I don't think that something like this exists or is at least useful. The only chance could be if the category has an anti-autoequivalence (e.g. finite abelian groups, $A \mapsto Hom(A,S^1)$). I want to comment on

 > By Yoneda's lemma, these two functors contain the same information as X  itself

This is not true. For a covariant functor $F$, morphisms $Hom(X,-) \to F$ correspond to elements of $F(X)$, and for a contravariant functor $G$, morphisms $Hom(-,X) \to G$ correspond to elements of $G(X)$. But what about morphisms in the other direction? I think that these hom-functors, regarded as objects in the functor category, contain much more information than $X$, and they are not related at all. Of course, you could restrict yourself to the category of representable functors, but then somehow it is artificial to talk about these functors, right?

I think it would be the best if you give us at least one example?