As perhaps you surmise, the relation between "free" and "cofree" is not one of a formal duality. Ordinarily, a module is said to be "free" if it occurs in the essential image of the left adjoint to the forgetful functor $\text{Mod}_R \to \text{Set}$. There is no formal dual of this notion because this forgetful functor does not possess a right adjoint (if it did, it would have to preserve for example coproducts, which it clearly does not).
So the formal notion of "cofree module" has to be taken with a slight grain of salt; one has to argue largely by analogy, and some of the analogies might appear ad hoc. In the category of abelian groups, one might reasonably consider $\mathbb{Q}/\mathbb{Z}$ to play a dual role to $\mathbb{Z}$. For example, $\mathbb{Z}$ is a projective generator, where "generator" means that every abelian group is a quotient of a direct sum of copies of $\mathbb{Z}$. Likewise, $\mathbb{Q}/\mathbb{Z}$ is an injective cogenerator, where "cogenerator" means that every abelian group can be exhibited as a subobject of a product of copies of $\mathbb{Q}/\mathbb{Z}$. Now there are plenty of projective generators and plenty of injective cogenerators, but $\mathbb{Z}$ could be distinguished by the fact that it is "minimal": it is (up to isomorphism) the only one that appears as a direct summand of any other projective generator. Likewise, $\mathbb{Q}/\mathbb{Z}$ is the only injective cogenerator that appears as a direct summand of any other. Extending this analogy, if a free module is a direct sum of copies of $\mathbb{Z}$, then a "cofree module" is a direct product of copies of $\mathbb{Q}/\mathbb{Z}$.
Another way to think of this is that $A^\vee = \hom(A, \mathbb{Q}/\mathbb{Z})$ is sometimes called the algebraic character group of an abelian group $A$ (analogous to the topological character group $A' = \hom(A, S^1)$, aka the Pontryagin dual of $A$). It is sometimes called an algebraic dual of $A$. Then the algebraic dual of a free abelian group is a cofree abelian group.
Crossing over to module land, the forgetful functor $\text{Mod}_R \to \text{Ab}$ has both a left adjoint $R \otimes -$ and a right adjoint $\hom(R, -)$. The left adjoint applied to a free abelian group $\bigoplus_I \mathbb{Z}$ gives a free $R$-module $\bigoplus_I R$. Aanalogously, the right adjoint applied to a cofree abelian group $\prod_I \mathbb{Q}/\mathbb{Z}$ gives a cofree $R$-module $\prod_I R^\vee = \prod_I \hom(R, \mathbb{Q}/\mathbb{Z})$. Perhaps this helps explains why $\hom(R, \mathbb{Q}/\mathbb{Z})$ appears prominently in the literature on cofree modules.
Again, the analogy is not based on a perfect formal duality. In my comment I mentioned that while projective modules over a PID are free, it is not the case that injective modules over a PID are cofree. One might also note while modules always have injective hulls, the dual statement (that every module possesses a projective cover) does not hold in general, except over special classes of rings (perfect rings).
Edit: see also this discussion.