Dear Bryden,

Hopefully I have things straight, and there is a general formula $i^!\omega\_X = \omega\_{X\_{red}}$. One then has the functorial isomorphism (of sheaves on $X$)
$RHom_{\mathcal O_{X\_{red}}}({\mathcal F},\omega_{X\_{red}}) = Rhom_{\mathcal O_X}(i_\*{\mathcal F}, \omega\_X),$
for a coherent sheaf $\mathcal F$ on $X_{red}$. (Normally we would have to apply
an $Ri_*$ to the source of this isomorphism, to put the RHom sheaves on the
same space, and would have to have an $Ri_*$ in the formula on the RHS. But
$i_*$ is exact, and in fact just identifies sheaves on $X_{red}$ with sheaves
on $X$ via the identification of their underlying topological spaces.

Now $RHom_{\mathcal O_X}(i_*{\mathcal F},\omega_X) = Hom_{\mathcal O_X}(i_*{\mathcal F}, {\mathcal J}^{\bullet})$,
where ${\mathcal J}^{\bullet}$ is an injective resolution of $\omega_X$, which
in turn equals $Hom_{\mathcal O_{X_{red}}}(\mathcal F,{\mathcal J}^{\bullet}[\mathcal I]),$
where $\mathcal I$ is the ideal sheaf of $X_{red}$ in $X$.

Finally, this last complex can be identified with
$RHom_{\mathcal O_{X_{red}}}(\mathcal F, RHom_{\mathcal O_X}(\mathcal O_{X_{red}},
\omega_X)).$

So we get the formula
$\omega_{X_{red}} = RHom_{\mathcal O_X}(O_{X_{red}}, \omega_X).$
(And the derivation shows that this should be valid for any closed immersion,
provided one is in a context where the dualizing complex formalism is satisfied,
except that probably there should be some shifts in dimension in general,
because the dualizing complex probably coincides with the dualizing sheaf place
not in degree 0, but in degree $-dim X$. However, in our case the dimensions
of $X$ and $X_{red}$ coincide, so this shift can be ignored.)

Note that, as this formula shows, $\omega_{X_{red}}$ could be a complex, not
just a sheaf. This is reasonable, I guess; in general, even if $X$ is CM,
this needn't imply that $X_{red}$ is (I imagine).

If in fact $X_{red}$ is CM, then I guess we find just one non-zero term in the formula
for $\omega_{X_{red}},$ and so have $\omega_{X_{red}} = \omega_X[\mathcal I].$

With a bit of luck, the above is not bogus, and answers your question.