Interesting question! Here is a proof when $R$ is a Dedekind domain and $M=R$. The general case for $M$ arbitrary may follow but I have not checked carefully.

Let $N^*$ denote $Hom(N,R)$ and $D(N)$ denote $Ext_R^1(N,R)$. It is easy to check that for a torsion $R$-module $D^2(N)\cong N$. In particular $ann(D(N)) = ann(N)$.

Now dualize the exact sequence $0 \to I \to R \to R/I \to 0$ one gets: $$ 0 \to R\to I^* \to D(R/I) \to 0$$

Since $R$ is a Dedekind domain $I^*$ is also an ideal of $R$. It is well-known that ideals in $R$ are strongly 2-generated (they can be generated by at most 2 elements with the first one arbitrary), so $D(R/I)$ is a cyclic $R$-module. But since the annihilator of that module is equal to $I$, it must be $R/I$.

About the general case, there is a natural map:

$$ Ext^n_R(N,R)\otimes M \to Ext^n_R(N,M) $$ By the case $M=R, N=R/I$ and $n=1$ it gives a map $M/IM \to Ext^1_R(R/I,M)$. If one can check that this map localizes naturally, then it reduces to the PID case, but I have not checked. UPDATE: actually it can be shown that if $pd_RN = n$ then the map above is an isomorphism, completing the proof.

Here's why: Let $P_{\bullet}: 0\to P_n \to \cdots \to P_0$ be a projective resolution of $N$. Then one compute the LHS by taking cohomologies at the end of $Hom(P_{\bullet},R)$ and tensor with $M$, and the RHS by taking cohomology at the end of $Hom(P_{\bullet},M)$. But tensor product is right-exact, proving that we get the same answer.

Finally, when $R$ is regular local of dimension $n$ then $Ext^n_R(N,R)\cong N$ if $N$ has finite length (by Grothendieck local duality). So perhaps your question can be viewed as some kind of non-local duality.

Interesting question! Here is a generalization which includes the Dedekind domain case:

Proposition: Let $R$ be a Noetherian regular domain with $n=\dim R$ and $I\subset R$ an ideal such that $R/I$ is artinian and Gorenstein. Then for a finitely generated $R$-module $M$, we have $Ext^n(R/I,M)\cong M/IM$.

(when $R$ is Dedekind $R/I$ is locally a quotient of a DVR, so it is locally a hypersurface, thus Gorenstein).

Proof. It is enough to prove for $M=R$ because of the following

Claim: if $N$ is an artinian $R$-module then the natural map $Ext^n_R(N,R)\otimes M \to Ext^n_R(N,M)$ is an isomorphism.

Here's why: We have $pd_RN = n$. Let $P_{\bullet}: 0\to P_n \to \cdots \to P_0$ be a projective resolution of $N$. Then one compute the LHS by taking the cohomology at the end of $Hom(P_{\bullet},R)$ and tensor with $M$, and the RHS by taking cohomology at the end of $Hom(P_{\bullet},M)$. But tensor product is right-exact, proving that we get the same answer.

Assuming we can prove for $M=R$, then apply the claim with $N=R/I$ completes the proof.

Now we prove it for $M=R$. Since $R/I$ is artinian, we only need to prove it after localizing at each maximal ideal containing $I$. Thus we can assume $(R,m)$ is local. But by Local Duality $Ext^n(R/I,R)$ is isomorphic to the Matlis dual of $H^0_m(R/I) =R/I$. But since $R/I$ is Gorenstein this dual is actually isomorphic to $R/I$. QED

Interesting question! Here is a proof when $R$ is a Dedekind domain and $M=R$. The general case for $M$ arbitrary may follow but I have not checked carefully.

Let $N^*$ denote $Hom(N,R)$ and $D(N)$ denote $Ext_R^1(N,R)$. It is easy to check that for a torsion $R$-module $D^2(N)\cong N$. In particular $ann(D(N)) = ann(N)$.

Now dualize the exact sequence $0 \to I \to R \to R/I \to 0$ one gets: $$ 0 \to R\to I^* \to D(R/I) \to 0$$

Since $R$ is a Dedekind domain $I^*$ is also an ideal of $R$. It is well-known that ideals in $R$ are strongly 2-generated (they can be generated by at most 2 elements with the first one arbitrary), so $D(R/I)$ is a cyclic $R$-module. But since the annihilator of that module is equal to $I$, it must be $R/I$.

About the general case, there is a natural map:

$$ Ext^n_R(N,R)\otimes M \to Ext^n_R(N,M) $$ By the case $M=R$ it gives a map $M/IM \to Ext^1_R(R/I,M)$. If one can check that this map localizes naturally, then it reduces to the PID case, but I have not checked.

Finally, when $R$ is regular local of dimension $n$ then $Ext^n_R(N,R)\cong N$ if $N$ has finite length (by Grothendieck local duality). So perhaps your question can be viewed as some kind of non-local duality.

Interesting question! Here is a proof when $R$ is a Dedekind domain and $M=R$. The general case for $M$ arbitrary may follow but I have not checked carefully.

Let $N^*$ denote $Hom(N,R)$ and $D(N)$ denote $Ext_R^1(N,R)$. It is easy to check that for a torsion $R$-module $D^2(N)\cong N$. In particular $ann(D(N)) = ann(N)$.

Now dualize the exact sequence $0 \to I \to R \to R/I \to 0$ one gets: $$ 0 \to R\to I^* \to D(R/I) \to 0$$

Since $R$ is a Dedekind domain $I^*$ is also an ideal of $R$. It is well-known that ideals in $R$ are strongly 2-generated (they can be generated by at most 2 elements with the first one arbitrary), so $D(R/I)$ is a cyclic $R$-module. But since the annihilator of that module is equal to $I$, it must be $R/I$.

About the general case, there is a natural map:

$$ Ext^n_R(N,R)\otimes M \to Ext^n_R(N,M) $$ By the case $M=R, N=R/I$ and $n=1$ it gives a map $M/IM \to Ext^1_R(R/I,M)$. If one can check that this map localizes naturally, then it reduces to the PID case, but I have not checked. UPDATE: actually it can be shown that if $pd_RN = n$ then the map above is an isomorphism, completing the proof.

Here's why: Let $P_{\bullet}: 0\to P_n \to \cdots \to P_0$ be a projective resolution of $N$. Then one compute the LHS by taking cohomologies at the end of $Hom(P_{\bullet},R)$ and tensor with $M$, and the RHS by taking cohomology at the end of $Hom(P_{\bullet},M)$. But tensor product is right-exact, proving that we get the same answer.

Finally, when $R$ is regular local of dimension $n$ then $Ext^n_R(N,R)\cong N$ if $N$ has finite length (by Grothendieck local duality). So perhaps your question can be viewed as some kind of non-local duality.