I'm not sure what the purpose of the assumption on the ranks of$E$ and $G$ is (by the way, how are you defining rank? Are you making some assumptions about the sheaves, and is $S$ smooth?). I guess it's so that you can make your assumption about the double dual of $H$.

As YBL mentioned, there is always an isomorphism $det(G) \simeq det(E) \otimes det(H)$ whenever you have a short exact sequence. I'm assuming that your map $0 \to det(E) \to det(G)$ is, under this isomorphism, equivalent to picking out a section of $det(H).$ It is then clear that the cokernel of your map will end up being the same as the cokernel of the map $0 \to \mathcal{O}_S \to det(H).$ This will be true in general, without any of the extra assumptions you've made.

In your particular case, if $det(H) \simeq \mathcal{O}_S(D),$ then this cokernel will be the skyscraper sheaf supported at $D$. Maybe I'm being stupid, but does the assumption on the double dual of $H$ imply this about the determinant?