The question has some minor misstatements. You didn't really want to mention $R$ anywhere: the base ring is $A$, over which $A'$ should be assumed to be "finite locally free" (equivalently, "finite flat and finitely presented"); this is the same as finite flat if $A$ is noetherian. Likewise for $A' \rightarrow A''$ (which was accidentally not stated in the question). And in the displayed identity replace $R$ with $A$ everywhere.
Also, one should assume that $A''$ has constant rank as a locally free $A'$-module (automatic when $A'$ is a domain).
[It is puzzling why the non-triviality of this discriminant identity does not appear to be as widely recognized as it should be. One cannot reduce such a ring-theoretic identity to any kind of "universal case", in contrast with module-theoretic identities like Cayley-Hamilton; I suspect that most people who cite this identity without reference fall into the trap of incorrectly thinking it is somehow easily reduced to a version with fields. It is also tempting to think that certain similar-looking module-theoetic results for vector bundles relative to ring extensions, such as the canonical module isomorphism in Ch. II 4.2 in Oesterle's amazing 1984 Inventiones paper on Tamagawa numbers and its counterpart in Bourbaki, recover this discriminant identity as a special case. But I don't see how to make a clean direct link via trace pairings.]
Since a vector bundle of constant rank over a semi-local ring is free, if we localize throughout at a prime of $A$ then not only does $A'$ become free over $A$ but also $A''$ becomes free over $A'$. Hence, Zariski-locally on $A$ we can make $A'$ become $A$-free and $A''$ become $A'$-free. In particular, $\delta_{A''/A'}$ admits its usual generator (not necessarily a nonzero-divisor!) well-defined up to unit square even Zariski-locally on $A$. Hence, that norm is defined Zariski-locally on the $A$ by applying ${\rm{N}}_{A'/A}$ to the usual generating element of the ideal in $A'$ Zariski-locally on $A$ (not just locally on $A'$!) and gluing to get a global norm ideal. (This is not a special case of norm of line bundles as in EGA II, 6.5.5 since these discriminant ideals need not have local generator that is not a zero-divisor.)
That being said, the handout you link to does give a completely general proof (with arbitrary $A$) upon noting that each ring extension becomes module-free upon localizing at a prime of $A$ and then inputting the transitivity of trace for such ring extensions (which is handled in that handout by appeal to the more well-known case of fields). Indeed, the core of that handout is specifically written in the generality of arbitrary $A$ (no Dedekind condition) and module-free ring extensions if one looks at what is written in the proof itself. If you did read the entire proof given there then look at it again.
So it seems the only issue is to prove ${\rm{Tr}}_{A''/A} = {\rm{Tr}}_{A'/A} \circ {\rm{Tr}}_{A''/A'}$ as $A$-linear maps $A'' \rightarrow A$. We may assume for the proof of such an identity that $A$ is local, so $A'$ is semi-local and $A$-free with some rank $r$. But then $A''$ is $A'$-free by the constant-rank hypothesis, say with rank $n$. Now we argue exactly as in the case of field extensions (where one uses bases too). Namely, upon choosing an $A'$-basis of $A''$ and forming the matrix over $A'$ for a multipler on $A''$ this becomes the assertion that if $M'$ is an $n \times n$ matrix over $A'$ and we let $M$ be the corresponding $nr \times nr$ matrix
over $A$ by multiplying elements of an $A$-basis of $A'$ against the chosen $A'$-basis of $A''$ (to get an $A$-basis of $A''$) then
$${\rm{Tr}}_{A'/A}({\rm{Tr}}(M')) = {\rm{Tr}}(M).$$ But this
is an elementary exercise with staring at the "diagonal" in block matrices.