If you have an abelian category $A$ with enough injective, for any functor defined on bounded below cochain complexes $\Phi\colon C^+(A)\to D$ sending cohain homotopy equivalences between degree-wise injective cochain complexes to isomorphism, there is a total right derived functor $R\Phi\colon D^+(A)\to D$. This means that there is a natural transformation $\eta_K\colon\Phi(K)\to R\Phi(K)$ for any bounded below cochain complex $K$ and that the pair $(R\Phi,\eta)$ is universal (initial) among all possible choices (i.e. it is the left Kan extension of the functor $\Phi$ along the localization functor from cohain complexes to the derived category). Furthermore, $R\Phi$ is not only a derived functor, but a *universal* one: this means that, for any functor $\Psi:D\to E$, the pair $(\Psi\circ R\Phi,\Psi(\eta))$ is the right derived functor of the composition $\Psi\circ\Phi$. All this works the same for unbounded complexes is $A$ is an abelian category and if we replace degree-wise injective cochain complexes by a suitable notion (e.g. fibrant of objects in a suitable Quillen model structure).

In particular, in practice, an identification of the form $R(g_*\circ f_*)\cong Rg_*\circ Rf_*$ never comes out of the blue, but rather always as derived functors: the isomorphism above is the unique map which is compatible with the canonical map $(g_*\circ f_*)(K)\to R(g_*\circ f_*)(K)$ and the image by $Rg_*$ of the canonical map $f_*(K)\to Rf_*(K)$ (functorially in $K$). Furthermore, even better, for any functor $\Psi$ whose domain is the codomain of $Rg_*$, the isomorphism
$$\Psi(R(g_*\circ f_*)(K))\cong \Psi(Rg_*\circ Rf_*(K))$$
is the unique map which is compatible with the image by $\Psi$ of the canonical map $(g_*\circ f_*)(K)\to R(g_*\circ f_*)(K)$ and of the image by $Rg_*$ of the canonical map $f_*(K)\to Rf_*(K)$ (functorially in $K$).

For the compatibility problem you mention, we have to compare two maps from $R(h_* g_* f_*)$ to $Rh_* Rg_* Rf_*$. Using the universal property of $R(h_* g_* f_*)$, we just have to check that, for any cohain complex $K$, the two induced maps
$$(h_* g_* f_*)(K)\to (Rh_* Rg_* Rf_*)(K)$$
agree, which comes right away from the compatibilities mentioned before.

As final remark: all this reasoning can be made in much more general context (e.g. with derived functors defined on homotopy categories of model categories). This extra level of generality means that all the reasoning takes place in a context where there are no possiblities to make sign mistakes because signs do not exist. All the ingredients of the proof are available since the late 1960's.