By using a spectral sequence for the composite functor $T:\Gamma_S (f_*)$ where $f:X \to S$ is the structure morphism and $\Gamma_S$ is the global sections functor on $S$. Duality of the fibers and the base give rise to duality of the terms of the spectral sequence. The main non-formal part is in making the duality map $P: R^iT (-) \times R^jT(-)$ which involves, in particular, knowledge of the dualising object. Once you have $P$, showing that the pairing is non-degenerate etc comes from duality on the terms of the spectral sequence and the five-lemma.

The simplest instance of the non-formal part referred to above is as follows:
If $$0 \to A \to B \to C\to 0$$ and $$0 \to E \to F \to G \to 0$$ are exact sequences and one has a duality pairing between $A$ and $E$ as well as between $C$ and $G$, one needs to construct a compatible map for $B$ and $F$; that the map induces a duality will follow formally in most situations (finite groups etc).

Milne's Arithmetic Duality Theorems should have this.

Actually, duality theorems are better stated in the appropriate derived category and so if one is used to derived categories, then Kashiwara-Schapira (Sheaves on manifolds) should be enough to formulate the appropriate theorem.

Hope this helps.