This answer is a translation of the much more general [Stacks, Tag 0B68] to this setup:

Write $\mathcal O_X = \mathbf F_X$ and $\mathcal O_Y = \mathbf F_Y$. Then $f \colon (X,\mathcal O_X) \to (Y,\mathcal O_Y)$ is a morphism of ringed spaces, with $f^{-1}\mathcal O_Y = \mathcal O_X$. In particular, $f^* = f^{-1}$ is exact, so $Lf^* = f^*$. Moreover, since $\mathbf F$ is a field, we also have $(-) \otimes_{\mathcal O_X}^{\mathbf L} (-) = (-) \otimes_{\mathcal O_X} (-)$ and similarly on $Y$.

Then the counit $\varepsilon \colon f^*Rf_* \to \operatorname{id}$ of the adjunction $f^* \dashv Rf_*$ gives a canonical map
$$f^*Rf_* \mathbf F_X \underset{\mathbf F_X}\otimes f^*Rf_* \mathbf F_X \stackrel{\varepsilon \otimes \varepsilon}\longrightarrow \mathbf F_X \underset{\mathbf F_X}\otimes \mathbf F_X.$$
By [Stacks, Tag 079U], the source is canonically isomorphic to $f^*(Rf_* \mathbf F_X \otimes_{\mathbf F_Y} Rf_* \mathbf F_X)$. The target simplifies to $\mathbf F_X$. Passing through the adjunction then produces the desired map
$$c \colon Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X \to Rf_* \mathbf F_X.$$
To check associativity [Stacks, Tag 0FP4], we need to show that the diagram
$$\begin{array}{ccc}Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X & \overset{1 \otimes c}\to & Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X \\ \!\!\!\!\!\!\!\!\!\!\!{\scriptsize c \otimes 1}\downarrow & & \downarrow{\scriptsize c}\!\!\!\! \\ Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X & \underset{c}\to & Rf_* \mathbf F_X\end{array}$$
commutes. Both compositions are adjoint to the map
$$f^*\left(Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X \underset{\mathbf F_Y}\otimes Rf_* \mathbf F_X\right) \to \mathbf F_X$$
given by identifying the left hand side with $f^*Rf_*\mathbf F_X \otimes_{\mathbf F_X} f^*Rf_* \mathbf F_X \otimes_{\mathbf F_X} f^*Rf_* \mathbf F_X$, and using associativity $(\varepsilon \otimes \varepsilon) \otimes \varepsilon = \varepsilon \otimes (\varepsilon \otimes \varepsilon)$ in $D(X,\mathbf F_X)$. $\square$