A very good reference for these topics is 

Lipman, Joseph: Notes on derived functors and Grothendieck duality. Foundations of Grothendieck duality for diagrams of schemes, 1–259, Lecture Notes in Math., 1960, Springer, Berlin, 2009.

For your first question, the issue is the pseudo functoriality of $(-)^!$ together with the characterization of this functor in terms of its value in the structure sheaf. 

In more detail, with $f : X \to Y$, $g : Y \to Z$, $h : X \to Z$ and $h = g \circ f$. Assume that all maps are *finite type separated map of noetherian schemes*. In this case, we have that $h^! \cong f^! \circ g^!$ (loc. cit. Th (4.8.1)).

Second If moreover $f$ is perfect, i.e. $\mathcal{O}_X$ is relatively perfect over $Y$ then
$$
f^! \mathcal{F} \cong f^* \mathcal{F} \otimes f^! \mathcal{O}_X
$$
(loc. cit. Th (4.9.4)).
By introducing the notation $\omega_X = f^! \mathcal{O}_X$, you get your desired result under the hypothesis mentioned.

As for the formula $\omega_f \cong \det \mathbb L_{f}$ looks plausible to me under complete intersection hypothesis. I don't know of a published proof. And I don't think it holds under more general hypothesis.

Finally, if $f$ is finite if follows from sheafified duality (loc. cit. Cor. (4.3.6)) that
$$
f^! \mathcal{F} \cong \mathbf{R}\mathcal{H}om(f_*\mathcal{O}_X, \mathcal{F})^{\tilde{}}
$$
If you substitute by Frobenius you get you last formula, if I understand well.