As mentioned in the comments, once you're comfortable with indices, this is just an exercise in calculus. Differentiating both sides of the identity $g^{\alpha\gamma}g_{\gamma\zeta} = \delta^{\alpha}_{\zeta}$ with respect to the parameter $t$, we see that

\begin{align*}
\frac{d}{dt}\left(g^{\alpha\gamma}g_{\gamma\zeta}\right) &= \frac{d}{dt}\delta^{\alpha}_{\zeta}\\
\dot{g}^{\alpha\gamma}g_{\gamma\zeta} + g^{\alpha\gamma}\dot{g}_{\gamma\zeta} &= 0\\
\dot{g}^{\alpha\gamma}g_{\gamma\zeta} &= -g^{\alpha\gamma}\dot{g}_{\gamma\zeta}\\
\dot{g}^{\alpha\gamma}g_{\gamma\zeta}g^{\beta\zeta} &= -g^{\alpha\gamma}\dot{g}_{\gamma\zeta}g^{\beta\zeta}\\
\dot{g}^{\alpha\gamma}\delta_{\gamma}^{\beta} &= -g^{\alpha\gamma}g^{\beta\zeta}\dot{g}_{\gamma\zeta}\\
\dot{g}^{\alpha\beta} &= -g^{\alpha\gamma}g^{\beta\zeta}\dot{g}_{\gamma\zeta}.\\
\end{align*}

Replacing the index $\zeta$ by the index $\delta$, you get precisely the expression you were looking for. I avoided using $\delta$ as an index to avoid confusion with the expressions involving $\delta$ in the first and penultimate lines.