Curvature as infinitesimal holonomy Let $P \to M$ be a principal $G$-bundle, assume as much regularity as you want (compact $G$ or compact base manifold, ect). Via parallel transport, a connection $A$ on $P$ gives rise to the holonomy map
$$
Hol_p: L_m M \to G
$$
where $p \in P$ is a chosen reference point projecting on $m \in M$ and $L_mM$ denotes the loop group based at $m$ (i.e piecewise smooth, closed curves starting at $m$).

Question: What is the exact relationship between the curvature $F_A$ of $A$ and the derivative of $Hol_p$ at the constant loop (the infinitesimal holonomy).

In the abelian case, I can answer this question as follows:
Let $\gamma_s$ be a family of loops which represent a tangential vector $X \in T_\gamma L_mM$, that is $\gamma_0 = \gamma$ and $\frac{d}{ds}{\big|_0} \gamma_s = X$. Then the derivative of $Hol_p$ in the direction of $X$ evaluates to
$$
\frac{d}{ds}{\big|_0} Hol_p (\gamma_s) = \frac{d}{ds}{\big|_0} \exp (\int_{\gamma_s} A) = (\exp)'_0 \int_0^1 \frac{d}{ds}{\big|_0} A(\dot \gamma_s(t)) dt = \int_0^1 dA(\frac{d}{ds}{\big|_0} \gamma_s(t), \dot \gamma_0(t)) dt,
$$
That is, in this case we get a close expression for the derivative of $Hol_p$ at an arbitary loop $\gamma$ in terms of the curvature $F_A = dA$. Furthermore, the Ambrose-Singer theorem follows from this expression.
I was hoping to get a similar result also for the non-abelian case.
Sidequestion: conjugacy classes of (topological) homomorphisms $h$ between the loop gorup and $G$ uniquely determine a equivalence class of principal bundles with curvature by a result of Kobayashi (1954). Which additional properties on $h$ have to be impsoed to characterize all (equivalence classes of) connections on a fixed principal bundle $P$.
 A: I have no idea about the sidequestion. For the main question, there is a general answer applicable for any parallel transport, not only for holonomy.
Let $\gamma_{s}$ a family of smooth paths such that $\gamma_{s}(0)=p,\gamma_{s}(1)=q$ for every $s$ and fixed points $p$, $q$, and let $P_{s}(t’,t)$ the parallel transport from $\gamma_{s}(t)$ to $\gamma_{s}(t’)$ along $\gamma_{s}$. Here I try to prove the formula
$$\begin{equation}
\frac{d}{ds}P_s(1,0)= \int_{0}^{1}P_{s}(1,t)F(\partial_{t}\gamma_{s}(t),\partial_{s}\gamma_{s}(t))P_{s}(t,0)dt
\end{equation},
$$
where $F$ is the curvature. (There might appear a negative sign depending on convention. I use a convention where $F=dA+A\wedge A, \nabla=d+A, F(w,v)=F_{\nu\mu}w^{\nu}v^{\mu}$.) Especially if $\gamma_{0}$ is the constant loop, then $\partial_{t}\gamma_{0}(t)$ is always zero, so the derivative map of $Hol_{p}$ at the constant loop is just the zero map.
The proof goes as if we are discussing a connection of vector bundles. Let $H_{t}(s’,s)$ the parallel transport from $\gamma_{s}(t)$ to $\gamma_{s’}(t)$ along $\gamma_{\bullet}(t)$ and let $S(s,s’;t)=P_{s}(1,t)H_{t}(s,s’)P_{s’}(t,0)$. Then,
$$
P_{s’}(1,0)-P_{s}(1,0)=S(s,s’;1)-S(s,s’;0)
=\int_{0}^{1}\partial_{t}S(s,s’;t)dt
=\int_{0}^{1}\lim_{\epsilon\rightarrow 0} P_s(1,t+\epsilon)\frac{H_{t+\epsilon}(s,s’)P_{s’}(t+\epsilon,t)-P_{s}(t+\epsilon,t)H_{t}(s,s’)}{\epsilon}P_{s’}(t,0)dt,
$$
so
$$
\partial_{s}P_{s}(1,0)=\int_{0}^{1}\lim_{(\epsilon,\delta)\rightarrow 0}P_{s}(1,t+\epsilon)\frac{H_{t+\epsilon}(s,s+\delta)P_{s+\delta}(t+\epsilon,t)-P_{s}(t+\epsilon,t)H_{t}(s,s+\delta)}{\epsilon\delta}P_{s+\delta}(t,0)dt.
$$
From the parallelogram discussion follows the identity
$$
\lim_{(\epsilon,\delta)\rightarrow 0} \frac{H_{t+\epsilon}(s,s+\delta)P_{s+\delta}(t+\epsilon,t)-P_{s}(t+\epsilon,t)H_{t}(s,s+\delta)}{\epsilon\delta}=F(\partial_{t}\gamma_{s}(t),\partial_{s}\gamma_{s}(t)),
$$
and thus also does the formula I’m proving.
I’d like to comment that integrating both sides of the formula reads a relationship between $P_{s’}(1,0)-P_{s}(1,0)$ and the curvature, if we focus on connections of vector bundles. (If you want to discuss a $G$-principal bundle $Q$, you should remember that each $P_{s}(1,0)$ can be canonically identified with an element of $(Q_{q}\times Q_{p}\times G)/\sim$, where $(v,u,g)\sim(vy,ux,y^{-1}gx)$. Then the formula I have shown describes the path $P_{\bullet}(1,0)$ in this manifold in terms of the curvature.)
