Physicists often use functional integrals and I'm trying to make sense of it in more precise terms. As you can see here, the functional derivative in Physics is defined in terms of Taylor expansions. Let me elaborate.

**[The Physicist point of view]** Let $f$ be a functional defined on some convenient function space and let $\varphi, \eta$ be fixed functions on this space. We expand $f$ in Taylor:
\begin{eqnarray}
f(\varphi + t\eta) = f(\varphi) + \frac{df}{dt}(\varphi + t\eta)\bigg{|}_{t=0}t + \frac{1}{2}\frac{d^{2}f}{dt^{2}}(\varphi+t\eta)\bigg{|}_{t=0}t^{2} +\cdots + \frac{1}{n!}\frac{d^{n}f}{dt^{n}}(\varphi+t\eta)\bigg{|}_{t=0}t^{n} +\mathcal{o}(t^{n+1}) \tag{1}\label{1}
\end{eqnarray}
where $\frac{d^{k}f}{dt^{k}}(\varphi+t\eta)\bigg{|}_{t=0}$ denotes the $k$-th Gâteaux derivative of $f$ at $\varphi$ evaluated at $\eta$. Thus, the $k$-th functional derivative of $f$ at $\eta$ is the function $\frac{\delta^{k} f}{\delta \varphi(x_{1})\cdots\delta\varphi(x_{k})}$ satisfying the equality:
\begin{eqnarray}
\frac{d^{k}f}{dt^{k}}(\varphi+t\eta)\bigg{|}_{t=0} =\int dx_{1}\cdots dx_{k} \frac{\delta^{k}f}{\delta \varphi(x_{1})\cdots\delta\varphi(x_{k})}\eta(x_{1})\cdots \eta(x_{k}) \tag{2}\label{2}
\end{eqnarray}

**[The Mathematician point of view]** Let $E, F$ be Banach spaces. A continuous bilinear functional ${\langle \cdot\,, \cdot \rangle }: E \times F \to \mathbb{R}$ is called $E$-non-degenerate if $\langle x,y\rangle = 0$ for all $y \in F$ implies $x=0$ (Similarly for $F$-non-degenerate). Equivalently, the two maps of $E$ to $F^{*}$ and $F$ to $E^{*}$ defined by $x \mapsto \langle x, \cdot \rangle$ and $y \mapsto \langle \cdot\,, y\rangle$, respectivelly, are one-to-one. If they are isomorphisms, $\langle \cdot\,, \cdot \rangle$ is called $E$ or $F$-strongly non-degenerate. We say that $E$ and $F$ are in duality if there is a non-degenerate bilinear functional $\langle \cdot\,, \cdot \rangle: E\times F \to \mathbb{R}$, also called a *pairing* of $E$ with $F$. If the functional is strongly non-degenerate, we say the duality is strong.

Consider the following definition (from this book).

**Definition:** Let $E$ and $F$ be normed spaces and $\langle \cdot, \cdot \rangle$ a $E$-non-degenerate pairing. Let $f: F \to \mathbb{R}$ be Fréchet differentiable at the point $\varphi \in F$ (denote this derivative as $Df(\varphi)$). The functional derivative $\delta f/\delta \varphi$ of $f$ with respect to $\varphi$ is the unique element in $E$, if it exists, such that:
\begin{eqnarray}
Df(\varphi)(\eta) = \left\langle \frac{\delta f}{\delta \varphi}, \eta\right\rangle\quad\forall\gamma \in F. \tag{3}\label{3}
\end{eqnarray}

Now, take $E=F=C(\Omega)$ to be a space of functions defined on a region $\Omega \subset \mathbb{R}^{n}$, which is Banach, and take the pairing $\langle \cdot, \cdot \rangle : C(\Omega)\times C(\Omega) \to \mathbb{R}$ given by:
\begin{eqnarray}
\langle f,g\rangle := \int_{\Omega}f(x)g(x)dx \tag{4}\label{4}
\end{eqnarray}
If $f$ is Fréchet differentiable at $\varphi$, then it is also Gâteaux differentiable at $\varphi$ and the following identity holds:
\begin{eqnarray}
Df(\varphi)(\eta) = \frac{df}{dt}(\varphi+t\eta)\bigg{|}_{t=0} \tag{5}\label{5}
\end{eqnarray}
Thus, the above definition together with the pairing \eqref{4} and \eqref{5} implies that the functional derivative of $f$ at $\varphi$ is the element $\delta f/\delta\varphi$ satisfying:
\begin{eqnarray}
\frac{df}{dt}(\varphi+t\eta)\bigg{|}_{t=0} = \int \frac{\delta f}{\delta \varphi}\eta(x)dx \tag{6}\label{6}
\end{eqnarray}
Note that \eqref{6} is *exactly* the physicist definition \eqref{2} for $k=1$. Now, my question is how to extend the mathematician's definition to consider higher order derivatives. If $f$ has, say, $k$ Fréchet derivatives at $\varphi$, then it has $k$ Gâteaux derivatives at this point. But now, the $k$-th Fréchet derivatives is a $k$-linear map, so I wonder if I should extend the definition by considering not pairings but $k$ linear maps instead, and then demand that these $k$-linear maps satisfy something like:
\begin{eqnarray}
D(\varphi_{1},\ldots,\varphi_{k})(\eta) = \left\langle \frac{\delta^{k}f}{\delta \varphi^{k}},\eta,\ldots,\eta\right\rangle \nonumber
\end{eqnarray}
where, now, $\langle \cdot, \cdots, \cdot \rangle$ is a $k$-linear non-degenerate map. Another possible approach is to use the same pairings and define high order derivatives as successive applications of the first derivative (I don't know how to do it though) and then prove a representation theorem when $E=F=C(\Omega)$, i.e. prove that if we take $E=F=C(\Omega)$ and use the pairing \eqref{4} then this $k$-th functional derivative becomes \eqref{2}. I'm really lost at this point, and I'd appreciate any help or tips on how to proceed.

**EDIT:** A nice discussion in my previous question led me to some clarifications and possible directions. First, suppose that $f$ is twice Fréchet differentiable at $\varphi \in E$. Then, there exists a bounded bilinear functional $D^{2}f[\varphi]$ satisfying
\begin{eqnarray}
\lim_{\eta \to 0}\frac{Df[\varphi+\eta](\gamma)-Df[\varphi)](\gamma)-D^{2}f[\varphi](\eta,\gamma)}{\Vert\eta\Vert} = 0. \tag{7}\label{7}
\end{eqnarray}
But, using \eqref{3}, we also have
$$
\begin{split}
Df[\varphi+\eta](\gamma)-Df[\varphi](\gamma) &= \left\langle \frac{\delta f}{\delta(\varphi+\eta)},\gamma\right\rangle - \left\langle\frac{\delta f}{\delta \varphi},\gamma\right\rangle \\
&=\left\langle\frac{\delta f}{\delta(\varphi+\eta)}-\frac{\delta f}{\delta \varphi},\gamma\right\rangle = \langle \mathcal{L}[\varphi](\eta),\gamma\rangle
\end{split}
$$ for some linear operator $\mathcal{L}[\varphi]:E\mapsto E$. If we take $E=F=C(\Omega)$ as I mentioned before, it seems that the physicist's result is obtained by taking
$$
\begin{eqnarray}
\mathcal{L}[\varphi](\eta) := \int \frac{\delta^{2}f}{\delta \varphi^{2}} (x,y)\beta(x) dx \tag{8}\label{8}
\end{eqnarray}
$$
where, now, $\delta^{2}f/\delta\varphi^{2} = \delta^{2}f/\delta\varphi^{2}(x,y)$ is a function on $C(\Omega\times\Omega)$ and this would be our second order functional derivative of $f$. But I still have doubts about that. Why taking \eqref{8} as my linear map? It seems very arbitrary.