This page on Wikipedia defines the so-called functional derivative as follows: "Given a manifold $$M$$ representing (continuous/smooth) functions $$\rho$$ (with certain boundary conditions, etc.) and a functional $$F: M \to \mathbb{C}$$, the functional derivative of $$F[\rho]$$, denoted by $$\delta F/\delta \rho$$ is defined by: $$\int \frac{\delta F}{\delta \rho}(x) \phi(x) dx = \lim_{\epsilon \to 0}\frac{F[\rho+\epsilon \phi]-F[\rho]}{\epsilon} = \bigg{[}\frac{d}{d\epsilon}F[\rho+\epsilon \phi]\bigg{]}_{\epsilon = 0} \label{1}\tag{1}$$ where $$\rho$$ is an arbitrary function. In other words, $$\phi \mapsto \bigg{[}\frac{d}{d\epsilon}F[\rho+\epsilon \phi]\bigg{]}_{\epsilon = 0} \label{2}\tag{2}$$ is a linear functional, so by Riesz-Markov-Kakutani Representation Theorem, this functional is given by integration against some measure. Then $$\delta F/\delta \rho$$ is defined to be the Radon-Nikodym derivative of this measure."

Now, regarding this definition, I have two questions:

1. I understand that the linear functional \eqref{2} is nothing but the Gâteaux derivative of $$F$$ (if it exists). Now, as far as I know, the Riesz-Markov-Kakutani Representation Theorem is related to positive linear functionals, not just arbitrary linear functionals and I see no reason why the Gâteaux derivative \eqref{2} should be (always) positive. Does it mean that the functional derivative of $$F$$ exists if it is Gâteaux differentiable and its Gâteaux derivative is positive? If this is the case, this seems to imply that existence of Gâteaux derivative does not imply existence of functional differentiable but the converse is holds.
2. Is the limit in \eqref{1} uniform, i.e., does it depend on the choice of $$\phi$$? I assume it does not because functional derivatives are usually referred to as Frechet derivatives and the latter is some sort of uniform Gâteaux derivative. Is this correct?

Premise (a long one): before answering your questions, I must say that, if your are searching mathematically rigorous informations, you should not rely on Wikipedia entry "Functional derivative" in its current status, since it is seriously flawed due to an "edit war" between me and another contributor (or perhaps it would be better to say between him and all other contributors, as you can notice having a look at the talk page of the entry). Due to this, the entry is written more from a theoretical physicist's point of view than from a contemporary mathematical perspective, and its content adheres strictly and tacitly to the hypotheses assumed (even implicitly) by Vito Volterra ([6], §II.1.26-II.1.28, pp. 22-24). Namely

1. Volterra implicitly assumes that the functional $$F$$ is of integral type, i.e. similar to the functionals encountered in the classical calculus of variation. However, in general functional analysis, this is not always true: for example, the following functional, defined defined on $${C}^1(\Omega)$$, $$\Omega\subseteq\Bbb R^n$$ $$F[\rho]=\sum_{i=1}^n\frac{\partial\rho}{\partial x_i}(0)=\langle\vec{\mathbf{1}},\nabla \rho(0)\rangle \neq\int\limits_{\Omega}\!\rho(x)\,\mathrm{d}\mu_x,\label{nif}\tag{NIF}$$ cannot be expressed in the form of an integral respect to any given measure, as it is well known from the theory of distributions.
In general, the functional derivative cannot always be represented as the left side term of \eqref{1} since it may not be defined and, even if it happens to be so, it can be different from the central and right side ones (which represent however the true definition of functional derivative), unless it is interpreted as a distribution or as another kind of generalized function by abuse of notation. However, there is a deeper issue, described in the following point.

2. Volterra explicitly assumes that the variation of $$F$$ i.e. the quantity $$\Delta F[\rho]=F[\rho+\delta\rho]-F[\rho]=F[\rho+\varepsilon\phi]-F[\rho]$$ is linear respect to the increment $$\delta\rho=\varepsilon \phi$$ apart from a remainder behaving as $$o(\varepsilon)$$ as $$\varepsilon\to 0$$. Now , while the requirement on asymptotic behavior is basically equivalent to the existence of the limit \eqref{1}, the linearity hypothesis is not always satified ([3], §3.1-3.3, pp. 35-40, and [4] §2.1 p. 15, §3.1-3.3 pp. 30-33). For example, the following functional defined on $${C}^1(\Bbb R^n)$$ by using a function $$\rho_o\in C^1(\Bbb R^n)$$ such that $$\rho_0\not\equiv 0$$, $$F[\rho] = \int\limits_{G} \frac{|\nabla(\rho(x)-\rho_o(x))|^2}{\rho(x)-\rho_o(x)}\exp\left(-\frac{|\nabla(\rho(x)-\rho_o(x))|^4}{|\rho(x)-\rho_o(x)|^2}\right) \mathrm{d}x, \quad G\Subset\Bbb R^n \label{nlf}\tag{NLF}$$ (the specification of the precise form of the integrand on the zero set of $$\rho-\rho_0$$, as well as on intersection between this set and the zero set of its gradient, would require a little more care, but this is only a technical detail and adds nothing to the answer) has a functional derivative which is not linear at the point $$\rho_o$$. Indeed, given $$\phi\in C^1(\Bbb R^n)$$ such that $$\phi\neq 0$$ in $$G$$, $$F[\rho_o+\varepsilon \phi] = \varepsilon\int\limits_{G} \frac{|\nabla \phi(x)|^2}{\phi(x)}\exp\left(-\varepsilon^2\frac{|\nabla\phi(x)|^4}{|\phi(x)|^2}\right) \mathrm{d}x,$$ thus $$\bigg{[}\frac{\mathrm{d}}{\mathrm{d}\varepsilon}F[\rho+\varepsilon \phi]\bigg{]}_{\varepsilon = 0} = \int\limits_{G} \frac{|\nabla \phi(x)|^2}{\phi(x)} \mathrm{d}x$$

Furthermore, while Volterra developed his functional calculus having in mind Banach spaces of continuous functions with respect to the uniform norm (even if the concept of a Banach spaces was not yet defined at the time), theoretical physicists apply it to far more general contexts, in general without any formal justification.

1. I understand that the linear functional \eqref{2} is nothing but the Gâteaux derivative of $$F$$ (if it exists). Now, as far as I know, the Riesz-Markov-Kakutani Representation Theorem is related to positive linear functionals, not just arbitrary linear functionals and I see no reason why the Gâteaux derivative \eqref{2} should be (always) positive. Does it mean that the functional derivative of $$F$$ exists if it is Gâteaux differentiable and its Gâteaux derivative is positive? If this is the case, this seems to imply that existence of Gâteaux derivative does not imply existence of functional differentiable but the converse is holds.

As is, that statement in the entry is not correct without assuming something on where the functional $$F$$ is defined and thus on its structure. You have correctly noticed one of the basic issues: the functional derivative of $$F$$ is assumed to be a Gâteaux derivative, but this does not implies its positivity, and moreover it does not need to be representable as a measure, as example \eqref{nif} above shows. For example it can be thought as a distribution, as shown in this answer. Volterra derives the integral representation for the functional derivative on the left side of \eqref{1} under precise hypothesis ([6] §II.1.27 pp. 23-24 and reference [5] §2, pp. 99-102 cited therein), having in mind applications to the classical calculus of variation: under different hypotheses, this may not be true.

1. Is the limit in \eqref{1} uniform, i.e., does it depend on the choice of $$\phi$$? I assume it does not because functional derivatives are usually referred to as Frechet derivatives and the latter is some sort of uniform Gâteaux derivative. Is this correct?

The limit depends on the structure of $$\phi$$, not only on its "size" (i.e. its norm when $$M$$ is a Banach space): this is probably the core difference between Gâteaux and Fréchet derivatives of functionals, with the former one playing the infinite dimensional analogue of the directional derivative ([1] §1.1 p. 12 and [2] §1.B p. 11). When $$M$$ is Banach, the statement is clear since $$\phi$$ enters the definition, equivalent to \eqref{2}, of Fréchet derivative only with its norm, and this implies that any $$\phi$$ with the same norm does the job: for more general topological vector spaces, things are more complex, but you can have a look at references [4], §3.2-3.2 pp. 30-32 for Gâteaux derivatives and to [2] §1.B p. 11 for Fréchet derivatives (see however [1] remark 1.2 pp. 11-12, on the definition of Fréchet derivatives in locally convex spaces and the issues involved in defining higher order derivatives).

Bibliographical note

Vainberg ([3], [4]) explicitly says that the functional derivative can be a nonlinear functional of the increment: however, he calls it Gâteaux differential, reserving the name "derivative" for the cases where it is a linear functional, and this nomenclature seems to be non standard. All other authors deal extensively only with functionals having linear functional derivatives, sometimes not even mentioning the possibility of the existence of functionals like \eqref{nlf}.

Bibliography

[1] Ambrosetti, Antonio; Prodi, Giovanni, A primer of nonlinear analysis, Cambridge Studies in Advanced Mathematics, 34. Cambridge: Cambridge University Press, pp. viii+171 (1993), ISBN: 0-521-37390-5, MR1225101, ZBL0781.47046.

[2] Schwartz, Jacob T., Nonlinear functional analysis, Notes by H. Fattorini, R. Nirenberg and H. Porta. With an additional chapter by Hermann Karcher. (Notes in Mathematics and its Applications.) New York-London-Paris: Gordon and Breach Science Publishers, pp. VII+236 (1969), MR0433481, ZBL0203.14501.

[3] Vaĭnberg, Mikhail Mordukhovich, Variational methods for the study of nonlinear operators. With a chapter on Newton’s method by L.V. Kantorovich and G.P. Akilov, translated and supplemented by Amiel Feinstein, Holden-Day Series in Mathematical Physics. San Francisco-London- Amsterdam: Holden-Day, Inc. pp. x+323 (1964), MR0176364, ZBL0122.35501.

[4] Vaĭnberg, Mikhail Mordukhovich, Variational method and method of monotone operators in the theory of nonlinear equations. Translated from Russian by A. Libin. Translation edited by D. Louvish, A Halsted Press Book. New York-Toronto: John Wiley & Sons; Jerusalem-London: Israel Program for Scientific Translations, pp. xi+356 (1973), MR0467428, ZBL0279.47022.

[5] Volterra, Vito, "Sulle funzioni che dipendono da altre funzioni [On functions which depend on other functions]" (in Italian), Atti della Reale Accademia dei Lincei, Rendiconti (4) III, No. 2, 97-105, 141-146, 153-158 (1887), JFM19.0408.01.

[6] Volterra, Vito, Theory of functionals and of integral and integro-differential equations. Dover edition with a preface by Griffith C. Evans, a biography of Vito Volterra and a bibliography of his published works by Sir Edmund Whittaker. Unabridged republ. of the first English transl, New York: Dover Publications, Inc. pp. 39+XVI+226 (1959), MR0100765, ZBL0086.10402.

• This is an amazing answer! Thank you so much! – IamWill Jan 2 '20 at 16:01

Let me first attend your qualms about requiring positivity in the use of the representation theorem. In any version which might be relevant in your context, this condition is not required. Having said that, I would have to add that your quotation is a terrifying example of what can go wrong if one turns to a wikipedia article for an introduction to a (mathematical) subject. I can‘t even begin to formulate a critique of this passage and I would strongly advise you to turn to a competent mathematician or a decent text if you require a clean mathematical treatment of differentiation of functionals on function spaces or, more generally, Banach (or locally convex) spaces. Of course, if you merely require a cookbook version such as one often finds in calculus of variations texts, then there is no point in worrying about the niceties of the representation theorem. This is just a personal opinion, of course.

• Sorry, this was intended as a comment, not an answer. Not sure how to recategorise it. – user131781 Dec 25 '19 at 9:37
• Thanks for the answer. The problem is that I can not find any other reference on this subject. Physicists use requirement (1) as a definition of functional derivative and they are not rigorous só that there is no good physics text where this is discussed in detail. On the other hand, I know a bit about Fréchet and Gâteaux derivatives, which are well-defined and etc but I don't know how to connected with the physicists vision of functional derivative. – IamWill Dec 25 '19 at 12:10
• Do you know any good reference on this subject? – IamWill Dec 25 '19 at 12:10
• Well, it is hard for me to know what your precise interests are. You could start with P. Olver‘s chapter on the calculus of variations which has the advantage that it is easily available online (just google it) and a standard text like „Introduction to the calculus of variations“ by Dacorogna. – user131781 Dec 25 '19 at 13:00