Weird claims and conclusions in "Introduction to Shape Optimization" I'm trying to understand the notions of Euler and Hadamard derivatives of shape functionals. All the lecture notes and papers on this topic that I've found seem to build up on the books Shapes and Geometries and Introduction to Shape Optimization, which are both co-authored by Zolésio.
I've got a hard time trying to understand what they are doing. And if I'm not totally wrong (which is not unlikely) many things they are claiming don't make sense.
The basic idea should be to consider what happens to shape functions under an infinitesimal perturbation of the shape. So, it make sense to consider families $(T_t)_t$ of transformations $T_t$. But here starts the pain. The following excerpt is taken from section 2.9 of Introduction to Shape Optimization:



I don't even know where to start:

*

*I heavily doubt that the conclusion in the line immediately below (2.74) is correct (and it's weird that they use $C([0,\epsilon))$ on the rhs of (2.74), since this is usually a space of real-valued functions)

*How can (2.75) be well-defined if $t\mapsto T_t(x)$ is not even assumed to be differentiable for fixed $x$?

*And even if we assume that both $t\mapsto T_t(x)$ and $t\mapsto T_t^{-1}(x)$ are $C^1$-differentiable (which they do in a few sections before), I don't think that we can conclude (2.76); neither as stated with $C(0,\epsilon,C^k(\overline D,\mathbb R^N)$ nor with $C^1(0,\epsilon,C^k(\overline D,\mathbb R^N)$ as they seem to assume later on.

Now let's take a look at the definition of the Euler derivative:



I don't know what the space $\mathcal D(\mathbb R^N,\mathbb R^N)$ is, since they haven't defined this space at any point. From the notation it seems to be a space of distributions, but from its usage this doesn't seem to be the case. It's not clear to me how their notion of "shape differentiable in direction $V$" depends on $k$ and I actually don't even understand why we need $V\in C(0,\epsilon;V^k(D))$. In fact, it should be sufficient to assume that $T_t$ is any family of $C^1$-diffeomorphisms on $\mathbb R^N$ for $t\in[0,\epsilon)$ with $T_0=\operatorname{id}_{\mathbb R^N}$, $[0,\tau)\ni t\mapsto T_t(x)$ is differentiable for $x\in\mathbb R^N$ and $V_t:=\frac\partial{\partial t}T_t\circ T_t^{-1}$ for $t\in[0,\tau)$.
I guess, in analogy to the Fréchet derivative on Banach spaces, one wants to obtain a bounded linear operator $V\mapsto{\rm d}J(\Omega;V)$ and that's why we need to take $V$ from a suitable function spaces. I've seen other sources taking $V$ from some kind of Lipschitz functions or to be independent of time and from some Sobolev space. I'm really lost at this point by these apparently conflicting definitions.
Is there any better reference on this topic? I don't want to dive to deep into this stuff. It's sufficient to me to have rigorous treatment of basic shape functionals given by basic domain and boundary integrals which may or may not depend on the shape itself.
 A: Not an answer, but too long for a comment. The general idea with this stuff seems to be to pair your family $\mathscr{D}$ of admissible domains with a(ny) suitable normed-space $\mathscr{V}$ of vector fields and then insist that the 'shape derivative' be the element of $\mathscr{V}^*$ such that
$$
J(\Omega+V) = J(\Omega) + J'(\Omega)V + o(\Vert V\Vert)
$$
as $\Vert V \Vert\to 0$ in $\mathscr{V}$ (where $\Omega+V$ is either $\{x+V(x):x\in \Omega\}$ or something similar). This seems like the minimal property which a 'derivative' should satisfy in an affine setting (domains are 'points', vector fields are 'vectors').
As for what 'suitable' means in this context will - I think - generally depends on what sort of regularity you want for the associated flow. It's common to choose $\mathscr{V}$ so that its elements are Lipschitz continuous because then you can apply the Picard–Lindelöf theorem to associate a unique $C^1$ path germ with every point of $\Omega$.
The one-parameter flow seems like a bit of a distraction in all this - choose $\mathscr{V}$ right and you'll get the properties you want from $(T_t)_{t>0}$ from an appropriate ODE existence theorem.
Note: I've gone for the 'full' (Frechet-like) shape derivative above, you could also work just in terms of directional derivatives $\nabla_VJ(.):\mathscr{D}\to \mathbb{R}$ defined by requiring that
$$
J(\Omega+tV)  =J(\Omega)+t(\nabla_VJ)(\Omega)+o(t) \;\;\mbox{as $t\to 0$},
$$
Either way, I think it's best to choose $\mathscr{D}$ and $\mathscr{V}$ based on where you want to go, and define 'derivatives' in terms of their essential property of being the linear bit of a first-order Taylor expansion (and not get too hung up on the setup used in any particular book).
This is all just my opinion of course :)
