Do you count problems from shape analysis to be a relevant optimisation problem? Here you try to find for example the distance between shapes (=unparametrised curves) in euclidean space to each other. This is implemented by looking at immersions modulo the orientation preserving diffeomorphisms (quotienting out all possible reparametrisations). A lot in this geometric theory depends on understanding the diffeomorphism groups or working with Riemannian metrics which are right invariant under the action of the diffeomorphisms. There is a lot of literature on this, but maybe the somewhat dated overview https://arxiv.org/pdf/1305.1150.pdf can get you started.

Also we recently wrote a nice little paper on deep neural networks on diffeomorphism groups for the reparametrisation task in shape analysis. There the algorithms exploit the Lie group structure of the diffeomorphisms and in particular there is a chapter deducing (rough) a priori estimates for Lipschitz constants on iterated compositions of diffeomorphism group elements. All of this yields a framework to find the optimal alignment of two curves (one of the basic shape analysis problems). The arXiv version of that paper can be found here: https://arxiv.org/pdf/2207.11141.pdf