Hard to answer it since every example you gave arises as 'naturally' in machine learning as optimization does.

I believe if (2) has all eigenvalues with positive real parts, then it is possible. The trace is not enough to judge that.

For global stability of a fixed point (say 3 here), you need derivative of Lyapunov function to be negative definite everywhere. However, any function you pick will have derivative zero at the "other" fixed point (2).

Yes, you can have local lyapunov function, but not global even if the other fixed point is unstable.

Have you looked into compressive sensing literature, specifically lasso problem where 0 norm is replaced by one norm without changing the solution.

It will help clarify your conjecture (it could be true if all eigenvalues for 2 are non-negative and for 3 are non-positive). In any case, you cannot have a global lyapunov function simply because 2 is a fixed point and hence 3 cannot be globally stable fixed point.

What do the eigenvalues of linearized system look like at the two fixed points?

It depends on the class of perturbations one maybe interested in. For example, in study of navier stokes, it is common to consider symmetric subspace stability, i.e. only consider initial perturbations that respect a certain symmetry possessed by the solution under consideration , and then study the growth of those perturbations. So this process does infact consider the 'other' types of symmeteries than just orbital

What does the problem being convex have to do with "handy expression" ? Convex just means you get a global minima. The expression I was referring to is the output of matlab from method 1.

Why do you believe a handy expression exists? In other words, why so you ecpect a simpler expression than the result of your method 1.

PDEs can come from other sources than physics: geometry , stochastic processes and economics are examples of such fields.

IN general, the strategy is to show that the correct diagonal matrix is the only stable fixed point, and hence the convergence is exponential. Are you looking to prove more than this ?