If $M$ is small enough, It should be active. However, I am not sure if the proposed convex constraint gives sparse solutions ($\|x\|_2$ only shrinks coefficients). The $\|\cdot\|_{\infty}$ constraint may only force large coefficients to take the value $\pm sM$. Though there might be some ratio of $M/s$ that leads to sparse solutions, which would be interesting and novel. I actually think your proposed nonconvex constraint is interesting, assuming it does indeed give sparse solutions. Unfortunately, it will be a challenge to implement. To estimate $M$, I would use cross-validation.

I don't think its anti-sparsity. $s = 1/\sqrt{d}$ implies $\|x\|_2 = \|x\|_{\infty}$. So assuming you defined $\|x\|_2 = \sqrt{\sum_i x_i^2}$, you must have exactly one entry equal to $\|x\|_{\infty}$. and the rest equal to zero (since one term contributes $\|x\|_{\infty}^2$ to the inside of the $\sqrt{}$). So this suggests it may have sparse solutions.

Thus, $\log N(\varepsilon, X(\alpha, L), \|\cdot\|_{\infty}) \lessapprox m(1/\varepsilon)^{n/\alpha}$ in the general case.

For the case of general $m$, note that the i-th component function $f_i$ of a vector-valued Holder function $f$ is Holder as well. This follows since $|f_i(x) - f_i(y)| \leq \sup_i |f_i(x) - f_i(y)| \leq \|f(x) - f(y)\|_2 \leq \sqrt{m} \sup_i \|f_i(x) - f_i(y)\|$. Thus, we can $\varepsilon$-cover the space of each $ith$ component functions with $\exp((1/\varepsilon)^{n/\alpha})$ balls. Such a cover for each $i$ implies a cover for vector-valued Holder functions with at most $\Big[\exp((1/\varepsilon)^{n/\alpha})\Big]^m$ balls.

$\log N(\varepsilon,X(\alpha,L), \|\cdot\|_{\infty}) \approx (1/\varepsilon)^{\frac{n}{\alpha }}$ is what they give for $m=1$ (Theorem 2.7.1. in the reference I mentioned).

This type of result is standard in the statistical literature. See e.g. van der Vaart, Wellner, Empirical Process Theory, 1998, chapter 2, where covering numbers are provided for higher-order smoothness classes as well. I believe they take $m=1$ but I imagine their proof can be adapted.

Clearly, the correct definition is that the determinant is any antisymmetric multilinear functional $(\mathbb{R}^{d})^d \equiv \mathbb{R}^{d\times d} \rightarrow \mathbb{R}$ that assigns the value 1 to the canonical basis $e_1, e_2, \dots, e_d$. Obviously, the uniqueness is a trivial exercise left for the reader, so this need not be specified in the definition. The canonical basis is also obvious so this too need not be specified. /s

The wikipedia page on Schauder fixed-point theorem says: " In 1934, Tychonoff proved the theorem for the case when K is a compact convex subset of a locally convex space. This version is known as the Schauder–Tychonoff fixed-point theorem. B. V. Singbal proved the theorem for the more general case where K may be non-compact; the proof can be found in the appendix of Bonsall's book (see references)." en.wikipedia.org/wiki/Schauder_fixed-point_theorem

Yes, these are sufficient. I am not sure if they are necessary.