Your $f(x) := \langle x^{\otimes r}, A x^{\otimes r}\rangle$ is a homogeneous polynomial of degree $2r$ of $d$ variables, and $q(x) := ||x||^2$ is a homogeneous quadratic polynomial, so you look for the maximum of the rational function $g(x) := \frac{f(x)}{q(x)^r}$ under the assumption that $q$ is everywhere positive.

In fact, the linear map $S^2 \big(V^{\otimes r}\big)^* \to S^{2r} V^*$ is surjective, and polynomials $f$ in the image of the positive-definite cone are so-called [polynomials SOS][1], i.e. sums of squares of degree $r$ polynomials.

The critical points of $g$ are given by $d\log(f) = r\cdot d\log(q)$, i.e. $d$ equations of $d$ variables of degree $2r$ each,
$\frac{\partial{f}}{\partial x_i} \cdot (x_1^2 + \dots + x_d^2) = 2 x_i \cdot f(x)$, and I'm afraid no formula is much "closer" than this.

Geometrically you can also think of the function $g(x)$ as a pencil of degree $2r$ hypersurfaces in $(d-1)$-dimensional projective space $\mathbf{P}(V)$, such that the fiber over $\infty$ is non-reduced, a quadric $q(x)=0$ with multiplicity $r$, and the fiber over $0$ if $f(x)=0$, and then you are looking for the biggest $R$ such that fiber over $R$ is singular and has at least one real singular point. For generic $f(x)$ the function $g(x)$ is Morse, so the number $D(d,r)$ of its complex critical values can be computed topologically and explicitly, from Euler characteristics. Your number $R = max g(x)$ is a root of the discriminantal polynomial $P(z) \in \mathbf{R}[z]$ of one variable and degree $D(d,r)$. For generic $f(x)$ (and $d,r$) this polynomial $P$ will be irreducible, so in some sense the closest formula is saying that $R$ is the biggest real root of $P$.

Maybe there exists some more practical non-algebraic formula, like a continued fraction, based on some iterative approximations.


  [1]: https://en.wikipedia.org/wiki/Sum-of-squares_optimization#Sum-of-squares_background