Problem 1 is essentially finite dimensional and quite geometric in nature.

For a real square matrix $M$ of order $n$ and $1\le p\le\infty$ denote $\big| M\big|_p$ its $p$-trace norm.

Consider the $n\times n$ matrix $Q$ with entries $Q_{ij}:=(q_0^i, q_1^j)_{ 2}:=\int_0^1q_0^i(x)q_1^j(x)dx$. Then, for any $A\in SO(n)$

$$\|q_0 - Aq_1\|_2^2:=\int_0^1\big|q_0(x) - Aq_1(x)\big|^2dx=\|q_0\|_2^2 +\|q_1\|_2^2- 2\int_0^1\sum_{ij} A_{ij}q_0^i(x) q_1^j(x)dx $$

$$=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \sum_{ij} A_{ij}Q_{ij}=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \operatorname{tr}(A^TQ). $$

Therefore, the initial minimization reduces (up to an additive constant) to maximizing the trace of a matrix in the $SO(n)$-orbit of $Q$. For any $A\in SO(n)$ we have, by duality,

$$ \operatorname{tr}(A^TQ)\le |A|_\infty |Q|_1= |Q|_1 .$$

The nuclear norm of $Q$ is indeed attained by a suitable $A\in SO(n)$, and here the SVD of $Q$ may help. Write $Q=UDV^T$ with $D$ non-negative diagonal, and choose $A:=UV^T$; then $A^TQ=VDV^T$ and by invariance of the trace $\operatorname{tr}(A^TQ)=\operatorname{tr}(D)= |Q|_1 .$ $$*$$

Note that the latter optimization problem is also equivalent to a point-set distance minimization, that is, finding the closest rotation to $Q$ in the Frobenius distance. Indeed, for any rotation $A$, since $\big|A\big|_2^2=n$ we have

$$ \big| A-Q\big|_2^2 = n + \big|Q\big|_2^2 -2\operatorname{tr}(A^TQ)$$ so that maximizing $\operatorname{tr}(A^TQ)$ is again equivalent to minimizing a distance.

Problem 1 is essentially finite dimensional and quite geometric in nature.

For a real square matrix $M$ of order $n$ and $1\le p\le\infty$ denote $\big| M\big|_p$ its $p$-trace norm.

Consider the $n\times n$ matrix $Q$ with entries $Q_{ij}:=(q_0^i, q_1^j)_{ 2}:=\int_0^1q_0^i(x)q_1^j(x)dx$. Then, for any $A\in SO(n)$

$$\|q_0 - Aq_1\|_2^2:=\int_0^1\big|q_0(x) - Aq_1(x)\big|^2dx=\|q_0\|_2^2 +\|q_1\|_2^2- 2\int_0^1\sum_{ij} A_{ij}q_0^i(x) q_1^j(x)dx $$

$$=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \sum_{ij} A_{ij}Q_{ij}=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \operatorname{tr}(A^TQ). $$

Therefore, the initial minimization reduces (up to an additive constant) to maximizing the trace of a matrix in the $SO(n)$-orbit of $Q$ by (left or right) multiplication. Consider first the case $\operatorname{det}(Q)\ge0$. For any $A\in SO(n)$ we have, by duality,

$$ \operatorname{tr}(A^TQ)\le |A|_\infty |Q|_1= |Q|_1 .$$

The nuclear norm of $Q$ on the RHS is indeed attained by a suitable $A\in SO(n)$ on the LHS, and here the SVD of $Q$ may help. Write $Q=UDV^T$ with $U, V$ in $SO(n)$ and $D$ non-negative and diagonal, and choose $A:=UV^T$; then $A^TQ=VDV^T$ and by invariance of the trace $\operatorname{tr}(A^TQ)=\operatorname{tr}(D)= |Q|_1 .$ $$*$$

Note that the latter optimization problem is also equivalent to a point-set distance minimization, that is, finding the closest rotation to $Q$ in the Frobenius distance. Indeed, for any rotation $A$, since $\big|A\big|_2^2=n$ we have

$$ \big| A-Q\big|_2^2 = n + \big|Q\big|_2^2 -2\operatorname{tr}(A^TQ)$$ so that maximizing $\operatorname{tr}(A^TQ)$ is again equivalent to minimizing a distance.

Problem 1 is essentially finite dimensional and quite geometric in nature.

For a real square matrix $M$ of order $n$ and $1\le p\le\infty$ denote $\big| M\big|_p$ its $p$-trace norm.

Consider the $n\times n$ matrix $Q$ with entries $Q_{ij}:=(q_0^i, q_1^j)_{ 2}:=\int_0^1q_0^i(x)q_1^j(x)dx$. Then, for any $A\in SO(n)$

$$\|q_0 - Aq_1\|_2^2:=\int_0^1\big|q_0(x) - Aq_1(x)\big|^2dx=\|q_0\|_2^2 +\|q_1\|_2^2- 2\int_0^1\sum_{ij} A_{ij}q_0^i(x) q_1^j(x)dx $$

$$=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \sum_{ij} A_{ij}Q_{ij}=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \operatorname{tr}(A^TQ). $$

Therefore, the initial minimization reduces (up to an additive constant) to maximizing the trace of a matrix in the $SO(n)$-orbit of $Q$. For any $A\in SO(n)$ we have, by duality,

$$ \operatorname{tr}(A^TQ)\le |A|_\infty |Q|_1= |Q|_1 .$$

The nuclear norm of $Q$ is indeed attained by a suitable $A\in SO(n)$, and here the SVD of $Q$ may help. Write $Q=UDV^T$ and choose $A:=U(\operatorname{sgn}D)V^T$; then $A^TQ=V|D|V^T$ and by invariance of the trace $\operatorname{tr}(A^TQ)=\operatorname{tr}(|D|)= |Q|_1 .$ $$*$$ Incidentally, note that such $Q$ may be any square matrix of order $n$, so we may really forget about $q_0$ and $q_1$ and only keep a generic square matrix $Q$ of order $n$ as datum. Also note that the latter optimization problem is also equivalent to a point-set distance minimization, that is, finding the closest rotation to $Q$ in the Frobenius distance. Indeed, for any rotation $A$, since $\big|A\big|_2^2=n$ we have

$$ \big| A-Q\big|_2^2 = n + \big|Q\big|_2^2 -2\operatorname{tr}(A^TQ)$$ so that maximizing $\operatorname{tr}(A^TQ)$ is again equivalent to minimizing a distance.

Problem 1 is essentially finite dimensional and quite geometric in nature.

For a real square matrix $M$ of order $n$ and $1\le p\le\infty$ denote $\big| M\big|_p$ its $p$-trace norm.

Consider the $n\times n$ matrix $Q$ with entries $Q_{ij}:=(q_0^i, q_1^j)_{ 2}:=\int_0^1q_0^i(x)q_1^j(x)dx$. Then, for any $A\in SO(n)$

$$\|q_0 - Aq_1\|_2^2:=\int_0^1\big|q_0(x) - Aq_1(x)\big|^2dx=\|q_0\|_2^2 +\|q_1\|_2^2- 2\int_0^1\sum_{ij} A_{ij}q_0^i(x) q_1^j(x)dx $$

$$=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \sum_{ij} A_{ij}Q_{ij}=\|q_0\|_2^2 +\|q_1\|_2^2- 2 \operatorname{tr}(A^TQ). $$

Therefore, the initial minimization reduces (up to an additive constant) to maximizing the trace of a matrix in the $SO(n)$-orbit of $Q$. For any $A\in SO(n)$ we have, by duality,

$$ \operatorname{tr}(A^TQ)\le |A|_\infty |Q|_1= |Q|_1 .$$

The nuclear norm of $Q$ is indeed attained by a suitable $A\in SO(n)$, and here the SVD of $Q$ may help. Write $Q=UDV^T$ with $D$ non-negative diagonal, and choose $A:=UV^T$; then $A^TQ=VDV^T$ and by invariance of the trace $\operatorname{tr}(A^TQ)=\operatorname{tr}(D)= |Q|_1 .$ $$*$$

Note that the latter optimization problem is also equivalent to a point-set distance minimization, that is, finding the closest rotation to $Q$ in the Frobenius distance. Indeed, for any rotation $A$, since $\big|A\big|_2^2=n$ we have

$$ \big| A-Q\big|_2^2 = n + \big|Q\big|_2^2 -2\operatorname{tr}(A^TQ)$$ so that maximizing $\operatorname{tr}(A^TQ)$ is again equivalent to minimizing a distance.