Existence of matrix diagonalizing $x A + y B$ for all $x, y$ and independent of $x, y$

Question

Let $A_1, A_2$ and $B_1, B_2$ be real symmetric matrices. Suppose $x A_1 + y B_1$ is cospectral with $x A_2 + y B_2$ for all real numbers $x, y$. Is it true that there exists a fixed orthgonoal matrix $Q$ independent of $x, y$ such that $(x A_1 + y B_1) Q = Q (x A_2 + y B_2)$?

I want to generalize Theorem 2 in Johnson, Charles R, and Morris Newman. “A Note on Cospectral Graphs.” Journal of Combinatorial Theory, Series B 28, no. 1 (February 1, 1980): 96–103. https://doi.org/10.1016/0095-8956(80)90058-1.

In the proof of Theorem 2, it says such a matrix can be concluded by a continuity-compactness argument.

Answer 1

This is false in general. Here is a counterexample: $$A_1 = A_2 = \begin{pmatrix} 2 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 6 \\ \end{pmatrix}$$

$$B_1 = \begin{pmatrix} 1 & 2 & 2 \\ 2 & 1 & 2 \\ 2 & 2 & 11 \\ \end{pmatrix}$$

$$B_2 = \begin{pmatrix} 1 & \sqrt{\frac{3}{2}} & -3 \\ \sqrt{\frac{3}{2}} & 1 & \sqrt{\frac{3}{2}} \\ -3 & \sqrt{\frac{3}{2}} & 11 \end{pmatrix}$$

You an easily verify with Mathematica that $\det(x A_1 + y B_1 + z I) = \det(x A_2 + y B_2 + z I)$, so the eigenvalues of $x A_1 + y B_1$ and $x A_2 + y B_2$ are the same for all $x,y$. You can also check that the only matrix $Q$ such that $A_1Q=QA_2$ and $B_1Q = QB_2$ is $Q=0$, i.e. the similarity matrices for $x A_1 + y B_1$ and $x A_2 + y B_2$ cannot be the same at $(x,y)=(1,0)$ and $(x,y)=(0,1)$.

The claim is true if both $B_1$ and $B_2$ have rank one, which is the case for the matrix in the linked paper.