Just a quick answer with some approaches sketched. From a theoretical point of view, I would start from considering the even Kronecker canonical form (see e.g. my paper https://doi.org/10.1137/120861679 for a self-contained statement), which is essentially a canonical form for pencils of Hermitian matrices, and check which blocks correspond to semidefinite matrices. This will give you at least an existence proof. From a numerical point of view, at least when the problem is nonsingular and does not have multiple eigenvalues at $\infty$, you can reduce to the case in which $B = \operatorname{diag}(I,0)$, and then I think that the finite eigenvalues of $\begin{bmatrix}A_{11}-zI & A_{12} \\ A_{21} & A_{22}\end{bmatrix}$ are the ones of $A_{11} - A_{12}A_{22}^{-1}A_{21}$.

From the theoretical point of view: there is a Jordan-like canonical form for pairs of Hermitian matrices $(A,B)$ under the equivalence relation $(A,B) \sim (PAP^T, PBP^T)$ (with a nonsingular $P$), see https://www.sciencedirect.com/science/article/pii/0024379576900215 . This is known today in my area as "(a variant of) the even Kronecker canonical form"; see e.g. my paper https://doi.org/10.1137/120861679 which contains a self-contained statement for Hermitian-anti-Hermitian (which is the same up to multiplication by $i$ of one of them).

It is easy to check that only a few of these blocks can be used to produce positive semidefinite $A$ and $B$: if I am not wrong, if $A,B$ are spd then you can only get in the canonical form size-1 blocks corresponding to purely real eigenvalues (E2), size-1 blocks corresponding to infinite eigenvalues (E3), and size-1 singular blocks (E4, corresponding to common kernel of the two matrices). All these blocks are size-1: in particular, this means that this decomposition will produce diagonal $PAP^T,PBP^T$, and hence it is the decomposition that you want. I.e., the form that you ask for exists for all pairs $(A,B)$, and you can obtain it even if you restrict to transformations with $Q=P^T$ (which is nice, since they preserve symmetry and definiteness).

Numerically, you can first identify the common kernel of $A$ and $B$ and then change basis to put it in evidence, to reduce the problem to $$ \begin{bmatrix}A_{11} & 0\\ 0 & 0\end{bmatrix}, \begin{bmatrix}B_{11} & 0\\ 0 & 0\end{bmatrix} $$ and then apply a generalized eigensolver (e.g., Matlab's eig(A11, B11)) to $A_{11}$ and $B_{11}$, which is now a nonsingular problem (i.e., $\det(A+Bz)$ is not identically zero) in view of the previous discussion.