The proposed result holds true.
I am assuming throughout that $spd$ means symmetric positive definite and that the matrices $A$ and $B$ are $n$-by-$n$ matrices over $\mathbb{R}$ for some $n > 0$.
Indeed, since $B = P^{\top}P$ for some $P \in GL_n(\mathbb{R})$ by hypothesis (see e.g., Sylvester's law of inertia), we can assume, without loss of generality, that $B = I_n$, the $n$-by-$n$ identity matrix.
Let us assume that $A - B = A - I_n$ is positive semi-definite and let $O$ be an $n$-by-$n$ orthogonal matrix over $\mathbb{R}$ such that $O^{\top}AO$ is diagonal. Clearly, the matrices $O^{\top}(A - I_n)O$ and $O^{\top}(I_n - A ^{-1})O$ are also diagonal. Conjugating then both sides of the identity $A - I_n = A(I_n - A^{-1})$ by $O$, it immediately follows that $I_n - A^{-1}$ is positive semi-definite, hence the result.