I'm trying to use the log-determinant to regularize an optimization problem. To make the argument work, I need to bound the second derivative of the log-determinant.

I need to prove that $\text{Tr}\left( \left(A^{-1} B\right)^2\right) \geq 1$ whenever:

1. $A$ is positive
2. $B$ is symmetric and has at least one entry which is $\geq 1$.
3. The diagonal entries of $A$ are 1.

From messing around a bit it seems like in fact there are always unit vectors $v, w$ such that $v^T A^{-1} B w \geq 1$, which would imply the conclusion and seems like it should be easy enough to confirm or deny, but I'm stumped. Any thoughts?