Motivated by the same issue Berger and Boos (1994)Berger and Boos (1994) proposed to maximize over a confidence set for $\theta$ (ie a subset of $B_0$), so may be a good starting point to look: https://www.tandfonline.com/doi/abs/10.1080/01621459.1994.10476836
They. They give a nice overview of various alternatives that have been proposed and interesting examples where the usual p-value is unsatisfactory (illustrating your point).
Instead of changing the definition of p-value Lehmann suggested to replace the usual "maximum likelihood ratio" by an averaged likelihood ratio. He also give interesting examples where the usual approach fails.
In spite of all its possible problems, I keep being surprised how well the usual maximum likelihood ratio test works in many situations.