Motivated by the same issue Berger and Boos (1994) proposed to maximize over a confidence set for $\theta$ (ie a subset of $B_0$), so may be a good starting point to look: https://www.tandfonline.com/doi/abs/10.1080/01621459.1994.10476836
They give a nice overview of various alternatives that have been proposed and interesting examples where the usual p-value is unsatisfactory (illustrating your point).