2
$\begingroup$

According to Williams, D.: Weighing the Odds the p-value of observed data in the likelihood ratio setting is defined as

$$\mathrm{p_{val}}(y^{obs}) := \mathrm{sup}_{\theta \in B_0} \mathbb{P}\big(\mathrm{lr}(Y) \geq \mathrm{lr}(y^{obs}) \vert \theta \big)$$

where

$$ \mathrm{lr}(y) := \frac{\mathrm{sup}_{\theta \in B_A} \mathrm{lhd}(\theta, y)}{\mathrm{sup}_{\theta \in B_0} \mathrm{lhd}(\theta, y)} $$

where $\mathrm{lhd}$ is the likelihood function. He further calls it a 'convoluted concept', which got me thinking what is the reason for this comment. One thing I realized, which I did not realize before, is that in the first equation the supremum can be achieved by a value of $\theta^*$ that could have never generated the $y^{obs}$ (could be $f_{\theta^*}(y^{obs}) = 0).$ So I thought maybe it would make sense to define something like

$$\mathrm{p^{mod}_{val}}(y^{obs}) := \mathrm{sup}_{\theta \in B_0} \bigg (\mathbb{P}\big(\mathrm{lr}(Y) \geq \mathrm{lr}(y^{obs}) \vert \theta \big) \mathbb{P}(y^{obs} \vert \theta) \bigg )$$

Has this been done before? Does it have some name? What about if we change it to

$$\mathrm{p^{mod 2}_{val}}(y^{obs}) := \mathrm{sup}_{\theta \in B_0} \bigg (\mathbb{P}\big(\mathrm{lr}(Y) \geq \mathrm{lr}(y^{obs}) \vert \theta \big) \mathbb{I}_{\mathbb{P}(y^{obs} \vert \theta) > 0} \bigg )$$

$\endgroup$

1 Answer 1

2
$\begingroup$

Motivated by the same issue Berger and Boos (1994) proposed to maximize over a confidence set for $\theta$ (ie a subset of $B_0$), so may be a good starting point to look. They give a nice overview of various alternatives that have been proposed and interesting examples where the usual p-value is unsatisfactory (illustrating your point).

Instead of changing the definition of p-value Lehmann suggested to replace the usual "maximum likelihood ratio" by an averaged likelihood ratio. He also give interesting examples where the usual approach fails.

In spite of all its possible problems, I keep being surprised how well the usual maximum likelihood ratio test works in many situations.

$\endgroup$
2
  • $\begingroup$ (You deleted your previous answer posted a few minutes earlier: I'd recommend next time rather edit the previous answer.) $\endgroup$
    – YCor
    Commented Jan 8, 2020 at 15:40
  • $\begingroup$ Thanks for the advice, will do! $\endgroup$
    – Wicher
    Commented Jan 8, 2020 at 17:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .