5
$\begingroup$

Suppose I have some random variable X with probability distribution P(.;theta). Suppose I have a single sample x from this distribution.

Does it make any sense to ask for the derivative of x with respect to theta?

I believe it does not, but am having trouble convincing some of my colleagues.

Improve this question
$\endgroup$
2
  • $\begingroup$ in the end, what did you end up telling them to convince them? $\endgroup$
    – Charlie Parker
    Commented Aug 19, 2016 at 2:08
  • $\begingroup$ actually Ian, I don't understand your question. Are you taking the derivative of P with respect to x (the sample) or $\theta$? I can't conceive how it makes sense to take the derivative wrt to x if x is a sample. If its a sample, then its observed and fixed, so taking the derivative wrt to it is simply zero. Or am I missing something? $\endgroup$
    – Charlie Parker
    Commented Aug 19, 2016 at 23:11

2 Answers 2

Reset to default
2
$\begingroup$

It does not because $x$ is just some element of your state space. You could conceivably choose $x$ as your sample point for any or all of the $\theta$'s. So what it would make sense to differentiate with respect to $\theta$ is $$\mathbb{P}_{\theta}(X=x)$$ as long as you have a discrete distribution. For a continuous distribution you could substitute $x$ with some interval.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I read your answer but I didn't understand it. Why is my counter example wrong. Consider $P_{\theta}(X = x) = \theta^x (1 - \theta)^{1-x}$ and let $x=1$ be the sample observed. Then $P_{\theta}(X = 1) = \theta $. One can easily take the derivative of that function, it has $\theta$ as a variable, its derivative is simply 1. Why is that not correct? $\endgroup$
    – Charlie Parker
    Commented Aug 19, 2016 at 2:12
  • $\begingroup$ @Pinocchio because you did not take the derivative of little x as a function of theta but rather of the probability that big X = 1 as a function of theta. $\endgroup$
    – BSteinhurst
    Commented Aug 19, 2016 at 22:59
  • $\begingroup$ now that Im re-reading the question I think I misunderstood it. It seems that the issue was that he couldn't take the derivative of of x, of course if x is an observed sample its a fixed number so taking a derivative of it leads to 0. It seems that taking the derivative of $\theta$ remains sensible even with samples observed (i.e. its similar to MLE, maximum likelihood estimation). $\endgroup$
    – Charlie Parker
    Commented Aug 19, 2016 at 23:08
5
$\begingroup$

What might make sense is if $X$ is a function $g(U,\theta)$ of some underlying random variable $U$ (with distribution not depending on $\theta$) and $\theta$, where the differentiable function $g$ is chosen so that $g(U,\theta)$ has the given (continuous) distribution. Then you could say $\dfrac{dX}{d\theta} = \dfrac{\partial g}{\partial \theta}(U,\theta)$. Of course this depends on the "implementation", i.e. the choice of $g$ and $U$, rather than just on the distribution of $X$.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .