5
$\begingroup$

Suppose I have some random variable X with probability distribution P(.;theta). Suppose I have a single sample x from this distribution.

Does it make any sense to ask for the derivative of x with respect to theta?

I believe it does not, but am having trouble convincing some of my colleagues.

$\endgroup$
2
  • $\begingroup$ in the end, what did you end up telling them to convince them? $\endgroup$ Aug 19, 2016 at 2:08
  • $\begingroup$ actually Ian, I don't understand your question. Are you taking the derivative of P with respect to x (the sample) or $\theta$? I can't conceive how it makes sense to take the derivative wrt to x if x is a sample. If its a sample, then its observed and fixed, so taking the derivative wrt to it is simply zero. Or am I missing something? $\endgroup$ Aug 19, 2016 at 23:11

2 Answers 2

2
$\begingroup$

It does not because $x$ is just some element of your state space. You could conceivably choose $x$ as your sample point for any or all of the $\theta$'s. So what it would make sense to differentiate with respect to $\theta$ is $$\mathbb{P}_{\theta}(X=x)$$ as long as you have a discrete distribution. For a continuous distribution you could substitute $x$ with some interval.

$\endgroup$
3
  • $\begingroup$ I read your answer but I didn't understand it. Why is my counter example wrong. Consider $P_{\theta}(X = x) = \theta^x (1 - \theta)^{1-x}$ and let $x=1$ be the sample observed. Then $P_{\theta}(X = 1) = \theta $. One can easily take the derivative of that function, it has $\theta$ as a variable, its derivative is simply 1. Why is that not correct? $\endgroup$ Aug 19, 2016 at 2:12
  • $\begingroup$ @Pinocchio because you did not take the derivative of little x as a function of theta but rather of the probability that big X = 1 as a function of theta. $\endgroup$ Aug 19, 2016 at 22:59
  • $\begingroup$ now that Im re-reading the question I think I misunderstood it. It seems that the issue was that he couldn't take the derivative of of x, of course if x is an observed sample its a fixed number so taking a derivative of it leads to 0. It seems that taking the derivative of $\theta$ remains sensible even with samples observed (i.e. its similar to MLE, maximum likelihood estimation). $\endgroup$ Aug 19, 2016 at 23:08
5
$\begingroup$

What might make sense is if $X$ is a function $g(U,\theta)$ of some underlying random variable $U$ (with distribution not depending on $\theta$) and $\theta$, where the differentiable function $g$ is chosen so that $g(U,\theta)$ has the given (continuous) distribution. Then you could say $\dfrac{dX}{d\theta} = \dfrac{\partial g}{\partial \theta}(U,\theta)$. Of course this depends on the "implementation", i.e. the choice of $g$ and $U$, rather than just on the distribution of $X$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.