Given a probability model $\mathcal{P}=\{P_{\theta},\theta \in \Theta \}$ dominated by a $\sigma$-finite measure $\lambda$ (e.g. Lebesgue measure) on a locally compact space $\cal{X}$ along with $\sigma$-field $\cal{F}$. A sufficient statistic is defined to be an $\cal{F}$-measurable mapping $T:\cal{X}\rightarrow \mathbb{R}$ such that $P_{\theta}(X\mid T=t)$ does not depend on the $\theta\in\Theta$.

According to Neyman-Fisher factorization theorem, the sufficient and necessary condition that $T$ is a sufficient statistic for $\cal{P}$ is that $$p(X\mid \theta)=g(T,\theta)h(X),\forall\theta \in \Theta,X\in\cal X$$ where $p(X\mid \theta)=\frac{dP_{\theta}}{d\lambda}$ is the density/Radon-Nikodym derivative w.r.t. $\lambda$.

My questions:

(1) Since the conditional probability measure $P_{\theta}(X\mid T)$ is defined as a solution to the functional equation $\int_{B}P_{\theta}(A \mid \sigma(T))dP_{\theta}=P_{\theta}(A\cap B), \forall B\in \sigma(T)$, are there any characterization of the notion of sufficiency in terms of the $\sigma$-field $\sigma(T)$ generated by the $\cal{F}$-measurable mapping $T$ ,AND the $\sigma$-fields associated with $\cal{P}$?

(2) What is the implication of Neyman-Fisher Theorem in terms of the product measure $\lambda_\Theta\times\lambda$, assuming there is some measure $\lambda_\Theta$ given on the parameter space $\Theta$? It doe not seem like "independence" though.

I did not post this on stats.SE because I think it will receive better response here.

  • 1
    $\begingroup$ Does the paper by Halmos and Savage (1949) address your question? Theorem 1 in section 5 is phrased differently than Neyman-Fisher factorization, which is given as a corollary in section 6. projecteuclid.org/euclid.aoms/1177730032 $\endgroup$ – R Hahn Dec 4 '16 at 2:42
  • 1
    $\begingroup$ @RHahn It is a classical paper yet it does not address my problem. I am looking for more or less a relationship between $\sigma$-fields generated by sufficient statistics and those generated by a dominatd model $\cal{P}$. I came up with this question when I studied the gradient flows. I have actually made a few attempts looking into literature because such an inquiry seems naive at the first look. But thank you for your input. $\endgroup$ – Henry.L Dec 4 '16 at 4:15
  • 1
    $\begingroup$ Actually the Neymann-Fisher factorization is also known as Halmos-Savage Theorem due to this paper. See [Casella&Berger,2002] for example. $\endgroup$ – Henry.L Dec 4 '16 at 4:17
  • 1
    $\begingroup$ I thought probably it didn't answer your question, but I wasn't sure exactly what you are asking, so I figured I would point to it. I'm not used to thinking about "$\sigma$-field generated by a dominated model". $\endgroup$ – R Hahn Dec 4 '16 at 4:51
  • 1
    $\begingroup$ @RHahn For example, what I am expecting is some relation like $\sigma(T)\subset\cap_{\theta}\sigma(P_{\theta})$. And possibly some equality holds when the $T$ is minimal sufficient for $\cal{P}$. Halmos-Savage paper led to "partition method" on spaces yet I do not know what "partition method" imply on associated $\sigma$-fields. $\endgroup$ – Henry.L Dec 4 '16 at 5:31

I figured it out by looking at the last few chapters in [Shiryayev] and some thoughts. The problem can be considered in following way with the aid of a formalized definition of conditional expectation.

Without loss of generality we assumed the sample random variable $X$ from the model $\cal{P}$ to be $L^1(P),\forall P\in \cal{P}$. This is easily verified to be true for any $P$ dominated by a $\sigma$-finite measure $\mu$ on the probability space $(\Omega,\mathcal{F},P)$. Also we discuss the situation where $X,T\geq 0,\forall\omega\in\Omega $. Here $$X:\Omega\rightarrow\mathbb{R},\omega\mapsto X(\omega)$$ and $$T:\Omega\rightarrow\mathbb{R},\omega\mapsto T(X(\omega))$$ are random variables. $X$ is the sample random variable; $T$ is the sufficient statistics/random variable.

Invoke Radon-Nikodym Theorem on the measure defined by $\nu(G):=\int_{G}XdP$ on subspace $(\Omega,\sigma(T),P)$ where $\sigma(T)\subset\cal{F}$ is the $\sigma$-field generated by sufficient statistic $T$, then the R-N derivative $\frac{d\nu}{d\mu}$ is the conditional expectation (random variable) $\boldsymbol{E}(X\mid T)(\omega)$. This can be directly checked: $$\int_{G}\boldsymbol{E}(X\mid T)(\omega)d\mu(\omega)$$ $$=\int_{G}\frac{d\nu}{d\mu}d\mu(\omega)=\int_{G}d\nu(\omega)=\nu(G)$$ for any $G\in \sigma(T)$. With this definition, we can directly assert that a conditional expectation is a simple random variable if $A_n$ is a partition of $\Omega$, i.e. $\cup_n A_n=\Omega,A_n\in \sigma(T)$ such that (since we want $1_{A_n}$ to be $\sigma(T)$-measurable functions.) $$\boldsymbol{E}(X\mid T)(\omega)=\frac{\nu(A_n)}{P(A_n)}1_{A_n}(\omega)$$

We know that if a sufficient statistics $T$ for model $\mathcal{P}=\{P\}$ , then from the expression above, the partition $\{A_n\}$ will satisfy that $\frac{\nu(A_n)}{P(A_n)}=\frac{\int_{A_n}XdP}{P(A_n)}=\frac{\int_{A_n}XdP}{\int_{A_n}1dP}$ is independent of the choice of $P\in\cal{P}$.

The level sets of $T$ form a partition as well as generating class(not necessarily atoms) of $\sigma(T)$. So a sufficient statistic will have its level sets partition the $\Omega$ in such a way that the conditional expectation is a constant random variable on each partition component independent of the choice of $P$. In other words, $T$ is a "piece-wise" constant set function, being constant on those sets $A_n\in\cal{F}$ such that $P\{X\in A_n\}\equiv C_n\in\mathbb{R},\forall P\in\cal{P}$

The minimal sufficient statistics $U=\phi(T)$ for some measurable function $\phi$ because $\sigma (U)$ is the coarest partition possible satisfying above requirements. i.e. $\sigma(U)\subset\sigma(T)$ so $U$ must be $\sigma(T)$-measurable, and the relation $U=\phi(T)$ follows from the fact that any $\sigma(T)$-measurable random variable $U$ can be written as $\phi(T)$ for some Borel function $\phi$.

A direct consequence is: there exists a Borel-measurable function $\phi$, $\boldsymbol{E}(U\mid T)=\phi(T)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.