Deriving the distribution of standardized variables with empirical mean and standard deviation

Question

I'm working with a set of independent and identically distributed random variables $\{ x_i \}_{i=1}^N$, where each $x_i$ follows a Gaussian distribution $P_X(x) = \mathcal{N}(x; \mu, \sigma^2)$. This notation $\mathcal{N}(x; \mu, \sigma^2)$ represents the Gaussian probability density function (p.d.f.) with mean $\mu$ and variance $\sigma^2$. It's straightforward to show that, by defining $y_i \equiv \frac{x_i - \mu}{\sigma}$, we obtain $y_i \sim P_Y(y) = \mathcal{N}(y; 0, 1)$.

I'm interested in exploring a similar transformation but using empirical estimates of the mean and standard deviation instead: $$z_i \equiv \frac{x_i - \hat{\mu}}{\hat{\sigma}}\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(1)$$ where $\hat{\mu}$ and $\hat{\sigma}$ are the empirical mean and standard deviation, respectively, given by: $$\hat{\mu} \equiv \frac{1}{N} \sum_{i=1}^N x_i\; \;\;\;\;\;\;\;\;\;\;\; \;\;\;\;\;\;\;\;\;\;\;\;\;\;(2)$$ $$\hat{\sigma^2} \equiv \frac{1}{N} \sum_{i=1}^N (x_i - \hat{\mu})^2 \;\;\;\;\;\;\;\;\;\;\;\;\;\; (3)$$

My question is: How can we derive the distribution of $z_i$ for a given fixed $N$?.

It's important to note the differences from the initial example:

• Unlike $\mu$ and $\sigma$, which are constants, $\hat{\mu}$ and $\hat{\sigma}$ are random variables dependent on the set ${ x_i }_{i=1}^N$.
• Both $\hat{\mu}$ and $\hat{\sigma}$ are functions of the $x_i$'s.
• The distribution of $z_i$ is generally not normal. For instance, with $N=2$, any two distinct $x_1, x_2$ will be mapped to $z_i \in \{ -1, +1 \}$, resulting in a distribution for $z_i$ of $P_Z(z) = \frac{1}{2} \delta(z-1) + \frac{1}{2} \delta(z+1)$.

Answer 1

Anthony's comment has the correct answer: $Z=(z_1,\ldots ,z_N)$ is uniformly distributed on $\|Z\|^2=N$, $\langle e, Z\rangle=0$, $e=(1,1,\ldots, 1)$ (that is, the joint distribution is the $(N-2)$-dimensional Hausdorff measure restricted to this set and normalized).

As $Z$ is obviously supported by this set, this follows from the fact that the distribution is invariant under rotations $R\in O(n)$ with $Re=e$ and under $Z\mapsto -Z$. To see this, recall that $Y=RX$ has the same joint distribution as $X$. See here.

Moreover, $$ N\overline{Y}=\langle e, Y\rangle =\langle e, RX\rangle = \langle R^te, X\rangle = \langle e, X\rangle = N\overline{X} $$ and similarly $NS_Y^2=\|Y-\overline{Y}e\|^2=\|R(X-\overline{X}e)\|^2=NS_X^2$. Thus $$ RZ = \frac{1}{S_X}R(X-\overline{X}e) = \frac{1}{S_Y}(Y-\overline{Y}e) , $$ and indeed $Z$ and $RZ$ have the same distribution.