Suppose an urn contains unknown but non-random numbers of red and green marbles, and I take a random sample of a known and non-random size. Observing the numbers of red and green marbles in the sample, I need to hazard my best guess as to the proportion (not the total number) of red marbles in the urn.

The number of *distinct* marbles I observe is likely to be larger if the sample is taken *without* replacement than *with* replacement. Therefore, there should be less uncertainty in my estimate if I sample *without* replacement.

**THEREFORE** the standard deviation of the random number of red marbles in my sample is smaller if I sample *without* replacement than *with*.

Clearly the above is not a logically rigorous argument. (Of course, one can establish the same result by easy standard arguments, but that will be off-topic here.)

Now suppose I take a sample of $n$ independent measurements $X_1,\dots,X_n$ from a population that is normally distributed with unknown mean $\mu$ and with variance $\sigma^2$. Let $\overline{X} = (X_1+\cdots+X_n)/n$ and $S^2 = \sum_{i=1}^n (X_i - \overline{X})^2/(n-1)$ be respectively the sample mean and sample variance. Then
$$
\frac{\overline{X} - \mu}{\sigma/\sqrt{n}}
$$
is normally distributed with *known* mean and variance 0 and 1 respectivly, so the endpoints of a confidence interval for $\mu$ are
$$
\overline{X} \pm A \frac{\sigma}{\sqrt{n}}
$$
where $A$ is a suitable percentage point of the normal distribution (provided (unrealistically) that $\sigma$ is known).

And $$ \frac{\overline{X} - \mu}{S/\sqrt{n}} $$ has a Student's t-distribution with $n-1$ degrees of freedom, so $$ \overline{X} \pm B_n \frac{S}{\sqrt{n}} $$ where $B_n$ is the corresponding percentage point of Student's distribution, are the endpoints of a corresponding confidence interval if (more realistically) $\sigma$ is unknown.

There is more uncertainty if $\sigma$ is unknown than if $\sigma$ is known, **THEREFORE** $B_n > A$.

If $\sigma$ is unknown, then the amount of uncertainty decreases as the sample size $n$ increases, **THEREFORE** $B_n$ decreases as $n$ increases. (Of course, the fact that $B_2 > B_3 > B_4 > \cdots > A$ can be proved by standard arguments involving fiddling with integrals.)

The logical leaps corresponding to the **THEREFORE**s above have a seeming intellectual compellingness about them. It would be extraordinarily paradoxical if the conclusions following them failed to hold despite the reasoning preceding them. But these are not logically rigorous arguments.

So **my question is** whether there could be some theorem of logic that says things like those following the **THEREFORE**s are true when things like those preceding them are true.

11more comments