Ok, so I did some more digging and it turns out that Casella and Groutis (yes, again!) wrote a nice paper on approaches to frequentist post-data inferences. I won't summarize the 35 page paper here, but I'll point out the key points for purposes of making my answer self-contained.

**First**, they convincingly demonstrate that post-data confidence assignments cannot be derived using the usual pre-data covering metrics nor by partitioning the sample space. The former is relatively obvious, but the latter goes to the heart of Fisher's admonition on conditional inference via ancillaries -- Casella and Groutis argue that one cannot to rely on ancillary statistics to partition the sample space, since these are not guaranteed to exist and even if they do they can be non-unique (they cite well-known results from Basu here).

**Second**, they critique an early approach by Kiefer to post-data frequentist inference that relied on the experimenter partitioning the sample space. Simply put, one cannot get around the "reference class problem" and so there is no objective basis on which to perform this partitioning.

Unfortunately, if you don't create some sort of **coarse** partition (i.e., containing more than just the observed sample), then you are unable to develop a suitable reference set for coverage calculations (you're basically left evaluating the apriori likelihood of each parameter...quite a non-frequentist concept). Bayesians and Likelihoodists address this by inferring directly from the likelihood (with prior for Bayesians) at the expense of repeated-sampling guarantees on error rates.

## Post-Data Confidence as an Estimator

The "big idea" Casella and Groutis discuss is that freqentist post-data confidence should be seen as an *estimation* problem, not a probability calculation problem (i.e., calculating pre-data coverage probability). They make this transition by defining "confidence" as an estimate of the mean value of an *indicator* function for the non-random interval actually observed (I'll reproduce their math here because it is the heart of the approach):

$$P_{\theta}(\theta \in C(X)|X=x) = P_{\theta}(\theta \in C(x)) = I(\theta \in C(x))$$

The development of frequentist post-data inference follows from above using two obvious mathematical ideas:

- A fixed interval either does or does not contain a given number (e.g., $[2,3]$ does
**not** contain $5$).
- We can replace probabilities with expectations when we use indicator functions over a set.

They then discuss how one should evaluate post-data confidence using a *confidence estimator* $\gamma(X)$ which can then be assessed using a loss function (just like any other estimator). They showed this for the squared-error loss:

$$R(\theta,\gamma(X)) = E_{\theta}\left[I\left(\theta \in C(X)\right) - \gamma(X)\right]^2$$

Using this approach, we can then evaluate various confidence estimators to determine (a) their overall accuracy and (b) which one to use.

For example, if $R(\theta,\gamma_1) \leq R(\theta,\gamma_2) \; \forall \theta$ then we would want to use the estimates provided by $\gamma_1$ as a post-data measure of confidence (I'd add that we should probably want $RSME(\gamma) < \epsilon$ for some small $\epsilon$ like $0.05$ ... but this is a theory paper so that's probably not its focus).

Following the above technique, a post-data confidence assessment would then read like this: "$C(x)$ contains the true parameter with confidence $\gamma(x)$", where it is understood that "confidence" is an estimate of the expected value of the indicator.

Before data is collected, the expected value of $\gamma(X)$ provides a pre-data measure of confidence in the *procedure* defined by $\langle C(X),\gamma(X)\rangle$. Hence $E[\gamma(X)]$ resembles a coverage probability, so we can set our pre-data confidence using the properties of $\gamma(X)$ to get pre-data estimate of coverage as well (i.e., the usual theory).

## Discussion

While I find this a *theoretically* satisfying result (which is why I'm posting this as an answer to my question), I find it rather useless in *practice*. Casella and Groutis hint as much by way of a single sentence on page 19:

*"Construction of such estimators $\mathrm{[\gamma(X)]}$ is sometimes technically involved."*

To say that this is an understatement would in itself be an understatement. Even for well-known, analytically "nice" distributions like the normal distribution, the underlying mathematics is rather intense. I can't imagine deriving estimators for mixed-effects models and those with nuisance parameters in an analytically tractable way.

However, non-parametric/machine-learning approaches to developing estimators via simulations across broad grid of parameter values may yield interesting shortcuts for more realistic scenarios...but I'm not holding my breath. I think I'll stick with Bayesian inference with posterior model checking and frequentist evaluations.