My question concerns an operation on probability distributions which has arisen in some applied research. It is well-defined mathematically (at least in a limited context), but I don't know how to interpret it or express it formally despite the fact that I am well-versed in the language of measure-theoretic probability. I am hoping an interpretation, whether within standard probability theory or not, could lead to ways to extend this idea to other settings or see why it should not extend (maybe it's not internally consistent or something). For simplicity let's assume all probability distributions are discrete or have densities, along with whatever other technical assumptions make you happy.

Suppose we observe independent samples $y_1,\ldots,y_n$ from a parametrized family of distributions $p(y\mid \theta)$ over a finite set $Y$. As per the usual Bayesian framework we assume we have some prior distribution $p(\theta)$ on the parameter and we wish to compute the posterior given the data, $p(\theta \mid y_1,\ldots, y_n)$. The proper thing to do is to normalize with respect to $\theta$ the joint distribution $p(\theta)\prod_{i=1}^n p(y_i\mid \theta)$ evaluated at the data.

Since the observations are in a finite set we could just as easily write this expression as $p(\theta)\prod_{y\in Y} p(y\mid \theta)^{n_y}$, where $n_y$ denotes the number of observations equal to $y$. Now this expression can be evaluated for any $n_y\in [0,\infty)$, not just integers. Since $(p(y\mid \theta)^{1/2})^2 = p(y\mid\theta)$, two "half samples equal to $y$" make a "whole sample equal to $y$". **But what is a fractional sample from a probability distribution?**

Motivation: In my setting instead of individual samples I have a distribution $p(y)$ over observation values. One reasonable-seeming way to use this to infer $\theta$ is to assign some weight $\beta\in (0,\infty)$ to the observations en masse and let $n_y = \beta p(y)$, which is not necessarily an integer. Even if it is, we do not actually observe at any point $\beta$ "samples" of which $\beta p(y)$ of them were equal to $y$. We only receive the values $p(y)$ for all $y\in Y$. One way of looking at $\beta$ is as a degree of trust in our data $p(y)$ relative to the prior. We could let $\beta$ go to infinity, but this would imply infinite trust in the data relative to the prior $p(\theta)$, which is not appropriate in our case. How to select $\beta$ is another question which I'll ignore for now.

Mathematically, this sort of trick lets us break "samples" into arbitrary small pieces and put them back together again in a consistent way. We could even make these samples infinitesimal and replace the joint distribution evaluated at the data by $p(\theta)\exp\left(\beta\int_Y \log p(y\mid \theta)dp(y)\right)$.

**Is there a theory of such fractional or infinitesimal observations?** In addition to describing the above, I would expect such a theory to answer questions like, "What is the empirical distribution of $4$ units of mass worth of infinitesimal fair coin flips?" in a consistent manner -- presumably with a continuous probability distribution on $[0,1]$ representing the "observed" fraction of heads. This would be somehow analogous to the role a binomial distribution plays for the usual notion of samples. I have some intuition that perhaps a beta distribution is the right thing, but I can't say why because I'm not sure the question is even meaningful.

I've done a bunch of googling combining words like "fractional" or "partial" or "infinitesimal" with words like "sample" and "observation" and "likelihood" to no avail, probably because there are plenty of more reasonable ways to combine these words. **Has such a thing been studied? If so, where can I read about it?**

4more comments