Parameter estimation using bayesian update on moduli space?

Question

Scientists take a set of data points, say in ${\mathbb R}^2$, and, assuming that this data should fit a polynomial of degree $d$ (or an exponential, etc.), they estimate parameters.

I would think that a Bayesian update for probability distributions on a moduli space $M$ (say of degree $d$ polynomials in ${\mathbb R}^2$) would be the way to model this. Each time we are given a point $(x,y)$, we accordingly update our distribution on $M$. For example, if $M$ is the space of linear functions $y=mx+b$ then data points $(1,4), (2,5), (3,6)$ would increase the density of a distribution around the point classifying $y=x+3$.

Question 1: Does this perspective make any sense whatsoever? If so, is there a reference that might help mature my understanding?

I have questions about what happens when we change the model. Imagine a situation that we initially model as a degree $3$ curve in ${\mathbb R}^{\{x,y\}}$ and then realize, no, our situation is better understood as a degree $4$ curve in ${\mathbb R}^{\{x,y,z\}}$, divisible by $z$. The reason we had thought it was a degree $3$ curve in ${\mathbb R}^{\{x,y\}}$ is because we had been implicitly assuming $z=4.5$.

Suppose I want to change the model. Can I use maps of moduli spaces to make any guarantees about how our Bayesian reasoning in the new model aligns with our reasoning in the old model?

Question 2 Can you prove or find a counterexample to the following: Given a map of moduli spaces $f:M_1\to M_2$, a distribution on $M_1$, and data in $M_1$, performing Bayesian update in $M_1$ and then pushing forward the resulting distribution to $M_2$ is the same as (commutes with) pushing the distribution and the data forward to $M_2$, and doing the Bayesian update there?

Answer 1

The following example is central to modern macroeconomics. I think but am not certain that it's the kind of thing you're looking for.

Let $A$ be a space that parameterizes fundamental variables like tastes and technology.

Let $B$ be a space that parameterizes various stochastic processes, any one of which could describe the evolution of the money supply (or any other policy variable you care about).

Let $C$ be a space that parameterizes the current value of the money supply.

Let $E$ parameterize some variable of interest such as, say, employment.

Then employment is completely determined as a function $f:A\times B\times C\rightarrow E$. (Actually, the domain should include a fourth factor for ``stuff our model left out'', but let's ignore that.) And we don't have to estimate this function --- we can compute it from pure theory, by assuming that firms and consumers maximize subject to the constraints imposed on them by monetary policy and their interactions with each other.

Note that in general, $f(a,b,c)\neq f(a,b',c)$ --- in other words, even if we fix tastes, technology and the current value of the money supply, you can't predict employment without knowing the stochastic process that generats that current value --- because a change in the stochastic process changes consumers' and firms' expectations about the future, which affects the choices they make today.

Let's suppose the value of $a$ is fixed but unobservable.

Now if monetary policy (where a "policy" is a stochastic process) is stable for a long time with value $b$, then we get to make lots of observations of the function $g(c)=f(a,b,c)$ which gives employment as a function of the money supply. We can make a good Bayesian estimate of this function and use it to make predictions. Let's say the function $g$ lives in some function space I'll call $Hom(C,E)$.

Now suppose the government changes to a new monetary policy $b'$. Then the function $g(c)$ is replaced by a new function $g'(c)=f(a,b',c)$. Before adopting the new policy, the government would like to know what this new function is going to be.

At this point, we know the old policy $b$ and we know the old money/employment relationship $g=f(a,b,-)$. Now suppose there happens to be a function $H:B\times Hom(C,E)\rightarrow A$ with the property that $H(b,f(a,b,-))=a$. Then we can predict that the new money/employment relationship will be given by $g'(c)=f(H(b,g),b',c)$.

Economists became very sensitive to these issues in the mid-1970s. So (and of course I am way oversimplifying here) the "old" macroeconomics focused on estimating the function $g$, whereas the "new" macroeconomics has focused on the existence and properties of the function $H$.

One moral of that literature is that anything resembling commutativity is hopeless. That is, if we take $M_1=A\times B\times C$ and $M_2=C$, then it matters a lot whether you do your Bayesian updating over functions defined on $M_1$ or on $M_2$. The seminal example is in a paper of Lucas called "Expectations and the Neutrality of Money", but the fundamental idea appears also in Lucas's (probably more easily readable) paper "Econometric Policy Evaluation: A Critique".