What are the predictive implications of conditional non-commutative probability? To simplify things, let's consider the Hilbert approach to quantum probability over a finite dimensional vector space $V$ of dimension $n$.
In this context a state $S$ is a positive semi-definite trace 1 matrix, and an observable $X$ is any self-adjoint matrix.
The classical probability situation is where all matrices are diagonal, in which case $S$ is equivalent to a probability space over a set of $n$ points, and $X$ is a random variable assigning each sample in the sample space some real number.
In the non-commutative probability situation, the equivalent of sampling $X$ is Born's rule, which prescribes a (classical) probability measure on the eigenvalues of $X$: the probability of $\lambda$ is $tr(Proj_{E_{X, \lambda}}S)$ where $E_{X, \lambda}$ is the eigenspace of $\lambda$ for $X$. This sums up to $1$ because
$$S=\sum_{\lambda}Proj_{E_{X,\lambda}}S,$$
and trace is additive. In the commutative case this jibes with intuition of course.
Conditional probability works as follows: let $Y$ be another observable. Assume that using Born's rule you get $\lambda$ for $X$ (you got the eigenvalue $\lambda$ when you sampled via Born's rule). Then the "conditional Born's rule for $Y$ given that you got $\lambda$ for Born's rule for $X$" is a (classical) probability distribution over the eigenvalues of $Y$ given by the probability of $\lambda'$ being $tr(Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$. Again, in the commutative case this jibes with intuition about what conditional probability means.
I am not seeking direct intuition about what non-commutative probability means (though of course that would be nice!), rather I am attempting to ask the easier question: what is the predictive power of non-commutative probability?
Concretely:
Questions

*

*Assume that you know the probability measure on the eigenvalues of $X$ given by Born's rule (the $tr(Proj_{E_{X,\lambda}}S)$s), but you do not have access to $S$. Further, assume that you know all of the conditional probabilities for $Y$ given measurements of $X$ (all of the $tr(Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$s). Then does that determine the unconditional probability measure on the eigenvalues of $Y$ (the $tr(Proj_{E_{Y,\lambda'}}S)$s)?

*If the answer to the above question is yes, does it also determine the conditional probabilities of $X$ given values for $Y$ (the $tr(Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}S)/tr(Proj_{E_{Y,\lambda'}}S)$s)?

*This question is a little more vague and ambitious... Let's squint so that all of the linear algebra goes away. Each observable has some set of "possible (real) values" (the eigenvalues), and if you only look at one observable and one state, then all of the information in that interaction is just about how likely each value is to occur. Given another observable, the only funny business is in the conditional probabilities. In linear algebra terms, if one first measures $\lambda$ for $X$, then the probability of getting $\lambda'$ for $Y$ changes from $tr(Proj_{E_{Y,\lambda'}}S)$ to $tr(Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S))/tr(Proj_{E_{X,\lambda}}S)$. It kind of looks like there is some metric for how close $\lambda$ for $X$ is to $\lambda'$ for $Y$, and this affects the conditional probability in some way... Is there a way to abstract away all of the linear algebra here and come up with another formulation for non-commutative probability where the eigenvalues are baked into the definition of an observable rather than being eigenvalues of some matrix? In other words -- what is non-commuative probability all about? How should one understand conditional non-commutative probability? What is its predictive power and limitations? (Similar to Questions 1 and 2.) Is there an easier way to codify it? To put it differently, in exactly what ways is non-commutative probability more restrictive than: "for each measurement $\lambda$ of $X$ you can choose any conditional distribution you want for $Y$"? Can that restriction be written with no reference to linear algebra?

UPDATE: As JP McCarthy pointed out, there was an error in the previous version of this question (now fixed). Namely: I wrote that the conditional probability of getting $\lambda'$ for $Y$ given that you got $\lambda$ for $X$ was defined as $tr(Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$ rather than $tr(Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$. But this leads to another question:


*Why would defining the conditional probability of getting $\lambda'$ for $Y$ after getting $\lambda$ for $X$ as $tr(Proj_{E_{X,\lambda}}Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$ rather than $tr(Proj_{E_{Y,\lambda'}}Proj_{E_{X,\lambda}}S)/tr(Proj_{E_{X,\lambda}}S)$ make more sense? (Of course they are equivalent for the classical commutative case.) Is this one of those "shut up and compute moments" where the only answer is that only the latter definition jibes with experiments? That would be incredibly dissatisfying. What property does the latter definition have the that former lacks? Does it make more intuitive sense by some definition of intuition?

 A: I am not so sure about your conditioning. I understand the conditional probabilities should be, where $p_\lambda$ are spectral projections of $X$ and $q_\mu$ spectral projections on $Y$:
$$\mathbb{P}[(Y=\mu)\succ (X=\lambda)\,|\,S]=\frac{\operatorname{tr}(p_\lambda q_\mu p_\lambda S)}{\operatorname{tr}(p_\lambda S)}.$$
rather than:
$$\mathbb{P}[(Y=\mu)\succ (X=\lambda)\,|\,S]=\frac{\operatorname{tr}(q_\mu p_\lambda S)}{\operatorname{tr}(p_\lambda S)}.$$
I am using this "$\succ$" for sequential measurement so that the probability above is "the probability that $Y$ is measured to be $\mu$ after $X$ is measured to be $\lambda$.
I don't believe knowing all of $\mathbb{P}[X=\lambda\,|\,S]$ and $\mathbb{P}[[Y=\mu]\succ [X=\lambda]\,|\,S]$ gives you the $\mathbb{P}[Y=\mu\,|\,S]$. I am kind of in a rush so hopefully there are no dumb mistakes here.

Define
$$X=\begin{pmatrix}\frac12 & \frac12e^{i\pi/4}\\ \frac12 e^{i\pi/4} & \frac12 \end{pmatrix}\text{ and }Y=X^T.$$
Define $\overline{X}=I_2-X$ and $\overline{Y}=I_2-Y$.
Then $X=1X+0\overline{X}$ and $Y=1Y+0\overline{Y}$ are spectral decompositions of $X$ and $Y$.
Define states by:
$$S_1:=\begin{pmatrix}\frac12 & \frac{1}{20} \\ \frac{1}{20} & \frac12\end{pmatrix} \text{ and }S_2:=\begin{pmatrix}\frac12 & \frac{1}{10}+\frac{1}{20}i \\ \frac{1}{10}-\frac{1}{20}i & \frac{1}{2}\end{pmatrix}.$$
I know these are different states because where $Z=\begin{bmatrix}\frac12 & \frac12 \\ \frac12 &\frac12\end{bmatrix}$, and similarly to above $Z=1Z+0\overline{Z}$: I find:
$$\mathbb{P}[Z=1\,|\,S_1]=\operatorname{tr}(ZS_1)=\frac{11}{20}\neq \frac35=\operatorname{tr}(ZS_2)=\mathbb{P}[Z=1\,|\,S_2].$$
Let us collect some probabilities with these states. These "$\succ$" sequential measurements:
$$\mathbb{P}[X=1\,|\,S_1]=\frac{1}{2}+\frac{\sqrt{2}}{40}=\mathbb{P}[X=1\,|\,S_2]$$
$$\mathbb{P}[[Y=1]\succ [X=1]\,|\,S_1]=\frac{1}{2}=\mathbb{P}[[Y=1]\succ [X=1]\,|\,S_2]$$
$$\mathbb{P}[[Y=1]\succ [X=0]\,|\,S_1]=\frac{1}{2}=\mathbb{P}[[Y=1]\succ [X=0]\,|\,S_2]$$
The other probabilities are given by total probability one.
Now,
$$\mathbb{P}[Y=1\,|\,S_1]=\frac{1}{2}+\frac{3\sqrt{2}}{40}\neq \frac{1}{2}+\frac{\sqrt{2}}{40}=\mathbb{P}[Y=1\,|\,S_2].$$

The things you can know a priori are the distributions of sequential measurements, so you can calculate for projections $p_i\in B(\mathsf{H})$, outcomes $\theta_i\in\{0,1\}$, defining for notation $p_i^{0}=I-p_i$ and $p_i^1=p_i$ e.g.
$$\mathbb{P}[[p_m=\theta_i]\succ \cdots \succ [p_1=\theta_1]\,|\,S]=\operatorname{tr}(p_1^{\theta_1}p_2^{\theta_2}\cdots p_m^{\theta_m}\cdots p_2^{\theta_2}p_1^{\theta_1}S).$$
