Preface: TLDR: I'm trying to use the martingale CLT but I'm having trouble finding the right martingale for my situation. What is it and is a martingale approach even correct?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Because of orthogonality the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$ My goal is to show that the following is asymptotically normal as $n \to \infty$, assuming that $k(n) \to \infty$ with $n$, i.e.: $$ \frac{T_{n,k(n)}}{\textrm{Var}(T_{n,k(n)})^{1/2}} \to N(0,1) \textrm{ in distribution as } n \to \infty$$

I immediately reached for the martingale CLT, but I'm stuck on constructing a martingale array out of $T_{n,j}$ and $\textrm{Var}(T_{n,j})$.

Letting $\mathcal{F}_n$ be the sigma-algebra generated by all the data observed up to the $n$th point and $\mathcal{F}_{n,i}$ be the sigma-algebra generated by only the data up to and including all of slice $i$, one issue is that $\textrm{Var}(T_{n,i})$ is only $\mathcal{F}_{n,i}$-measurable so I can't build a martingale array in the typical way like: $$ M_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,k(n)}) \right)^{-1/2} $$ Since this would not be adapted to the filtration $\{\mathcal{F}_{n,i}: 1 \leq i \leq k(n)\}$. Instead, I tried to shoehorn something like $$ \tilde{M}_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{-1/2} $$ but this isn't a martingale as written because $$ E(\tilde{M}_{n,i} | \mathcal{F}_{n,i-1}) = T_{n,i-1} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{-1/2} = T_{n,i-1} \cdot \left( \sum_{j=1}^i \textrm{Var}(E_{n,j}) \right)^{-1/2} $$ Which is off by just a small yet annoying factor. Is there some easy way around this situation to get the martingale I need? One idea I had was to define the martingale I need (i.e. without the variance factor), show the CLT for that, and then show that $\tilde{M}_{n,i}$ is not far from the martingale in the limit. Does that even make sense?

EDIT: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

TLDR: I'm a statistician (bear with me!) trying to use the martingale CLT but I only can estimate the conditional variance instead of the unconditional one. Can I do anything to get a CLT with norming by the conditional variance?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Fortunately because of orthogonality of the $E_{n,i}'s$ the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$ and from here it is straightforward enough to show that $T_{n,k(n)} / \textrm{Var}(T_{n,k(n)})^{1/2}$ has a limiting normal distribution.

My question: Unfortunately, in practice it's way easier to compute the conditional variances $E(E_{n,i}^2 | \mathcal{F}_{n,i-1})$ than the unconditional ones $\textrm{Var}(E_{n,i})$. Is it still possible to obtain a CLT using a norming by this conditional variance instead? For example:

$$ \frac{T_{n,k(n)}}{\sum_{i=1}^{k(n)} E(E_{n,i}^2 | \mathcal{F}_{n,i-1})^{1/2}} \to N(0,1) \textrm{ in distribution as } n \to \infty$$

Is this workable at all (maybe with mild assumptions) or is it totally missing an important point about how the CLT works?

UPDATE 1: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

UPDATE 2: I realized that my initial problem about constructing a martingale really boiled down to the question of the conditional variance vs. the unconditional one. Restructured the question to make it more concrete.

Preface: TLDR: I'm trying to use the martingale CLT but I'm having trouble finding the right martingale for my situation. What is it and is a martingale approach even correct?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

I want to show that the following weighted quantity is asymptotically normal: $$ T_{n,j} = \sum_{i=1}^{j} \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta) $$ To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Because of orthogonality the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$

Assuming that $k(n) \to \infty$ with $n$, I want to show a CLT for $T_{n,k(n)}$ as $n \to \infty$. I immediately reached for the martingale CLT, but I'm stuck on constructing a martingale array out of $T_{n,j}$ and $\textrm{Var}(T_{n,j})$.

Letting $\mathcal{F}_n$ be the sigma-algebra generated by all the data observed up to the $n$th point and $\mathcal{F}_{n,i}$ be the sigma-algebra generated by only the data up to and including all of slice $i$, one issue is that $\textrm{Var}(T_{n,i})$ is only $\mathcal{F}_{n,i}$-measurable so I can't build a martingale array in the typical way like: $$ M_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,k(n)}) \right)^{1/2} $$ Since this would not be adapted to the filtration $\{\mathcal{F}_{n,i}: 1 \leq i \leq k(n)\}$. Instead, I tried to shoehorn something like $$ \tilde{M}_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} $$ but this isn't a martingale as written because $$ E(\tilde{M}_{n,i} | \mathcal{F}_{n,i-1}) = T_{n,i-1} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} = T_{n,i-1} \cdot \left( \sum_{j=1}^i \textrm{Var}(E_{n,j}) \right)^{1/2} $$ Which is off by just a small yet annoying factor. Is there some easy way around this situation to get the martingale I need? One idea I had was to define the martingale I need (i.e. without the variance factor), show the CLT for that, and then show that $\tilde{M}_{n,i}$ is not far from the martingale in the limit. Does that even make sense?

EDIT: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

UPDATE: I have been pointed towards Theorem 3.4 in Hall and Heyde, and on the surface it looks promising but I have yet to do the calculations. However, I'm also interested in rates of convergence of this CLT and I'm wary of losing all the relevant results for the martingale CLT such as Haeusler (1988).

Preface: TLDR: I'm trying to use the martingale CLT but I'm having trouble finding the right martingale for my situation. What is it and is a martingale approach even correct?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Because of orthogonality the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$ My goal is to show that the following is asymptotically normal as $n \to \infty$, assuming that $k(n) \to \infty$ with $n$, i.e.: $$ \frac{T_{n,k(n)}}{\textrm{Var}(T_{n,k(n)})^{1/2}} \to N(0,1) \textrm{ in distribution as } n \to \infty$$

I immediately reached for the martingale CLT, but I'm stuck on constructing a martingale array out of $T_{n,j}$ and $\textrm{Var}(T_{n,j})$.

Letting $\mathcal{F}_n$ be the sigma-algebra generated by all the data observed up to the $n$th point and $\mathcal{F}_{n,i}$ be the sigma-algebra generated by only the data up to and including all of slice $i$, one issue is that $\textrm{Var}(T_{n,i})$ is only $\mathcal{F}_{n,i}$-measurable so I can't build a martingale array in the typical way like: $$ M_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,k(n)}) \right)^{-1/2} $$ Since this would not be adapted to the filtration $\{\mathcal{F}_{n,i}: 1 \leq i \leq k(n)\}$. Instead, I tried to shoehorn something like $$ \tilde{M}_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{-1/2} $$ but this isn't a martingale as written because $$ E(\tilde{M}_{n,i} | \mathcal{F}_{n,i-1}) = T_{n,i-1} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{-1/2} = T_{n,i-1} \cdot \left( \sum_{j=1}^i \textrm{Var}(E_{n,j}) \right)^{-1/2} $$ Which is off by just a small yet annoying factor. Is there some easy way around this situation to get the martingale I need? One idea I had was to define the martingale I need (i.e. without the variance factor), show the CLT for that, and then show that $\tilde{M}_{n,i}$ is not far from the martingale in the limit. Does that even make sense?

EDIT: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

Preface: TLDR: I'm trying to use the martingale CLT but I'm having trouble finding the right martingale for my situation. What is it and is a martingale approach even correct?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

I want to show that the following weighted quantity is asymptotically normal: $$ T_{n,j} = \sum_{i=1}^{j} \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta) $$ To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Because of orthogonality the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$

Assuming that $k(n) \to \infty$ with $n$, I want to show a CLT for $T_{n,k(n)}$ as $n \to \infty$. I immediately reached for the martingale CLT, but I'm stuck on constructing a martingale array out of $T_{n,j}$ and $\textrm{Var}(T_{n,j})$.

Letting $\mathcal{F}_n$ be the sigma-algebra generated by all the data observed up to the $n$th point and $\mathcal{F}_{n,i}$ be the sigma-algebra generated by only the data up to and including all of slice $i$, one issue is that $\textrm{Var}(T_{n,i})$ is only $\mathcal{F}_{n,i}$-measurable so I can't build a martingale array in the typical way like: $$ M_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,k(n)}) \right)^{1/2} $$ Since this would not be adapted to the filtration $\{\mathcal{F}_{n,i}: 1 \leq i \leq k(n)\}$. Instead, I tried to shoehorn something like $$ \tilde{M}_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} $$ but this isn't a martingale as written because $$ E(\tilde{M}_{n,i} | \mathcal{F}_{n,i-1}) = T_{n,i-1} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} = T_{n,i-1} \cdot \left( \sum_{j=1}^i \textrm{Var}(E_{n,j}) \right)^{1/2} $$ Which is off by just a small yet annoying factor. Is there some easy way around this situation to get the martingale I need? One idea I had was to define the martingale I need (i.e. without the variance factor), show the CLT for that, and then show that $\tilde{M}_{n,i}$ is not far from the martingale in the limit. Does that even make sense?

EDIT: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

Preface: TLDR: I'm trying to use the martingale CLT but I'm having trouble finding the right martingale for my situation. What is it and is a martingale approach even correct?

Suppose I have two iid data streams: $X = (X_1, X_2, \ldots)$ and $Y = (Y_1, Y_2, \ldots)$ independent of each other with $E X_1^2 < \infty$ and $E Y_1^2 < \infty$. I'm trying to estimate $\theta = E X_1 - E Y_1$ by sampling $X$ and $Y$ into sample means but with a wrinkle:

The entire timeline is divided up into slices, so that at the time when there are $n$ total points there are $k(n)$ slices, where in slice $i$ we have a difference-in-sample-means estimate $\hat{\theta}_i = \bar{X}_i - \bar{Y}_i$ calculated only from data within that slice. Also suppose that at time $n$, the number of points within slice $i$ (denoted $n_i$), is non-zero, known and not random for each $1 \leq i \leq k(n)$.

Assume there is dependence between the within-slice estimates $\{\hat{\theta}_i: 1 \leq i \leq n\}$ in the following way:

• For each $i$, the probability $p_i$ that each of the $n_i$ total within-slice visitors will be sampled from stream $X$ (instead of $Y$) is a function of $\hat{\theta}_{i-1}$. Assume that $p_i \in (0,1)$ so that the $\hat{\theta}_i$'s are orthogonal.

I want to show that the following weighted quantity is asymptotically normal: $$ T_{n,j} = \sum_{i=1}^{j} \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta) $$ To ease notation let $E_{n,i} := \frac{n_i}{n} (\bar{X}_i - \bar{Y}_i - \theta)$ so that we can write simply $T_{n,j} = \sum_{i=1}^j E_{n,i}$ with each $E_{n,i}$ having zero mean. Because of orthogonality the variance of $T_{n,i}$ is just $$ \textrm{Var}(T_{n,j}) = \sum_{i=1}^j \textrm{Var}(E_{n,i}) $$

Assuming that $k(n) \to \infty$ with $n$, I want to show a CLT for $T_{n,k(n)}$ as $n \to \infty$. I immediately reached for the martingale CLT, but I'm stuck on constructing a martingale array out of $T_{n,j}$ and $\textrm{Var}(T_{n,j})$.

Letting $\mathcal{F}_n$ be the sigma-algebra generated by all the data observed up to the $n$th point and $\mathcal{F}_{n,i}$ be the sigma-algebra generated by only the data up to and including all of slice $i$, one issue is that $\textrm{Var}(T_{n,i})$ is only $\mathcal{F}_{n,i}$-measurable so I can't build a martingale array in the typical way like: $$ M_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,k(n)}) \right)^{1/2} $$ Since this would not be adapted to the filtration $\{\mathcal{F}_{n,i}: 1 \leq i \leq k(n)\}$. Instead, I tried to shoehorn something like $$ \tilde{M}_{n,i} = T_{n,i} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} $$ but this isn't a martingale as written because $$ E(\tilde{M}_{n,i} | \mathcal{F}_{n,i-1}) = T_{n,i-1} \cdot \left( \textrm{Var}(T_{n,i)}) \right)^{1/2} = T_{n,i-1} \cdot \left( \sum_{j=1}^i \textrm{Var}(E_{n,j}) \right)^{1/2} $$ Which is off by just a small yet annoying factor. Is there some easy way around this situation to get the martingale I need? One idea I had was to define the martingale I need (i.e. without the variance factor), show the CLT for that, and then show that $\tilde{M}_{n,i}$ is not far from the martingale in the limit. Does that even make sense?

EDIT: I am making no assumptions on the manner of dependence of $p_i$ on $\hat{\theta}_{i-1}$ aside that $p_i$ is bounded away from 0 and 1. If it turns out that this is intractable then I'd love to investigate what conditions must be imposed to make this work!

UPDATE: I have been pointed towards Theorem 3.4 in Hall and Heyde, and on the surface it looks promising but I have yet to do the calculations. However, I'm also interested in rates of convergence of this CLT and I'm wary of losing all the relevant results for the martingale CLT such as Haeusler (1988).