Skip to main content
fixed numerous typos
Source Link
YCor
  • 63.9k
  • 5
  • 187
  • 286

I know that for two i.i.d distributiondistributions $P$ and $Q$ the probability that $Q$ will produce a length $n$ sequence that is $\epsilon$-typicltypical according to $P$ is bounded by $$Q(T_{P,\epsilon})\leq e^{-nD(P||Q)-n|\mathcal{X}|\epsilon}$$ where $D(P||Q)$ is the KL-dvergence and $T_{P,\epsilon}$ is the (strong) $\epsilon$-typical set of the distribution $P$, i.e. sequences with empirical distribution that is $\epsilon$ close-close to $P$.

Will this bound hold for $Q$ that inis not i.i.d? For examplaexample, a markovMarkov source? That is, will I have $$Q^n(T_{P^n,\epsilon})\leq e^{-D(P^n||Q^n)-n|\mathcal{X}|\epsilon}$$ Wherewhere $Q^n$ is the distribution on sequences of length $n$?

I know that for two i.i.d distribution $P$ and $Q$ the probability that $Q$ will produce a length $n$ sequence that is $\epsilon$-typicl according to $P$ is bounded by $$Q(T_{P,\epsilon})\leq e^{-nD(P||Q)-n|\mathcal{X}|\epsilon}$$ where $D(P||Q)$ is the KL-dvergence and $T_{P,\epsilon}$ is the (strong) $\epsilon$-typical set of the distribution $P$, i.e. sequences with empirical distribution that is $\epsilon$ close to $P$.

Will this bound hold for $Q$ that in not i.i.d? For exampla, a markov source? That is, will I have $$Q^n(T_{P^n,\epsilon})\leq e^{-D(P^n||Q^n)-n|\mathcal{X}|\epsilon}$$ Where $Q^n$ is the distribution on sequences of length $n$?

I know that for two i.i.d distributions $P$ and $Q$ the probability that $Q$ will produce a length $n$ sequence that is $\epsilon$-typical according to $P$ is bounded by $$Q(T_{P,\epsilon})\leq e^{-nD(P||Q)-n|\mathcal{X}|\epsilon}$$ where $D(P||Q)$ is the KL-dvergence and $T_{P,\epsilon}$ is the (strong) $\epsilon$-typical set of the distribution $P$, i.e. sequences with empirical distribution that is $\epsilon$-close to $P$.

Will this bound hold for $Q$ that is not i.i.d? For example, a Markov source? That is, will I have $$Q^n(T_{P^n,\epsilon})\leq e^{-D(P^n||Q^n)-n|\mathcal{X}|\epsilon}$$ where $Q^n$ is the distribution on sequences of length $n$?

Source Link
Student88
  • 503
  • 2
  • 7

Joint typicality of sequences

I know that for two i.i.d distribution $P$ and $Q$ the probability that $Q$ will produce a length $n$ sequence that is $\epsilon$-typicl according to $P$ is bounded by $$Q(T_{P,\epsilon})\leq e^{-nD(P||Q)-n|\mathcal{X}|\epsilon}$$ where $D(P||Q)$ is the KL-dvergence and $T_{P,\epsilon}$ is the (strong) $\epsilon$-typical set of the distribution $P$, i.e. sequences with empirical distribution that is $\epsilon$ close to $P$.

Will this bound hold for $Q$ that in not i.i.d? For exampla, a markov source? That is, will I have $$Q^n(T_{P^n,\epsilon})\leq e^{-D(P^n||Q^n)-n|\mathcal{X}|\epsilon}$$ Where $Q^n$ is the distribution on sequences of length $n$?