Skip to main content
added 39 characters in body
Source Link
Davide Giraudo
  • 4k
  • 2
  • 30
  • 42

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Consider the case of martingale first, for submartingale in OP, see part III E., Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$$$P(|X_n-X_0|>\alpha n)\leq 2 \exp \left(n\cdot \operatorname{KLdiv}\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) \right)$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$$$P(|X_n-X_0|>\alpha n)\leq 2 \exp \left(\frac{n\alpha^2}{2c^2}\right)$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Consider the case of martingale first, for submartingale in OP, see part III E., Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Consider the case of martingale first, for submartingale in OP, see part III E., Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 \exp \left(n\cdot \operatorname{KLdiv}\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) \right)$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 \exp \left(\frac{n\alpha^2}{2c^2}\right)$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

added 27 characters in body
Source Link
Henry.L
  • 8.1k
  • 8
  • 48
  • 74

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Consider the case of martingale first, for submartingale in OP, see part III E., Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound. For discussion of submartingale in OP, see part III E.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound. For discussion of submartingale in OP, see part III E.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Consider the case of martingale first, for submartingale in OP, see part III E., Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf

Source Link
Henry.L
  • 8.1k
  • 8
  • 48
  • 74

Yes, but with the price of additional assumptions on variances $\sigma^2$ of samples $X_t$([1]'s proof can be extended to non-identical variance $\sigma ^2_i$). For example, [1] Theorem 2 strengthens the result to (28). These results can also be proved using the methods of types as mentioned in [1] part IV. Theorem 2 stated that $$P(|X_n-X_0|>\alpha n)\leq 2 exp (n\cdot KLdiv\left(\frac{\frac{\alpha}{c}+\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}},\frac{\frac{\sigma^{2}}{c^{2}}}{1+\frac{\sigma^{2}}{c^{2}}}\right) )$$ Specifically when $\sigma^2=c^2$, 1 showed this special case $$P(|X_n-X_0|>\alpha n)\leq 2 exp (\frac{n\alpha^2}{2c^2})$$ for some constant $d>0$. You can refine it further using (34). [1] Theorem 3 yields an even more complicated(but looser bound) Lambert-functional bound. For discussion of submartingale in OP, see part III E.

[1]Sason, Igal. "On refined versions of the Azuma-Hoeffding inequality with applications in information theory." arXiv preprint arXiv:1111.1977 (2011). https://arxiv.org/pdf/1111.1977.pdf