Skip to main content
edited title
Link
Theo Buehler
  • 5.7k
  • 1
  • 36
  • 30

convergence in distriubtiondistribution of stochastic gradient descent.

Source Link

convergence in distriubtion of stochastic gradient descent.

The stochastic gradient descent algorithm where only a noisy gradient (zero mean noise) is used to update current estimate is known to converge almost surely to the minimizer. However, if one is interested only in convergence in distribution (I understand this requirement is a weaker notion) and NOT almost sure convergence, how should the step sizes chosen so that only distributional convergence and not a.s is guaranteed?