Assume $\zeta_i=0$, Equations 1-6 in this note give 6 different sufficient conditions (bound on step size) for $w^*$ to be a stable fixed point. Equation 7 give restriction which is both necessary and sufficient.
I haven't seen anyone else derive necessary + sufficient condition on step size for convergence for realizable Gaussian linear least squares, please correct me if it occurs in literature..
I suspect that additive noise does not affect these conditions. IE, if process converges to $w^*$ in a noise-free case, then same step size guarantees convergence to stationary distribution with additive noise.