Write $\alpha_i$ for the expected time starting from “something then $i$”. Then $$ \alpha_i = \frac1n(\alpha_i+1) + \cdots + \frac1n(\alpha_n+1) + \frac{i-1}n \cdot 1 = 1 + \frac{\alpha_i + \cdots + \alpha_n}n. $$ From this one can deduce $\alpha_i = \left(\frac{n}{n-1}\right)^{n-i+1}$, and $$ E_n = \frac{\alpha_1}n+\cdots+\frac{\alpha_n}n = \left(\frac n{n-1}\right)^n $$ as explained by Kasper Andersen. Of course the limit is $e$.
The way I got the expression for $\alpha_i$ is setting $\beta_i = \alpha_i/n$ and realising that $$ (n-1)(\beta_{i-1} - \beta_i) = \left(\beta_i+\cdots+\beta_n+1\right) - \left(\beta_{i+1}+\cdots+\beta_n+1\right) = \beta_i, $$ from which we deduce $\beta_{i-1} = \frac n{n-1}\beta_i$, then from $\alpha_n = 1+\alpha_n/n$ we get $\beta_n = 1/(n-1)$ and $\beta_i=\frac1n\left(\frac n{n-1}\right)^{n-i+1}$.