The answer is negative if we believe [1] (which comes without proofs, unfortunately):

[1] defines the set of separable states as the convex closure of $\{\sum_{i=1}^n\sigma_i\otimes\tau_i\}$. I will call that set $S$. The set as defined in the original question I call $T$. Then $T\subseteq S$, and $T=S$ iff the answer is yes (i.e., iff $T$ is closed).

[1] shows that there is a $\rho\in S$ such that $\rho$ cannot be represented as a Bochner integral $\int\psi\psi^*\otimes\phi\phi^* \pi(d(\psi,\phi))$ for an atomic measure $\pi$. ($\rho$ is not "countably decomposable" in their language.)

Any $\rho\in T$ can be represented as such an integral with a discrete and hence atomic measure $\pi$. Thus $T\subsetneq S$.

On the positive side, if we define $T':=\{\int\psi\psi^*\otimes\phi\phi^* \pi(d(\psi,\phi))\}$ for probability measures $\pi$, then [1] shows $S=T'$. In particular, $T'$ is closed. So, in the spirit of the original question, the set of infinite convex combinations of $\sigma_i\otimes\tau_i$ is closed, only the notion of
"infinite convex combination" must be changed: Not infinite sums, but integrals.

[1] *Werner, R. F.; Kholevo, A. S.; Shirokov, M. E.*, **On the concept of entanglement in Hilbert spaces.**, Russ. Math. Surv. 60, No. 2, 359-360 (2005); translation from Usp. Mat. Nauk 60, No. 2, 153-154 (2005). ZBL1098.47019.