There is a rigorous way to deal with such series analytically, within the framework of distribution theory, more precisely of their representation as boundary values of anslytic functions. The theory was initiated by Gottfried Köthe in a paper entitled "Distributionen als Randwerte von analytischen Funktionen" which can easily be found online. (Translation of the title: Distributions as boundary values of analytic functions. The author shows how distributions on the unit disc---essentially, periodic ones---can be regarded as (generalised) boundary values of functions which are analytic on its complement in the complex plane and, in fact, the example in the question is just such a representation for the Dirac delta function). I would guess that this was already known to und used by mathematical physicists (on a non-rigorous basis).

There is a rigorous way to deal with such series analytically, within the framework of distribution theory, more precisely of their representation as boundary values of anslytic functions. The theory was initiated by Gottfried Köthe in a paper entitled "Die Randverteiling analytischer Funktionen" which can easily be found online (Math. Zeitschr. 1952. (Translation of the title: The boundary values of analytic functions). The author shows how distributions on the unit disc---essentially, periodic ones---can be regarded as (generalised) boundary values of functions which are analytic on its complement in the complex plane. In fact, the example in the question is just such a representation for the Dirac delta function. I would guess that this was already known to und used by mathematical physicists (on a non-rigorous basis). If I recall correctly, this material is expounded in his classical monograph and so can be found in an english version.