Background
A useful trick when trying to analyze a series $\sum_{n=0}^\infty f(n)$ is to expand $f(n)$ as some kind of series, swap the order of summation, and then evaluate the inner infinite sum. However, this trick doesn't always work when the inner sum diverges.
Consider we wish to evaluate the series $$\sum_{n=1}^\infty \frac{1}{n^2+1} = \sum_{n=1}^\infty \sum_{k=0}^\infty (-1)^k n^{2k} = \sum_{k=0}^\infty (-1)^k \sum_{n=1}^\infty n^{2k} = \sum_{k=0}^\infty (-1)^k \zeta(-2k)$$ However, we have obviously failed, since $\zeta(-2k)=0$ for $k>0$. The problem is of course much worse, since for any analytic even function $f(n)$ with $f(0)=0$ we have $$\sum_{n=1}^\infty f(n)= \sum_{n=1}^\infty \sum_{k=1}^\infty a_k n^{2k} = \sum_{k=1}^\infty a_k \zeta(-2k) = \sum_{k=1}^\infty 0=0$$ After becoming aware of examples such as these, I was convinced that there was really no way to save the trick in general when the inner sum diverges. However, recently, I have reconsidered this view, and I am curious whether there is a general theorem that relates the original sum to the series after we apply the trick of expanding the inner sum and switching the order (and, whether this method is equal regardless of our choice of expansion).
The proper way to evaluate $\sum_{n=1}^\infty \frac{1}{n^2+1}$
The missing piece, it seems, between $\sum_{n=1}^\infty \frac{1}{n^2+1}$ and $\sum_{k=0}^\infty (-1)^k \zeta(-2k)$ is the extra residues introduced by $\zeta(-2k)$. $\zeta$ adds a extra residue at $k=-\frac{1}{2}$ and $+\infty$. We can pick up the residue at $-\frac{1}{2}$ easily, but the one at $+\infty$ is a bit more tricky, since we don't want to also pick up any of the residues near $-\infty$. We get this residue mapping by from $f(z)dz \to f(\frac{1}{z})d(\frac{1}{z})$ so that the residue at $\infty$ is at zero, and then integrating the following contour
Adding all these residues together does in fact obtain the correct value of $\frac{1}{2}(\pi\coth(\pi)-1)$. Note that an easier way to sum these residues is to notice that integrating the contour $\frac{1}{2i} \int_{-\frac{3}{4} - i \infty}^{-\frac{3}{4} + i \infty} \csc(\pi z) \zeta(-2z)dz$ also picks up the residues at $-\frac{1}{2}$ and $\infty$, but we could also view the contour as closing in the opposite direction which gives us that $\sum_{n=1}^\infty \frac{1}{n^2+1} = -\sum_{n=1}^\infty (-1)^n \zeta(2n) = -\sum_{n=1}^\infty (-1)^n (\zeta(2n)-1) + -\sum_{n=1}^\infty (-1)^n = \frac{1}{2}(\pi \coth(\pi)-1)$ as long as we allow $\sum_{n=1}^\infty (-1)^n = -\frac{1}{2}$
The same type of behavior happens with other series. For instance, if we look at the similar series $$\sum_{n=1}^\infty x^{-n^2} = \sum_{n=1}^\infty \sum_{k=0}^\infty \frac{(-\ln(x))^k}{k!} n^{2k} = \sum_{k=0}^\infty \frac{(-\ln(x))^k}{k!} \zeta(-2k)$$ we can again sum the series by picking up the residue at $-\frac{1}{2}$ and $+\infty$ or equally by integrating $\frac{1}{2i}\int_{-\frac{3}{4}-i \infty}^{-\frac{3}{4}+i\infty} \csc(\pi z) \frac{(-\ln(x))^z}{z!} \zeta(-2z)dz$
This approach also works for more complicated series. For instance, we formally have that $$\sum_{n=0}^\infty \frac{(-1)^n}{2^n} x^{a^n} = \sum_{n=0}^\infty \sum_{k=0}^\infty (-1)^n \frac{\ln(x)^k a^{nk}}{2^n k!} = \sum_{k=0}^\infty \frac{\ln(x)^k}{k!(1+\frac{1}{2}a^k)}$$ and we have equality when $a<1$, since the inner series is convergent. However, for $a>1$ we have to pick up the residues at $\frac{1}{1+\frac{1}{2} a^k}$, and doing so gives us an easy way to obtain the same results of the behavior of the function found by others in this question: On an example of an eventually oscillating function
Based on the preceding examples, I am curious if something like the following is true.
Consider $\sum_{n} f(n)$ where $f(n)$ is formally equal to $\sum_{k} g(n,k)$ and $h(k)$ formally equals $\sum_{n} g(n,k)$ (perhaps formally should be interpreted in terms of analytical continuation?). Then $\sum_{n} f(n) = \sum_{k} h(k)+\sum {\text{Res}(\frac{h(z)}{e^{2\pi i z}-1},z_k)}$
Some thoughts
One difficulty in understanding this problem is that $g(z)$ often doesn't decay for large $z$, and in some the previous examples it instead grows quickly as $z \to \infty$. This means we can't actually apply the residue theorem since we can't close the contour, and therefore we would need to generalize the residue theorem before we could view the problem in a way related to the Abel-Plana formula. Moreover, in the examples with $\zeta$ function an extra residue at $+\infty$ is created, and this residue doesn't exactly behave like a usual pole.
A second difficulty is that we can't apply the identity theorem in general. Notice that we have $$\sum_{n=0}^\infty (-1)^n \left(\frac{1}{2}\right)^{z^n}-\sum_{k=0}^\infty \frac{\ln(\frac{1}{2})^k}{k!(1+z^k)} = \begin{cases} 0 & |z|<1 \\ \sum \text{Res}(f,a_k) & |z|>1 \\ \end{cases}$$ Thus the difference between the two functions is not analytic on the unit circle, and this type of behavior should happen any time $z$ is sometimes within the radius of convergence and sometimes outside.
Update: It looks like the relationship is more difficult than these examples suggest. In general, we cannot just integrate along some path in the complex plane and end up getting the correct residues. For instance, consider the two different functions $$ f_1(s) = \sum_{n=1}^\infty e^{-n^s} \qquad f_2(s) = \sum_{n=1}^\infty \left(e^{-n^s}-1\right) $$ We have that $\sum_{n=1}^\infty e^{-n^s} = \sum_{k=0}^\infty \frac{(-1)^k}{k!} \zeta(-ks)$, and accordingly we should pick up the residue $ \frac{\csc(\pi z) \zeta(-zs)}{\Gamma(z+1)2i} \big|_{z=\frac{-1}{s}} = \Gamma(1+\frac{1}{s})$ which agrees with the analytical continuation found in this question. However, near $s=+0$ this residue can end up with an arbitrarily large negative real part, so there isn't a single contour that would work to pick up all the residues.
Even more, if we look at $\sum_{n=1}^\infty e^{-n^s}-1 = \sum_{k=1}^\infty \frac{(-1)^k}{k!} \zeta(-ks)$ this has same residue but it isn't picked up. This creates the opposite problem since there is no fixed contour that doesn't pick up that residue for some values of $s$.
Thus, I am not sure, in general, how to determine if certain residues should be picked up in the extension or not
