Whenever we can interchange summation (perhaps due to Tonelli-Fubini), good things happen. Otherwise, one has to struggle evaluating double sums in just one way, because the alternative results in a divergent series. Having said that, I'm currently interested in the following:
Have you encountered in your own research or do you recall from someone's research paper "interesting" double sums that are convergent where reversing order of summation does not work? Please provide references.
Since there's a "number theory" tag, I suggest the quasimodular form $E_2(\tau)$, defined for $\tau$ in the upper half-plane as a multiple of $\sum_{m\in\bf Z} {\sum_{n\in\bf Z}}' (m\tau+n)^{-2}$ where the $\prime$ indicates omission of the term $(m,n)=(0,0)$. For even $k>2$, the corresponding sum $\sum_{m\in\bf Z} {\sum_{n\in\bf Z}}' (m\tau+n)^{-k}$ converges absolutely and yields modularity of $E_k$. But for $k=2$, switching the sums yields $\tau^{-2} E_2(1/\tau)$, which is not the same thing as $E_2(\tau)$! (But you can still recover the formula for the difference by carefully keeping track of how switching $\sum_m$ and $\sum_n$ changes the sum).
Alice and Bob play a game in which each flips a fair coin repeatedly until it turns up tails, earning a score equal to the number of times it turns up heads. (Thus if Alice flips $HHHT$, her score is $3$.) The high scorer wins, and collects a prize of $4^n$ from the loser, where $n$ is the loser's score.
Let $P(a,b)$ be Alice's payoff when she and Bob earn scores of $a$ and $b$. Then Alice's expected payoff is $$\sum_{b=0}^\infty{1\over 2^{b+1}}\sum_{a=0}^\infty {1\over 2^{a+1}}P(a,b)=1/2>0$$ so that Alice, if she's an expected-value maximizer, should certainly agree to play this game.
On the other hand, her expected payoff is also $$\sum_{a=0}^\infty{1\over 2^{a+1}}\sum_{b=0}^\infty {1\over 2^{b+1}}P(a,b)=-1/2<0$$ so she should certainly refuse to play.
Some further words on the matter (including a reference to a MathOverflow post!) can be found here.
At L. Spice's suggestion, I hereby reify (!) my comment "Fourier inversion on the real line? Too mundane?" :)
But while we're here, I'd want to remark that to my mind much modern analysis (e.g., post-Schwartz, post-Grothendieck, et al) amounts to reconsideration of seemingly illegitimate symbol manipulation that mysteriously gives reliable answers (e.g., in Heaviside's work in the 1890's and thereafter, in Dirac's physics beginning c. 1928, H. Bethe's and others' work in the early 1930s).
In that context, one might argue that the ground-breaking work of the Italian, German, and French analysts in the late 19th and early 20th century was wonderful, but did not go far enough. In particular, although (as I've ranted elsewhere or other occasions) the precisification of the notion of "function" in set-theoretic terms was an excellent thing, it was/is arguably too restrictive (witnessed already by the "almost everywhere" kludge/revelation in Lebesgue-et-al theory).
In particular, although Fubini-Tonelli's theorem can fail due to "pointwise" problems, if sufficiently recast about not-literal integrals but continuous functionals, it can be designed-to-succeed in many situations where pointwise failure is irrelevant. Thus, in many practical situations, a "literal failure"'s incorrect conclusions can sometimes be salvaged by not being sucked down into the miasma of (irrelevant) pointwise issues.
To my mind, an archetype for this is Clairault's theorem on the interchangeability of partial derivatives ... under some conditions. Naturally, at the time, and still nowadays for many, this is a pointwise issue. However, taking Fourier transforms, we find that distributionally the derivatives are always interchangeable.
Back to the immediate question: another number-theoretic example (which I included as a prank-question in my old book on Hilbert modular forms) is about the "inner product" (which it cannot literally be...) of a holomorphic Poincare series and a holomorphic Eisenstein series. The seeming paradox is that if we unwind the Eisenstein series, we (correctly) compute the zeroth Fourier coef of the Poincare series, which is (in the holomorphic case only!) $0$. But, if we unwind the Poincare series (which is not ok... ) we seem to compute a higher Fourier coefficient of the Eisenstein series, thus, seeming to show that Eisenstein series are constants. The fallacy is that the Poincare series cannot be unwound here because there was cancellation in the summing of it: when unwound, Fubini-Tonelli does not apply, and, indeed, the seeming conclusion is incorrect.
The non-frivolousness of that example resides in a standard procedure in analytic number theory (and in automorphic extensions of the classical versions thereof) where both Eisenstein series and Poincare series are extended by a sort of Hecke-summation device to depend on a complex parameter $s$, and wherein a fairly arbitrary thing is "wound up" to make an automorphic form, and then spectrally decomposed, etc. It is common practice to simply ignore convergence issues in such spectral expansions, with a few exceptions. And, indeed, even in the tiniest cases, $L^2$ convergence is some distance from pointwise, but the discussion is all too often pointwise.
Another archetype is the spectral decomposition of pseudo-Eisenstein series (in various contexts) in terms of genuine Eisenstein series. This decomposition in the simplest cases is derived from ordinary Fourier inversion applied to test-function data for a pseudo-Eisenstein series, with the idea to wind up this expression to an integral of Eisenstein series. The obstacle is convergence. Thus, the Fourier-Mellin inversion path is moved sufficiently far to the right to legitimize the winding up (by Fubini-Tonelli, for example), and then the path is moved back to the critical line. The small surprise, which may seem innocuous in the simplest cases, is that residues of the Eis appear. For $SL(2,\mathbb Z)$, the residues are essentially constants, which may be misleading. Namely, for higher-rank groups (e.g., $SL(4)$ or $Sp(4)$, the residues are highly-nontrivial automorphic forms (Speh forms).
(The previous are arguably even-simpler issues than difficulties in evaluating traces in the obvious fashion.)
Would you accept a trace-integral as a double sum? The distribution character of a representation $\pi$ (of a, let's say semisimple, $p$-adic group $G$, although I think that most of what I say here is also true for real Lie groups) is given by $$ \operatorname{tr} \pi(f) = \operatorname{tr} \int_G \pi(g)f(g)\mathrm dg. $$ One would very much like to say, as in the case of $G$ compact, that $$ \operatorname{tr} \pi(f) = \int_G (\operatorname{tr} \pi(g))f(g)\mathrm dg. $$ This is not possible for $\pi$ infinite-dimensional unitary, but it is a very deep result of Harish-Chandra that there is a locally integrable function $\Theta_\pi$ with various nice properties (for example, it is locally constant on the regular semisimple locus) so that $$ \operatorname{tr} \pi(f) = \int_G \Theta_\pi(g)f(g)\mathrm dg. $$ The computation of this function $\Theta_\pi$ is, due to its indirect definition, quite difficult. My own research is largely concerned with the case where $\pi$ is an irreducibly compactly induced representation $\pi = \operatorname{Ind}_K^G \rho$, in which case more work of Harish-Chandra shows that $$ \Theta_\pi(\gamma) = \int_G \int_K \dot\theta_\rho(g k\gamma k^{-1}g^{-1})\mathrm dk\,\mathrm dg. $$ (The inner integral over a compact open subgroup $K$ also comes, if one digs into the details of the proof, from the failure of a double integral to be ‘switchable’.)
I believe that the (non-compact case of) the trace formula can also be viewed as a lengthy and difficult matter of dealing with the failure of Fubini, but I couldn't talk in any intelligible fashion about this (and could well be wrong). Perhaps, if I'm not completely off, someone else could discuss it.