I have read (but I cannot now find where) that V. I. Arnold & J.-P. Serre had a public debate on the value of Bourbaki. Does anyone have more details, or remember or know what was said?
I was there. Arnol'd is one of my big mathematical heros, but I found the whole thing really sad. It was in French, but my French is decent. Arnol'd began his part with a phrase I've heard him say before: “In Russia it is impolite to talk ill of the dead, so I will not talk about Bourbaki” and then he proceeded to lambast Bourbaki in as nasty a way as you've ever read in any of his writings. I just felt like hanging my head. It was embarrassing watching him insult French Mathematics in front of 500 French men and saying things that seemed silly. It went on from there, Serre with a kind of sad understated dignity, not fighting, Arnol'd wanting a fight, hurling insults. The two barely even addressed each other. And yes, he did mention Toth, and if memory serves, he stated that Toth was probably Thales and had most likely come up with Newton's inverse square law. For me, the whole event was sad, embarrassing, and myth-crushing. Well, us mathematicians, we are all humans.
Arnold's account of this debate (in Russian) is here: http://www.mccme.ru/edu/index.php?ikey=viarn_burbaki&post=25742517_1256 and here http://vivovoco.rsl.ru/VV/PAPERS/NATURE/BURBAKI.HTM
Let me translate few words from the beginning.
Serre challenged me with this motivation: "I wanted to tell about the influence of Bourbaki on mathematics. If everyone will speak the same, especially panegyrics, this will be boring. Thus I started to search who would express an opposite opinion to mine. After having leafed through the list of mathematicians of the world, I understood that this must be you."
The duel happened on 13 of March 2001 in Institute Poincare.
Then he describes Serre's speach and his own. According to Arnold's account, his talk in the debates is published as "Mathematics and Physics: a parent and a child, or sisters?" English translation: Physics-Uspekhi, 1999, 42, 12 1205-1217. Here is a free version: http://ufn.ru/en/articles/1999/12/c/
However, comparison of dates shows that this cannot be the true text of Arnold's talk in the debates. I re-read the account of the duel (cited in the beginning) and on my opinion 90% of what Arnold says there (about Serre and about other things) is just not true. Unlike this account, the paper in Physics-Uspekhi looks reasonable, when Arnold writes about math.
My impression is that I also read Serre's talk somewhere. But I cannot find any trace of it in the Internet. So all we have this biased (to say the least) account by Arnold.
It appears that there was an event on Bourbaki, for a general audience, at the Institut Henri Poincaré (Paris) on March 13th, 2001, with Arnold and Serre.
The announcement says:
J-P. Serre : L'apport de Bourbaki, V. Arnold : Mathématiques et Physique. Arnold et Serre sont deux des plus grands noms de l'histoire des mathématiques. Ils ont des conceptions "épistémologiques" très différentes. Leur confrontation, exceptionnelle en public, devrait intéresser tous ceux qu'intéressent une réflexion sur les Sciences.
Translating roughly to: Serre: The contribution of Bourbaki, Arnold: Mathematics and Physics. Arnold and Serre are two of the greatest names in the history of mathematics. They have very different 'epistemological' conceptions. Their confrontation, exceptional in public, should be ineteresting for everyone interested in reflections on the sciences.
Unfortunately I do not have more information about this event at the moment. However, since the existence of an actual event was also unclear it seems to be a partial answer.
Also, Pierre Colmez mentions a debate (put under quotation marks) between Arnold and Serre on Bourbaki, recalling how the latter began his contribution. I cannot be sure if this is this event he is talking about, but it would fit very well as it was not really debate but two talks yet still somehow set up one against the other, so a debate in quotation marks.
Regarding things that were said, it is recalled (by Colmez) that Arnold began with a list of bad deeds of the "Bourbaki criminals" [criminels bourbakistes], including the inclusion of $0$ among the natural numbers, something about the meaning of A implies B [that I do not understand/cannot translate], and that children cannot calculate anymore on account of Bourbaki -- recalling an anecdote about a child who in reply to the question "what is $2+5$?", said it is $5+2$ since addition is commutative [this anecdote was the starting point there, although it appears that while this apparently happened, it was actually a joke/prank of the child, whose parents were mathematicians and who is today also a mathematician].
In retrospect it does not seem that Arnold's 2+5=5+2 comment was that effective particularly in view of the information provided by Joël. On the other hand, there is one aspect of this matter that did not come out sufficiently in the earlier answers. Namely, Arnold is a fabulous pedagogue. To give a quick example, his book Mathematical methods of classical mechanics has benefited thousands, if not millions, of readers. Many people share his doubts about the effectiveness of the Bourbaki method. Verbal excesses of course cannot be justified but there is a genuine issue there nonetheless.