Should there be a specified standard knowledge of mathematicians? (Feel free to close this if it is too vague/chatty/soft/etc, I won't be offended!)
Some very quick background. I am a visitor this year at some university, and they have very kindly organized a number of events for all "temporary" people (so postdocs and visitors such as myself mainly). Some of these events are social, but I want to talk about the "Colloquium" which we run (mostly between us though some extra people show up). I want to talk about the difficulty we have to understand each other.
Apart from being the "temporary" people, on a mathematical level we have little in common. I am deeply frustrated by how little we are able to share in the colloquium.
The first talk was about number theory, modular forms in particular. The first sentence was "as you know the Galois group is profinite, that is, compact and totally disconnected". The PDE people in the audience rolled their eyes, as you can imagine, and pretty much stopped listening after this one sentence. 
We talked about this, and the next speaker decided he'd keep things very basic. He gave a talk on PDEs, and I was only able to follow about 10 minutes. Which is terrible. 
When it was my turn, I tried to do something about the quadratic reciprocity law (in fact based on a MO question!). It's not for me to say how it went, but I remember being in shock at some point: I was saying "this set $G$ is in fact a group, it has $n$ elements, so if $g \in G$ we have $g^n = 1$". Somebody (a respectable specialist in probability theory) asked "why is that?" Having learned from this experience, I know that it's worth saying that it's called Lagrange theorem, and believe me, whoever you are, you've learned it within the first two years of university. It blew my mind to find out that some mathematicans have forgotten this one -- but of course the things I have myself forgotten must be equally fundamental to others.
Hence my question:

Would it be useful to write down a document specifying what ALL working mathematicians can be expected to know? People giving a colloquium talk (as opposed to a seminar talk) could be required to adjust their presentation so that it is understandable by anyone knowing what's in that document.

I'm thinking of something similar to what the Word Wide Web Consortium (W3C) has achieved: a standard, a protocol.
I want to stress a difficult point: it's not only about known results, but also standard habits with notations. Let me give you an example of the things which confused me greatly during the aforementioned PDE talk. There was a map $(x, t) \mapsto f(x, t)$ and at some point the speaker wrote $\hat{f}(t)$. I was unable to decide if he meant (i) fix $t$, take $x\mapsto f(x,t)$, take the Fourier transform of that, call it $\hat{f}(t)$, it is a function; or (ii) fix $x$, consider $t\mapsto f(x, t)$, take the Fourier transform and evaluate it at $t$, call it $\hat{f}(t)$, it is a number; or (iii) something else. 
I was uncannily reassured to find out that next to me, some other specialist in PDE said "that's funny, I would have written $\hat{f}(x)$ for the same thing". But then I was more confused than ever when they agreed that the notation did not matter since the meaning was obvious anyway (!). I'm not more anal than the next person, and I'm certainly no Bourbaki fanatic, but I found the need for clarification. (I tried to ask a question but they thought, again, that I was argueing against the notation, not that I was utterly confused as to its meaning.)
I would be interested in reading your thoughts about this. Of course a subsidiary question is, who would write the document, and what authority would it have?
(anecdotal stories of complete confusion during a talk can also provide comic relief, by the way)
Thanks for reading,
Pierre
EDIT: based on one answer below, I want to add the following: I'm not looking for advice on how to improve my skills in exposition, I think I'm doing fine, thank you... The suggestion I'm making, should it be efficient at all, would be rather to improve the average quality of exposition in talks. And specifically, when a speaker adresses an audience of non-specialists. Of course whenever a speaker truly cares (and I think I do, for example) s/he will be doing fine. The point is to make them care.
EDIT: turned into community wiki.
 A: I really don't think any good or useful could come out of such a standard... 
I sat through a plenary talk in a conference with a Fields medal recipient sitting right beside me. The speaker was very much aware of his (rather imposing!) presence---we were sitting on the front row---and, after the initial minutes one could say that he was talking to the medalist. Now, after a good 30 minutes the said medalist asks me very quietly «do you know what some-concept-or-other is? I think I am being supposed to know about it...» I remember pondering at that moment the fact that I had taught a class recently to undergrads about that and, to be honest, the incident managed to considerably increase my respect for the guy.
A: The ability to give a good talk to a general mathematical audience is not something which is widely distributed, to say the least. At a set of short lectures intended for the whole faculty of an Oxford college recently, one mathematician simply took his research talk on technical PDE estimates and went through the slides twice as fast!  In the other direction, a remarkable story about someone who could do the job properly is ``My lunch with Arnol'd", told about how the existence of the recently discovered Gomboc was suspected.
A few years ago I heard a colleague in Computer Science express his surprise at the familiar fact that two random mathematicians thrown together are virtually always unable to communicate on a serious technical level. He felt at ease with any aspect of his subject, but he had been in it since the start about 25 years before. It was pointed out that smart people have been busily expanding mathematics in essentially modern fashion for upwards of 250 years now. My feeling is that it's rather unrealistic to expect a high level of common understanding in a subject that has been developing in so many different productive streams for so long.
I don't know V. I. Arnol'd's secret for giving great general talks. But a couple of rubrics for avoiding disasters might be suggested:  To encourage empathy with the audience, suggest the speaker should target most of the talk at graduate students, or a well-respected colleague with a different specialty. Encourage mixing in ``short stories,'' examples, or other materials related to the main theme that don't require specialist knowledge to comprehend.  And a general talk has to be at least partly a show --- it should be pleasant to experience.

"I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel."
  — Maya Angelou

A: Many (most? all?) North American graduate programs have some form of qualifying exam (which goes by different names at different institutions) whose goal is to establish a baseline knowledge of the kind that you ask about.  Typical topics are algebra and real and complex analysis.  
However, there is no law (natural or human) saying that a person will remember what they have once been taught, and if one's goal (in a colloquium, or equivalently, any lecture to non-sepcialists) is to be understood, it will not be any use to appeal to some alleged common knowledge specified in a document, or a qualifying exam syllabus, or anywhere else.  
I think a more realistic solution is to develop mechanisms for explaining ideas which appeal to a wide range of audience members, and include many different ways for them to try and understand what you are talking about.    This is never easy, unforutnately, but it can become easier with
practice.  I think one thing to remember is that simple geometric ideas are often easier to communicate than algebraic ones, and are (in my experience) more likely to appeal to a wide range of audience members.    Also, I think one should be especially receptive to the idea that audience members will be confused by your notation, and thus one should adopt simple notation and be prepared to explain it.  (So my sympathies are with you in your attempt to divine the meaning of $\hat{f}(t)$!) 
[Added in response to edit to question:] The best way to improve the average level of talks is, I think, to train graduate students to give good talks.  In general, the mathematical community is sensitive to the quality of talks: e.g. job candidates can fail an interview because of a badly delivered job talk; an important criterion for plenary speakers at AMS events is that they can give good talks; colloquium speakers are typically selected in part on the basis of their ability to give good talks.  So the ability to give good general audience talks is an important professional skill, and we should do our best to teach it (along with all the other important professional skills that we teach to our graduate students).    
A: Dear Pierre: it sounds like you may be a recent graduate (my apologies if you're not). Early in your career and getting to a new place is an exciting opportunity; usually it's the first time when you get to give talks where no-one has any idea of what you're talking about (graduate students usually either give talks to specialist audiences at conferences, or at home where their field should be familiar to some through their advisor).
The lesson I remember drawing from this is that it is virtually impossible to dumb down your talk excessively. And I don't mean that at all in a disparaging or insulting way: it is the opportunity for you to truly rethink what you know, faced with some unexpected questions from sharp people who simply have a very different view of things.
So rather than to try and look for "standard knowledge", I would suggest that you use this opportunity to rework your talks and make them more accessible. This will increase your skill as an expositor, make you popular (in most places, people enjoy an accessible talk, it makes them feel smart); and forcing yourself to reconsider your field through the eyes of a novice can be an opportunity to learn something new about it.
A: Like M. Emerton, Pierre's question makes me reflect on what most departments actually do to enforce common knowledge among (future) working mathematicians: their qualifying exams.  I hope that most departments do have qualifying exams, with set syllabi, and that faculty take those exams seriously to make sure students really do learn enough of the set topics.
In my department, we try to be as explicit as we possibly can about what we require of our graduate students, so for instance you can click here for an official list of qual syllabi, and you don't have to scroll down very far to see that Lagrange's Theorem is right there on the list.  The idea that a "respectable specialist in probability theory" disavowed any knowledge of Lagrange's theorem is disturbing to me, and I'm not willing to write it off so quickly.  (Keep in mind though that people, being people, sometimes ask stupid questions during talks or forget things that on a better day they would know cold -- I have certainly done such things in departmental seminars and colloquia.)  It makes me wonder whether this specialist really went to a graduate program which took this stuff seriously...and not, say, Cornell University, according to Thierry's distressing comment.  (Seriously, for a very good department they have what looks to be a very bad approach to educating their graduate students.)
About what you can forget in other fields of mathematics, however basic...this is an interesting question.  From my interactions with research mathematicians I have encountered quite a broad spectrum in terms of how much knowledge / awareness they have of fields other than their own.  It's a fun thought experiment (read: don't do it!) to contemplate what would happen if you gave the graduate student quals to the faculty members instead.  I myself am relatively fortunate to work in a "composite" field like arithmetic geometry which necessarily draws on multiple basic areas: there is of course no arithmetic geometry qualifying exam, but my knowledge in this subject makes the algebra and topology exams look pretty trivial to me.  I would do better on the complex analysis exam if it covered fancier material -- sheaves and manifolds rather than, say, Rouche's Theorem -- but I could still pass it.  The real analysis exam looks hard to me: I don't like my chances without studying for it.  But my proclivities in mathematical knowledge run more to the "broad and shallow" -- as those who have seen me answer questions on this site will know -- with the pleasant side effect that I can often plop myself down in "someone else's" seminar without getting completely lost...or rather no more lost than I would be if a visiting number theorist started filling up the board with Iwasawa theory.
One thing to keep in mind is that the colloquium is the primary opportunity for many grown up mathematicians to get exposed to any ideas outside of their narrow field.  That's why it's so important to have departmental colloquia (and so frustrating when the talks are bad enough to discourage people rather than drawing them in).  If there wasn't a colloquium in your department before and there is now, then if you can keep the probabilists coming, they'll probably remember Lagrange's Theorem eventually.  
Finally, there is so much art and skill in giving a good colloquium talk -- and this skill cuts transversally across other kinds of mathematical ability.  I can see how the number theorist's first sentence put people off, but that could have been as much of a mistake of delivery as content.  If s/he had simply said "As you may know..." people would probably not have rolled their eyes.  Then, depending upon how important it was to the rest of the talk, the speaker could give some skillful, but brief "reminders" about the structure of the Galois group as an inverse limit of finite discrete groups.  But only if that information turns out to be really important.  If not, it would be better to start with something else and say at the appropriate time, in passing: "Now the group of all field automorphisms of the algebraic numbers -- which number theorists call the absolute Galois group of $\mathbb{Q}$ -- is a big, uncountably infinite group.  However, it carries a natural topology with respect to which it is compact and totally disconnected, and this topology is important in the study of...."
A: Maybe the opposite approach is the solution. 
Instead of stating the required knowledge, use the average score for standardized exams in relevant courses/admissions exams as a starting point. Look at the exams in detail. Try and find out (if available) what types of problems most people get right. Then compile the content of those questions into a reference for yourself to use when preparing lectures for that particular group. 
(I don't know if this information is available.)
Stating a required body of information for mathematicians to know is rather pointless, since the term mathematician is much too broad. If you wished to define a certain type of mathematician, who is proficient in X, Y, and Z, then that would make a lot more sense. 
