What's the most harmful heuristic (towards proper mathematics education), you've seen taught/accidentally taught/were taught? When did handwaving inhibit proper learning?

Not the most harmful, but a fun example (credit due to Tony Varilly):
False. You can in the free abelian group generated by an apple and an orange. As Patrick Barrow says, "A failure of imagination is not an insight into necessity." 


I saw that for years, and I never understood it until I saw the real definition of a tensor. 


This isn't really a heuristic, but I hate "functions are formulas." It takes a lot of students a really long time to think of a function as anything other than an algebraic expression, even though natural algorithmic examples are everywhere. For example, some students won't think of f(n) = {1 if n is even, 1 if n is odd} as a function until you write it as f(n) = (1)^n. 


Along the same lines as Qiaochu's and Zach's responses, the commonly taught heuristics pertaining to functions, differentiability and integration are a pet hate of mine. I certainly left school thinking of functions as formulas involving combinations of elementary functions and having a very poor understanding of the relevance and correct relationship between integration and differentiation, the worst manifestation of which, now that I'm a bit older, seems to have been that Differentiation is a nice, computable operation and tells you about functions; integration is hard and tells you about areas under curves. Areas under curves never seemed interesting. As an analyst, my personal feelings towards them are now almost entirely reversed and I think of integration as my friend and differentiation as the enemy. Differentiation uses up regularity; integration smooths. 


"Stacks are schemes with groups attached to points."I don't know how much damage this has caused, but I never understood how it was actually helpful to anybody. Not only is it handwavy (which is okay for a heuristic), but it's handwavy in a way that can't really be corrected (because it's false). My feeling is that people who adopt this heuristic are trapped. If they use the heuristic to come up with a result, it's very hard to sharpen the reasoning to turn it into a proof. You have to just start from scratch and not use the heuristic. 


Twocolumn proofs Usually the only proofs that students see upon graduating from highschool are the geometry "twocolumn" proofs, and trying to convince them that the essence of mathematical proof lies not in the form but in the logical deductive argument takes a lot of convincing. 


Linear algebra purely as row manipulations. I've written about this here:



"Truth is binary. If a theorem has been proven once, there is no need in a second proof." 


The "FOIL" (first+outside+inside+last) mnemonic for multiplying two binomials is terrible. It suppresses what is really going on (three applications of the distributive property) in favor of an algorithm. In other words, it is teaching a human being to behave like a computer. The legacy of FOIL is clear when you ask your students to multiply three binomials, or two trinomials. Students usually either have no idea what to do, attempt it but get lost in the algebra, or succeed but complain about the arduousness of the task. 


"Generalization for the sake of generalization is a waste of time" I think that generalization for the sake of generalization can be rather fruitful. 


"Categories can be specified by objects alone." It's easy to get this impression, because people who are familiar with the categories in question already know the morphism structure, and don't bother to specify it. There is a related heuristic concerning the composition law, but it doesn't seem to burn people as often. 


One extremely harmful heuristic I held until fairly recently: identifying math with algebraic manipulation. When asked to prove an identity or an inequality I would often dive straight into algebraic manipulation of the relations that I knew, wasting many many hours of my time. I have found that it is much more useful to try and test statements against examples I already know, and to try and rephrase identities and inequalities in terms of a statement in natural language that I have some intuition for. 


"Vectors are directed line segments." When worded this way, this utterance is only acceptable if the student is satisfied with getting on his or her bicycle at the end of class and never returning to mathematics again. 


That there is something weird and unsavory about field extensions that are not separable and that serious contemplation of such things should be put off to the indefinite future. (In fact, much of the richness and "pathology" of geometry in characteristic p is easily understood once one has a firm grasp of how field extensions behave.) 


Similar to Tom's answer,
Useful for distinguishing between speed and velocity but little else. The above is a typical definition from a physics textbook I had on the shelf; here in British Columbia, vectors are introduced in high school physics but not high school math. By the time students get to linear algebra in first or secondyear university, it can be hard to convince them that a real number (much less a polynomial) can be a vector. Usually, you have to resort to "a real number does too have a direction: positive or negative" and even then they don't believe you because
and so if real numbers are vectors, how can they be scalars? Don't even ask about function spaces. 


The opposite of Qiaochu's dictum is just as misleading  "formulas are functions". There are a lot of nondenoting expressions! It's just that mathematicians don't tend to write nondenoting terms very often. Of course, there's a good reason for that  you can't prove anything interesting about nondenoting terms (or rather, way too much). But then students never get the intuition that there are expressions which are 'junk', nor tools to prove that something is 'junk'. My favourite 'junk' expression is $$1/\frac{1}{\left( x  x \right) } $$ Lest you think this is not very important, try to "teach" firstyear calculus to a computer, and you'll see how these nondenoting terms are most troublesome. 


"A continuous function is one you can draw without raising the pencil" This has terrible disadvantages when generalizing functions defined on a real interval to non connected sets, non compact sets and in general topological spaces. 


Not sure if this qualifies exactly, but I can never remember which theorems of group theory apply to finite groups, and which ones apply to groups in general. Anytime I remember a result, I have this sinking feeling that it appears in a textbook preceded by "for the remainder of this section, let G be a finite group." I'm not sure how wellfounded this fear is (other than the theorems that obviously don't make sense for infinite groups, like the Sylow theorems). 


Almost any heruistic can be "most harmful" if used by a teacher in a situation when the audience does not know why it makes sense, and without an explanation. This is especially dangerous in the frequent case that the heruistic does not actually seem reasonable to a person seeing it for the first time, since it makes sense only in some ways but not others. It might require months of experience for an uninitiated person to understand how and why it applies. For example, the heuristic of schemes as manifolds is such  every algebraic geometer understands it, but it actually is harmful to a person who is seeing schemes for a first time (such a person would vary likely interpret this heruistic as saying that affine schemes are trivial to understand). Same applies to "integration is the inverse of differentiation", and some of the other answers to this question. Of course, these heuristics are also the most useful ones, once you (and any audience you might have) actually understand them. The whole point of learning math is to gain more such heuristics, and to makes the ones you have more precise. For this reason, it seems to me that the use of such heruistics on an unprepared audience is the most common problem in the lectures by the very best mathematicians. A related problem is the an abundance of statements that are not strictly true, but "correct in spirit". Again, this may be very useful in research or when talking to a person of appropriate sophistication, but it is very bad for students if such statements are used carelessly and without explanation. P.S. This whole answer is generalization for the sake of generalization. Was it a waste of time, I wonder? 


Also not really a heuristic, but "differentiation is easy," as encoded in the following two subheuristics:
Edit: Someone doesn't seem to like this answer, so I'll expand. Students who leave calculus with this impression enter analysis with a disadvantage: differentiation is not a property that "most" functions have in any reasonable sense, not even continuous ones, and to compute the derivative of a function that isn't given as a sum of compositions of "elementary" functions requires an entirely different mindset than the one that values the product and chain rule. 


Writing a proof as a chain of expressions connected by equals signs whether they are appropriate or not. 


"Differentiation and integration are inverse operations." To many calculus students, this is their conception of the fundamental theorem. There's truth to this heuristic, of course, but one needs to be constantly informed by a much deeper understanding of integration (and differentiation) in order to properly wield this correspondence in most situations beyond those encountered in a first course in calculus. 


I wish to point the attention on Pete Clark's very relevant initial comment. The term heuristic is often taken as synonymous to nonrigorous method, only based on intuition or experience. I personally dislike this acceptance of the word in mathematics, and I suspect it is not even historically correct (now I'm curious to check the use of it in the classic authors). The etymology of the adjective, from the verb εὑρίσκω (to find, discover) means "aimed to find". As I see it, it is exactly the method we follow when looking for a solution of a problem: using all implications of being a solution in order to identify a candidate solution. Of course, the heuristic is only half the job, and it is only rigorous if followed by part 2: checking the solution. But there's a very smart idea in it. For instance: solving an equation, transform it, but do not check the equivalence of each single step, just follow a chain of implications. So, what is harmful is not the heurstic method, but leaving out the (often less creative) part 2. That said, here's my example: let F be a smooth function bounded below (or a functional) with only one critical point. Then one would argue:
False!, if one does not check that F(x_{0})≤F(x) for all x ("direct method in Calculus of Variations") or if one has not proved the existence of a minimizer (indirect method). Many students make this mistake... but not only them! 


"you'll need a computer for that". 


In elementary school, there are false principles which take a lot of effort to overcome:
These may be ok (though the second is debatable) when you are working on $1+2$, but not when you are supposed to isolate a variable, to graph a function, to recognize how you can apply the chain rule, to solve a complicated word problem, or to prove something. Many students don't think math is a place to experiment or to apply creativity. They are afraid to take incorrect steps even when it is no longer convenient or possible to say what the right first step is. There is an interesting app called Dragonbox. It is very popular in Norway. When children think of algebra as a puzzle or game, they feel free to experiment, and they quickly learn to do things like isolate variables which usually give algebra students trouble. See also Terry Tao's blog posts on gamifying algebra. Students can learn to solve the problems, but have difficulty because these incorrect principles get in the way. 


Any attempt to draw a fat Cantor set is a bad heuristic in my opinion. I saw such a diagram as an undergrad and believed for a while that there were intervals contained in the fat Cantor set. I don't think it's possible to express in a picture that a fat Cantor has positive Lebesgue measure and has empty interior. 


From Keith Devlin's article http://www.maa.org/devlin/devlin_06_08.html "Multiplication is repeated addition." This is true when multiplying natural numbers, but is a special case of a scaling operation in the reals. We know it is also a rotation in the complexes, but that should probably be left out at the beginning, although it might interesting to think about how one would include them at the beginning. Devlin also mentions "exponentiation is repeated multiplication." 


The excluded middle ( A Law or an Heuristic) . On a more general level given any closed question: Is it A or B ? , the heuristic says it is one or the other disregarding the option : the question is wrong or stupid or irrelevant or incomplete. The principle of excluded middle disregards intuitionist logic. And has been harmful in not providing direct (constructive) proofs which are often more clear  yet can be harder to find. Intuitionism is is also rather natural : being against anticommunists does not means you are a communist. 


"Mathematical knowledge is contained and communicated primarily by documents." I'm not sure if this is a heuristic, but in terms of beliefs that inhibit learning, this is definitely the one that hurt my mathematical development the most. I would say the correct statement is "Mathematical knowledge is contained primarily in the minds of mathematicians and communicated primarily by informal oral communication." This problematic belief grew out of the way that I (and pretty much everyone else) was taught mathematics at the undergraduate and beginning graduate level. In this setting texts are a central authority and a complete, wellwritten resource for the knowledge needed to solve any mathematical problem encountered. In the world of mathematical research, this is no longer the case. I finally figured this out by reading Thurston's essay "On proof and progress in mathematics", which I would strongly recommend for any beginning mathematician. Maybe it is possible to do research mathematics using papers as a primary resource, but I believe this is highly inefficient. I spent several years trying to learn the noncommutative standard model by reading the available papers on the subject and made no real progress. Looking back, I don't think I ever had a chance of succeeding with this approach. I would guess that to be successful in mathematics, it is absolutely vital to become regularly involved in conversations with working mathematicians, as awkward and intimidating as that might be. 


"A set is a collection of elements". Firstly, this does not distinguish sets and classes. Next in ZFC, sets are characterized more by their relation to each other by $\epsilon$, rather than that they contain anything. 

