I wish to point the attention on Pete Clark's very relevant initial comment. The term heuristic is often taken as synonimous of not rigorous synonymous to non-rigorous method, only based on intuition or experience. I personally dislike this acceptance of the word in mathematics, and I suspect it is not even historically correct (now I'm curious to check the use of it in the classic authors). The etimology etymology of the adjective, from the verb εὑρίσκω (to find, discover) means "aimed to find". As I see it, it is exactly the method we follow when looking for a solution of a problem: using all implications of being a solution in order to identify a candidate solution. Of course, the heuristic is only half of the job, and it is only rigorous if followed by part 2: checking the solution. But there's a very smart idea in it. For instance: solving an equation, transform it, but do not check the equivalence of each single step, just follow a chain of implications. So, what is harmful is not the heurstic method, but leaving out the (often less creative) part 2. That said, here's my example: let F be a smooth function bounded below (or a functional) with only one critical point. Then one would argue:
Any minimum point of F(x)=0 satisfy satisfies F'(x)=0, whose only solution is x0. Hence, x0 is the minimizer.
False!, if one does not check that F(x0)≤F(x) for all x ("direct method in Calculus of Variations") or if one has not proved the existence of a minimizer (indirect method). Many students do make this mistakes..mistake... but not only them!