What to do if you notice a substantial improvement to a result in a paper whilst refereeing it? What would you do/have you done in such a situation? 


*

*Hand out the improvement for free in your report

*Wait until the result is published and then submit elsewhere 

*Inform the editor about the situation and ask for advice
The paper is not posted publicly so contacting the authors directly informing them and asking what they want to do is out of the question.
 A: If you have tenure, or even a tenure-track job, I'd definitely advise just doing 1.  If you're in a less secure job position and the improved result is really good, I'd suggest doing roughly 3.  That is you tell the editor "I think I can significantly improve the main result, do you think it would be appropriate for me to suggest this improvement non-anonymously to the authors?
A: Option (1) is definitely the professional course of action in this case. As pointed out in the remarks, it is likely to lead to an offer of co-authorship from the original author, but that is purely within the author's discretion. If you feel that your improvement is really substantial and you are worried about credit you can try to increase the chances of co-authorship by asking the editor to put you in contact with the author (after explaining the situation to the editor). You may then discuss this with the author directly and suggest co-authorship, a situation in which the author is more likely to accept (but they still may insist to refuse, in which case you should give them the idea "for free"). If your improvement is sufficiently significant and novel that not getting credit for it seems an unacceptable injustice, then what you can do is wait for the paper to be published (or accepted and online) and then write to the author with your idea of improvement and suggest co-authorship for a second paper. Here of course if they refuse you can publish alone. In any case, you should not submit your own paper without giving the original author a chance of co-authorship. That would be rather unprofessional.
A: I went with (1), and we ended up writing a joint paper which got published not too long ago. Thanks for the advice 
A: I strongly support the above answers listed below. This is what I do and this is what other referees did with some of my papers:
fedja: I invariably do #1. Usually that is reciprocated with an offer of co-authorship, which I accept or decline based on the circumstances.
Nicholas Kuhn: Just do (1), and pat yourself on the back!
Yonatan Harpaz: Option (1) is definitely the professional course of action in this case.
However, sometimes an improvement is a jump from a 5 pages long paper to a 50 pages one. Let me say what happened to me once.
I refereed a paper of X. I liked the result and recommend it being published. But I also realized that the topic (not the method used in the paper) can lead to far better results. I asked Y (my student, postdoc or colleague) to work on it (the original paper has already been published). Then X submitted an $\varepsilon$ improvement of the previous result. That was a much weaker result than what I and Y could prove. What should I do? This is what I did:
I wrote to X asking them to withdraw the paper from the journal and join our team. The outcome was a win for everyone: I and Y were motivated by the work of X, and X became a coauthor of a very good paper, much better than their $\varepsilon$ improvement.
My advice is: be honest and straightforward and never let yourself go to the grey area as otherwise, sooner of later, you will have enemies. Sometimes it might mean giving up on your own ideas by leading others to improve their results. Well, someone will pay it pack, eventually.
A: Just do (1), and pat yourself on the back!
A: My advise: don't do 3) in any case. It is up to you to decide, not the editor. The rest depends on the paper and on the improvement.


*

*Would  you recommend to accept the paper as is?
Suppose the answer is yes. Now imagine that you see this paper published, and you see how to make an improvement. Would you publish this improvement as a separate paper of your own? If yes, then do 2). If not, do 1).

*You think the paper in its present state is not worth publishing but your improvement will make it worth. Then do 1). Then it is likely that the author of the paper will propose you joint authorship. And you may agree or not.

*If you think that the paper does not deserve to be published (with the improvement or without). Then recommend to reject and do nothing else.
A: Option 4. You could become a coauthor and then write up an improved paper.
That's what happened to me in one of my number theory; The referee had several interesting suggestions to improve my results, that he explained in his report, and the editor asked me if I wanted to accept the referee as a coauthor and submit the improved paper in another journal (because the new framework was way larger than number theory). That's what we did.
