This kind of situation is actually pretty normal in math research. The answer basically hinges on the meaning of "dramatically." If it was only a minor improvement, then the normal thing to do would be to work it in when you hear back from the referee. But, if it's a significant enough improvement to be worthy of publication in its own right, then you should try to publish it separately. It's not your fault that math research moves so fast these days, or that there's such a long delay in the publication pipeline. You could imagine that, in a world with fewer delays, the first paper would have already been accepted by the time you made the dramatic new discovery. Early on in your career, it's usually best to get two papers in this kind of situation. There's nothing unethical about still publishing the first paper, even knowing that you're working on a better paper that will subsume it. Most of us are working most of the time towards a better understanding, and realize that things we publish might get subsumed later. Sometimes the smaller result actually gets published after the better one, though it's usually best to avoid this situation if possible.
I can give you a concrete example from my case. My PhD thesis consisted of the following two papers:
https://arxiv.org/abs/1403.6759 (published in 2017)
https://arxiv.org/abs/1404.5197 (almost published in 2017 but then eventually published only in 2021)
Quickly, I saw that I could subsume them with
https://arxiv.org/abs/1503.06720 (published in 2018)
Nevertheless, I published all three. This kind of thing is common. Also, the first has more citations than the third even though the third subsumes it. Sometimes, it really helps the reader to have one clear and compelling concrete example, before you give them the general machinery that works for that example and all others.
More cynically, there's the notion of the "least publishable unit." I stop short of endorsing that, but I do think it's wise for junior researchers on the job market to know when they've got "enough" for a good paper, and get that submitted. When you're applying for jobs, it looks better to have several reasonable-length publications that assemble to a general theory rather than one long preprint that covers the whole theory at once, but will be very difficult to publish because it's so long and has so many ideas in it. It's better for the author, the readers, the referees, and the community, to get those ideas out in smaller units, at least until you have tenure.
Lastly, I'll remark that questions very similar to this one have been asked both on MO and Academia, and the answers seem to align with the advice I've written here.
Should I inform the editor about a generalized result of a result in a paper under review?
Submitting paper proving "X" soon after paper proving "X-epsilon"
Some recommend telling your editor about the improvement and letting the editor decide. However, that can lead to delays and an annoyed editor, though it's important if there was actually a major mistake in the first submission. But, if the improvement is "dramatic" and if the original paper was good enough to be published where you submitted it, then I think it's better to get two publications since you did two publications worth of work. Don't forget that checking details and writing takes time, and by the time you've got the dramatic improvement written up, you might already have a referee report in hand on the first submission. Also, the first result can build a desire in the community for more results (and stronger results) in the same vein, creating a demand that the "dramatic improvement" fills.
Anyway, if any reader finds themself in this situation, it's a good problem to have! Remember that progress in research can be somewhat random, and the literature is full of oddities due to situations like this. Just get things submitted, give good talks, and do your best to bounce back from setbacks (e.g., resubmitting to lower journals if something gets rejected), and usually it all works out in the end, even if the order things appear is wonky.