It helps me to remember that none of this stuff really matters.
That might sound discouraging at first; why do something if it doesn't matter? Because nothing really matters, and we have to occupy our time!
I was intentionally provocative in the presentation of the above idea, but in a more moderate tone it would probably read something like 'if mathematics makes you happy, do it because it makes you happy and forget the rest', or 'do it because it makes you happy and remember that we are all very small in a very big universe'.
It is somewhat Absurd to pursue challenging tasks in an indifferent universe, but while we're here we should occupy our time with things we find interesting and which bring joy to us and others -- we must imagine Sisyphus happy. Mathematical research is somewhat intrinsically difficult, but for most people who ultimately pursue it the labor is one of love, and a source of limitless wonderment and joy (or wonder and joy with limits if you work in category theory/calculus ;).
All of this is to say, don't sweat the small stuff. Pursue the things you find fascinating, and what will be will be -- que sera sera. (I find that good music helps too.)
After reading this answer before posting, it occurrs to me that none of the content of this message is specific to mathematics in the sense that we could swap out the word 'mathematics' for any other challenging pursuit and the same point would stand. I've flagged the question to be converted to CW (community wiki) by a moderator and would like to wait to post this answer until that happens, but I'm posting it now because I've waited an hour after flagging and would like to post it and go to sleep.