Re: ordinary mathematics, I think your assumptions are incorrect (although the question is interesting on its own, since ZFC+V=L is a natural set theory). ZFC+V=L is (most, including Simpson, would argue) more than sufficient for ordinary mathematics. But then, so is ZFC, and indeed much less; there's nothing that singles ZFC+V=L out, here. Moreover, "stronger" theories (in the sense of consistency or interpretive strength, not outright implication) like ZFC+large cardinals are, even more so, sufficient for ordinary mathematics. So "the largest large cardinal axiom compatible with V=L" doesn't really have any special connection with ordinary mathematics.
In particular, if you want a "conservative" set theory - that is, one sufficient for ordinary mathematics, but (close to) minimally so - then you've already overshot the mark with ZFC. I would argue ZC (= ZFC without replacement - vastly weaker) is even overshooting this, although at this point we get into the fine details of what exactly "ordinary mathematics" is (does Borel determinacy count? because if it does, then ZC isn't enough).
And if you don't want a conservative set theory, then in what sense are the large cardinals compatible with V=L the only ones "allowed by current mathematical practice"?
A comment on motivating foundations:
I suspect that what's going on is a sort of "hybrid foundations", as follows:
First, we commit ourselves to ZFC, together with the rule that we won't unnecessarily bound the height of the ordinals which exist (e.g. we won't add "No inaccessibles" as an axiom without good reason); this is for one set of reasons, which lean on the maximality side. In particular, Replacement is justified as its negation provides an "artificially small" universe of sets.
We then - within this framework - declare a "mood change": further axioms will be justified by appeal to mathematical practice, this time leaning towards the minimality side - no unneeded sets are conjured into existence. Things like nonprojective sets of reals are already provably extant, from our previous commitments; but, since nonconstructible reals don't currently play a role in ordinary mathematics, we not only don't assume their existence, we actively assume that they don't exist. In particular, we adopt V=L since it is the "narrowest" axiom compatible with ZFC.
This is a reasonably consistent approach; however, I find it flawed for several reasons. The choice of mood change is unjustified - why have we exhausted the foundational value of the "maximality" approach right at ZFC? Moreover, I'm not swayed by the "minimality" approach at all: I see no reason to artificially narrow the universe, except perhaps for reasons of consistency strength - but these reasons militate against small large cardinals like inaccessibles, and not against failures of constructibility (since we can force V=L to fail, without higher consistency strength)! Finally, I don't see why the background set theory has to limit itself to ordinary mathematics at all. While the question of exactly how much set theory is necessary for ordinary mathematics is an interesting one (I am a reverse mathematician among other things, after all), it doesn't carry a value judgment, at least for me; indeed, one of the roles of foundations of mathematics in my view is to go well beyond current ordinary mathematics, and blaze new trails, motivated by the mathematical optimism that those trails will eventually be useful.