Yes. This is obvious if there are no such cardinals. (I assume that the natural numbers of the universe of sets are the true natural numbers. Otherwise, the answer is no, and there is not much else to do.) 

Assume now that there are such cardinals, and that "large cardinal axiom" is something reasonable (so it implies that the relevant cardinals are weakly inaccessible). If $\kappa$ is the least such cardinal, then $L_\kappa$ and $V$ agree on all arithmetic statements, since they share the same natural numbers (in fact, they agree on all $\Sigma^1_2$ statements). 

Now argue in $L$: Either there are no such cardinals in $L_\kappa$, and we are done, or repeat the previous paragraph with $L_\kappa$ in place of $V$ and $L_{\kappa'}$ in place of $L_\kappa$, where $\kappa'$ is the least such cardinal from the point of view of $L_\kappa$. 

We are done if $L_\kappa'$ is a model where there are no such cardinals, otherwise, repeat the previous argument. This process necessarily stops since the sequence of ordinals we are producing is decreasing.

Now, if instead of weak inaccessibility you assume that consistency statements or the like qualify as large cardinal axioms, then "of course" the answer is no, as $\lnot A$ would be a false arithmetic statement.

[Note that Friedman's results, mentioned in another answer, assume more than the negation of the large cardinal in question in the "bad" case. The models of set theory where his statements fail are not $\omega$-models.]