Consider the following scenario: There is some mathematical constant $c$ that you want to compute. You don't have a formal proof for any particular value of $c$, but you have some sound statistical procedure that will determine $c$ (up to a certain error, perhaps) with high probability. For example, $c$ is the value of some integral and you estimate it with Monte-Carlo sampling.
Now, this is not as rigorous as a Hilbertian proof of the value of $c$. But it is also much more rigorous than a heuristic. One can say confidently, for instance, that your statistical procedure determines the correct value with an exponentially high probability.
(I think you need to be careful of Bayesian reasoning; you should not conclude that the value of $c$ is any particular value with high probability, as usually in the Bayesian paradigm we assume that the subject is capable of perfect reason in the presence of data, and a perfectly rational reasoner would probably deduce $c$ from a Hilbertian proof)
Could, or should, use of statistical procedures be considered valid rigorous reasoning? This scenario is not at all far-fetched.