I first encountered this kind of speculation in Chapter 3 of Richard Stanley's book Enumerative Combinatorics, where Exercise 3(c) (or, in the second edition, Exercise 5(c)) suggests that if $f(n)$ is the number of non-isomorphic posets on $n$ elements, then the assertion that infinitely many $f(n)$ are palindromes in base ten is independent of the axioms of ZF. Clearly, this exercise is not intended to be tackled seriously; it's really an expression of the sentiment that there are some statements out there that are either true or false, but that we cannot hope to prove one way or another because they almost certainly lack the "structure" that we normally seek when doing mathematics.
If you want to try to formalize this notion, then one approach is to look at certain proofs of Gödel's incompleteness theorem. For example, if you fix an axiomatic system such as ZF, then the theorems of the system are computably enumerable, but the set of arithmetical truths is not computably enumerable, so there must exist some truths that are not provable simply because (informally speaking) they are "too complicated to compute." If you want to emphasize the idea that the unprovable statements are "complex" or "unstructured" in some sense, then you might prefer Chaitin's proof of the incompleteness theorem, which shows that for any formal system $S$, there is a constant $L$ such that the statement "$K(s) > L$" is unprovable in $S$ for all strings $s$ (here $K$ denotes Kolmogorov complexity). The vast majority of such statements are true "at random" because a random string will have high Kolmogorov complexity.
However, you might not be satisfied with the above approach, because your intuition about the graph reconstruction conjecture is not based on the idea that the formal statement of the conjecture is so complex or uncomputable that it cannot be proved. The conjecture, after all, can be stated very simply. It's the apparent lack of relevant structure in the set of all graphs that is causing trouble.
It might be helpful to specify more carefully what kinds of "true by accident" statements you are thinking of. One approach would be to construct a heuristic probabilistic model that predicts that certain things ought to be true just for "random reasons." For example, there is Cramér's random model for the primes, which can be used to give heuristic "proofs" of various number-theoretic conjectures; e.g., one can use the model to predict that there will be only finitely many primes with such-and-such a property, because the probability that a prime $p$ has the property decreases rapidly to zero as $p\to\infty$. It is easy to come up with many such conjectures that have a "true by accident" feel to them. (In particular, I think it would be interesting if you could come up with a heuristic probabilistic model for graph theory, in the spirit of Cramér's model, that could "predict" various well-known graph-theoretic conjectures, including the reconstruction conjecture.)
The trouble with this approach is that there doesn't seem to be any clear way to declare that some particular statement of interest (such as the graph reconstruction conjecture) doesn't have a nice proof. In a related MO question, Goldbach's conjecture is proposed as an example of something that might be "true by accident," but there's enough relevant structure that such a claim is highly controversial. The space of all possible proofs is itself a highly complex mathematical object, so who is to say that the statement "Intractable-looking conjecture X has a simple proof" couldn't be "true by accident"? Maybe there exists a beautifully simple proof out there, but it's a tiny needle buried in a totally unstructured haystack, and so we humans will never be able to find it.
In summary, there are some ways one could try to formalize this notion, but unfortunately, I don't think any of them lead in a promising direction (other than perhaps my suggestion above that it would be interesting to formulate a heuristic probabilistic model for graph theory that could "predict" the truth of certain conjectures without actually proving them).
EDIT (August 2022): I recently learned of John Conway's 2013 Amer. Math. Monthly article, On unsettleable arithmetical problems, which among other things gives examples of probabilistic reasoning in support of a claim that some proposition is "unsettleable." To give you the flavor, let me quote from Conway's Postscript:
The following argument has convinced me that the Collatz $3n + 1$ Conjecture is itself very likely to be unsettleable, rather than this merely having the slight chance mentioned above. It uses the fact that there are arbitrarily tall “mountains” in the graph of the Collatz game. To see this, observe that $2m − 1$ passes in two moves to $3m − 1$, from which it follows that $2^k m − 1$ passes in $2k$ moves to $3^k m − 1$. Now by the Chinese Remainder Theorem we can arrange that $3^k m − 1$ has the form $2^l n$, which passes by $l$ moves to $n$. There is a very slight possibility that $n$ happens to be the same as the number $2^k m − 1$ that we started with. Let’s suppose that the starting number $2^k m − 1$ is about a googol; then the downward slope of the mountain certainly contains a number between one and two googols, so the chance that this is the same as the starting number is at least one googolth. (This is justified by observations for smaller $n$ showing that the first iterate that lies in the range $[n, 2n)$ is approximately uniformly distributed in this range.) In my view the fact that this probability, though very small, is positive, makes it extremely unlikely that there can be a proof that the Collatz game has no cycles that contain only large numbers. This should not be confused with a suggestion that there actually are cycles containing large numbers. After all, events whose probability is around one googolth are distinctly unlikely to happen!
EDIT (February 2024): Here's another potentially illuminating remark by Conway. Conway was convinced that some mathematical facts are definitely not "true by accident." As mentioned in the MO question Why does the monster group exist? Conway said that the monster group is "obviously not there just by coincidence; it's got too many intriguing properties for it to all be just an accident." Note that there mere fact that the existence of the monster group had already been proved did not satisfy Conway; he did not feel that the proof answered the "why?" question. So at least for Conway, having a proof does not automatically guarantee that something is not "true by accident" (or in other words, being "true by accident" does not automatically mean that something is not provable).