The article *Random strongly regular graphs?* by Peter Cameron http://www.maths.qmul.ac.uk/~pjc/preprints/randsrg.pdf provides some information about what is known and why someone might make that claim.

First an example: There are 11,084,874,829 strongly regular graphs with parameters SRG(57,24,11,9) which arise from a steiner triple system with 19 points (and 57 blocks); Of these 11,084,710,071 are rigid. (There might be other SRG(57,24,11,9))
MR2059752 (2005b:05035)
Kaski, P; Östergård, P
*The Steiner triple systems of order 19.*
Math. Comp. 73 (2004), no. 248, 2075--2092
http://www.ams.org/journals/mcom/2004-73-248/S0025-5718-04-01626-6/S0025-5718-04-01626-6.pdf

Cameron explains that the SRG with smallest eigenvalue -m are of 4 types:

1) a complete multipartite graph with km blocks of size m (so $v=km$)

2) Produced from m-2 mutually orthogonal kxk Latin squares (so $v=k^2$, nodes connected if they are in the same row or column, or have the same symbol in one of the squares)

3) The vertices are the blocks of a Steiner system with blocks of size m (so $v={\binom{k}{2}}/{\binom{m}{2}}$.

4) A finite list of exceptions $\mathcal{L}(m)$.

type 1 has a huge automorphism group, but there are not very many of them.
type 2 For m=3, there are on order of $n^{n^2/6}$ latin squares of order n, most with trivial automorphism group.
type 3 For m=3 one has Steiner triple systems as above, there are on order $n^{n^2}$ and most are rigid.

much less is known about sets of m mutually orthogonal latin squares and about Steiner systems with block size m.

There are also graphs whose lower two eigenvalues are irrational conjugates (in some ring).

Any graph with n vertices is an induced subgraph of a srg with at most 4n^2 vertices. On the other hand, every finite group is the automorphism group of a srg (if I recall correctly). So the feeling is that there are lots of srg with lots of freedom to construct them and most are rigid.

The notion of switching **is** useful. In a STS a Pasch Configuration is a set of 6 points and 4 triples abc ade fbe fcd This would correspond to a 4-clique in the corresponding graph. Switching these to abe acd fbc fde would still leave a 4 clique in the graph but would shift around the connections to the rest of the graph. There can be more elaborate switches too (I think). With enough room one can probably destroy all automorphisms this way. Of the rigid STS(19) above, 2538 dont have any Pasch configurations but over 1,000,000,000 have 14 (similarly for 15 and 16).