If you have a sequence $S$ of polynomials, the Magma command $RelationIdeal(S)$ should give the answer w/o precising any invariant theory. Whether or not it will work in finite time and memory is a different question.

They have 5 joint papers together and a book, Kahane, Jean-Pierre; Salem, Raphaël, Ensembles parfaits et séries trigonométriques. (French), Actualités Sci. Indust., No. 1301 Hermann, Paris 1963 192 pp. Second edition. With notes by Kahane, Thomas W. Körner, Russell Lyons and Stephen William Drury. Hermann, Paris, 1994. 245 pp. ISBN: 2-7056-6193-X. This book (w/o page number) seems to be the typical reference. The updated Zygmund (Trigonometric Series) cites it, as does Geometry of Sets and Measures in Euclidean Spaces (Mattila).

I would not say it is much of an oversimplification either. A similar heuristic can be seen in Poonen-Rains (or earlier work of de Jong) for Selmer groups. The idea at the bottom can be said: first to note the dimension of the invariant subspace is the same as the number of eigenvalues equal to 1, and then that RMT speculates such eigenvalues should account for the rank (exactly). In the function field analogue, much more is known about the validity of this second step.

The only place where Watkins uses RMT is the 3/8 exponent for log; he says the 19/24 can be derived more crudely. For RMT, model $L$-values via charpolys and apply a discretization process in an arithmetic analog. One has $Prob(P(1)\le t)\sim M^{3/8}\sqrt{t}$ as $t\sim 0$ for the charpoly $P$ of an orthog matrix, where $M$ is the matrix size. For ellcurv we guess the same for $Prob(L(E,1)\le t)$ with an arith factor that averages out. He matches $M\sim\log N$ as ellcurv conductor, mucks cond vs disc vs $1/\Omega^{1/12}$, discretizes as Sha is integral, and ends with the 3/8 he started with.

I think it is a different approach. The Watkins paper only references a to-come work with Granville, which seems to be the one cited in my answer. They work in the twist case there, but would basically heuristically bound the number of (integral?) points $(A,B,X,Y,Z)$ on $Y^2Z=X^3+AXZ^2+BZ^3$ in some ranges, and compare this to the number of such points (via ellipsoids) that a curve of rank $r$ with parameters of size $(A,B)$ would generate in said ranges. I think Lang-Vojta guessing is similar. Maybe I will try to write this as an answer, if I can work out how the non-twist case differs.

I am not sure how this squares with the "linear decay" in density suggested by the other 2 answers. It seems that to make this analogue, you think of the $O(n)$ over some large field of size $q$, and the number of rank $k$ curves is about $q^{(n-k)(n+k-1)/2}$. The heuristic mentioned by Matt Young OTOH says at "size $X$" the chance of a random curve being rank $r\ge1$ is $\sim1/X^{(r-1)/24}$ with $X^{5/6}$ total curves. Balancing $q$ and $X$ as you like, your calculation suggests a $1/q$ chance of $k=2$, then a $1/q^3$ of rank 3, then $1/q^6$ of rank 4, etc., or faster than linear decay.