Goldfeld made the conjecture for the family of quadratic twists of a given curve; it would be interesting to check the paper to see if he wrote anything about the more general case (I don't have it at hand). For the latter, the possibility that the average rank might be 1/2 was certainly folklore quite a while before Katz-Sarnak, but the numerical evidence did not (still doesn't, I think) really support this, so it was not usually stated as a formal conjecture. But see, for instance, the introduction to "The average rank of elliptic curves, I" by Brumer (Invent. math. 109, 1992), or even the last lines of the first page of "The rank of elliptic curves", by Brumer-Kramer (Duke Math. J. 44, 1977):

Our bounds suggest that, in general, curves of prime conductor have the smallest rank compatible with the parity predictions of Birch and Swinnerton-Dyer.