I think you want Theorem 2.1 On the Ehrhart Polynomial of Minimal Matroids by Ferroni. Note Theorem 2.1 cites previous work that may be of interest, but I know of these matroids from this recent paper.
Theorem 2.1 says if you have a connected matroid on $n$ elements of rank $k$ then the number of bases is at least $k(n-k)+1$. Furthermore, there is a unique matroid up to isomorphism that realizes this bound (and that matroid turns out to be graphic). The construction of this matroid is described in Section 2 of the linked paper.