Yes (we're over C yes?)- the space of line bundles with connection forms a torsor for the cotangent bundle over the Jacobian. The class of this torsor (as an element in $H^1(Jac,\Omega^1)=H^{1,1}$) is the Chern class of the theta line bundle. In fact one can construct canonical connections on generic line bundles of degree zero on a curve (in fact line bundles outside the theta divisor), once you give yourself the choice of a theta characteristic on the curve -- this is the theory of the "prime form" or Szego kernel (it's much much older, but I think a fairly easy "modern" exposition of this and the nonabelian version is in here).

[EDIT: Nonalgebraically it is very easy to see this assertion from the Hodge theorem: the space of line bundles with a flat connection is $H^1(X,C^\times)$, which can be identified with the product $H^1(X,O^\times)^{\circ}\times H^0(X,\Omega^1)$ just by exponentiating the Hodge theorem for H^1. Note that ANY holomorphic connection on a Riemann surface/algebraic curve is flat for dimension reason - there are no holomorphic 2-forms that could serve as curvature.. For higher rank bundles the analogous result is the topic of the Corlette-Simpson nonabelian Hodge theorem.]

More generally Andre Weil proved that a rank n bundle on a curve /C admits a connection if and only if every indecomposable summand is a vector bundle of degree zero. There's a nice proof of this by Atiyah as an application of the Atiyah class (which is the canonical obstruction to the existence of a connection on a vector bundle). In particular over the moduli of stable degree zero bundles we again have a nontrivial torsor for the cotangent bundle parametrizing bundles with connection, and the class of this torsor is again the Chern class of the determinant line bundle.