1d TQFT's are in 1-1 correspondence with finite dimensional vector spaces, and the image of the circle is the dimension of that vector space.

I think what you have in mind is instead the notion of $X$-structured TQFT, aka homotopy quantum field theory. Those are defined for pairs of a topological manifold together with a map into some fixed topological manifold $X$.

So in the 1d case, you are looking for symmetric monoidal functors into Vect from the category $Bord_1^X$ which has 

 - objects points together with a map into $X$, so the set of objects is just $X$.
 - morphisms bordisms between those, i.e.  maps between two points of $X$ are intervals equipped with a map into $X$ in a compatible way, i.e. at the end of the day this is just a path in $X$ between your points.

Now you want to look at this up to homotopy, so the bottom line is that $Bord_1^X$ really is just the fundamental groupoid of $X$, and a 1d $X$-HQFT is thus a **flat** vector bundle on $X$. So not only the connection is important, but it has to be flat. 

Now I'm not entirely sure, but I think given a vector bundle on $X$ with a non-necessary flat connection $A$, you get in fact an example of a 2-dimensional HQFT, where very roughly the value on a 2-dimensional surface equipped with a map into $X$ is computed by integrating the pull-back to your surface of the connection form on $A$. This is basically saying every connection is automatically "2-flat" thanks to the Bianchi identity.