1d TQFT's are in 1-1 correspondence with finite dimensional vector spaces, and the image of the circle is the dimension of that vector space.
I think what you have in mind is instead the notion of $X$-structured TQFT, aka homotopy quantum field theory. Those are defined for pairs of a topological manifold together with a map into some fixed topological manifold $X$.
So in the 1d case, you are looking for symmetric monoidal functors into Vect from the category $Bord_1^X$ which has
- objects points together with a map into $X$, so the set of objects is just $X$.
- morphisms bordisms between those, i.e. maps between two points of $X$ are intervals equipped with a map into $X$ in a compatible way, i.e. at the end of the day this is just a path in $X$ between your points.
Now you want to look at this up to homotopy, so the bottom line is that $Bord_1^X$ really is just the fundamental groupoid of $X$, and a 1d $X$-HQFT is thus a flat vector bundle on $X$. So not only the connection is important, but it has to be flat.
Now I'm not entirely sure, but I think given a vector bundle on $X$ with a non-necessary flat connection $A$, you get in fact an example of a 2-dimensional HQFT, where very roughly the value on a 2-dimensional surface equipped with a map into $X$ is computed by integrating the pull-back to your surface of the connection form oncurvature 2-form of $A$. This is basically saying every connection is automatically "2-flat" thanks to the Bianchi identity.