No, it is not enough to consider convex combinations of pairs of points in the connected set. A famous example is the moment curve $(t,t^2,t^3,\dots,t^n)$ where when you take the convex hull all convex combinations of [n/2] points form a face of the convex hull. Caratheodory theorem asserts that for every $X$ in $R^n$ a point in the convex hull of X is in the convex hull of $d+1$ points from $X$. I vaguely remember that when $X$ is connected you can replace $d+1$ by $d$ but I am not sure about it.
Added later: Indeed it is an old theorem that you can replace $d+1$ with $d$ when $X$ is connected. A recent theorem of Barany and Karasov Karasev assets that if $X$ is a set in $R^d$ with the property that all projections of $X$ into a $k$ dimensional space are convex, then every point in the convex hull of $X$ is already in the convex hull of d$d+1-k$ points from $X$.
