The set can be infinite (but only countable). For an example choose any $t_j$ linearly independent over $\mathbb Q$ and let $V$ be the set of all $v$ such that $vt_j\mod 1 \notin (\frac 12-a_j,\frac 12+a_j)$ for all $1\le j\le J(v)$ where $a_j$ decrease so fast that $\sum_j a_j<+\infty$ and the function $J$ increases to $+\infty$ so slowly that by the moment $v_0$ it reaches $j_0$ you may say that $e^{2\pi ivt_j}$ for $v=1,\dots,v_0$ model the joint distribution of $j_0$ independent variables equidistributed over the unit circle with high precision. That (with some small extra care and effort) will give you the existence of non-zero limits even for Cesaro means and the Poisson summation method is stronger.
Edit (about the sum of squares). Let us assume that the limit $L_j=c(t_j)$ exists for some finite collection of $t_j$. Consider
$$
L=\lim_{x\to 1}\sum_j \bar L_j(1-x)\sum_{v>0}\chi_V(v)x^ve^{2\pi i t_j v}\,.
$$
On the one hand, $L=\sum|L_j|^2$. On the other hand, by Cauchy-Schwarz,
$$
L^2\le \limsup_{x\to 1}\left[(1-x)\sum_{v>0}\chi_V(v)^2x^v\right]
\left[(1-x)\sum_{v>0}\left|\sum_j\bar L_j e^{2\pi i t_j v}\right|^2 x^v\right]
$$
The expression in the first brackets is at most $1$, while, if you open the absolute value and take into account that
$$
(1-x)\sum_{v>0}e^{2\pi i (t_j-t_k) v} x^v\to 0
$$
for $j\ne k$, you'll find that the limit of the sum in the second brackets is $L$. That gives $L^2\le L$, so $L\le 1$.
Apologies to everybody for endless edits. Here comes another one.
Consider a very fast increasing sequence $n_k$, $k\ge 1$ of positive integers (as usual, that means that we will choose them inductively). Let $N_k=\prod_{m=1}^k(2^{n_m}-1)$.
Define $V_k$ as the set of $v\ge 1$ congruent to $N_{k-1}2^\alpha$ modulo $N_k$ with some $\alpha\in\{0,1,\dots,n_k-1\}$ (we define $N_0=1$). It is clear that we just take $n_k$ possible residues modulo $N_k$, so the density of $V_k$ is $n_k/N_k$.
Put $V=\cup_{k\ge 1} V_k$ and $t_k=N_k^{-1}$. That's the whole construction. Now it remains to explain why it works.
First, $V_k$ are disjoint. Indeed, if $m<k$, all elements of $V_k$ are divisible by $N_m$ but no element of $V_m$ is.
Second, $V_k$ is $N_k$-periodic, so even the Cesaro averages $L_k(z,M)=\frac 1{M}\sum_{v\in V_k,v\le M}z^{v}$ have limits for all $z=e^{2\pi it}\in\mathbb T$ (we will always assume this relation between $z$ and $t$).
Third, $\max_M\frac{|V_k\cap[1,M]|}{M}=N_{k-1}^{-1}$, so as long as the series $\sum_{k\ge 1}N_{k-1}^{-1}$ converges, we can safely claim that the limit
$L(z)=\lim_{M\to\infty}L(z,M)=\lim_{M\to\infty}\frac 1{M}\sum_{v\in V,v\le M}z^{v}$ exists and equals $\sum_{k\ge 1} L_k(z)$ ($L_k(z)$ are defined similarly with $V_k$ instead of $V$).
Fourth, each $V_k$ satisfies the property that if $v\in V_k$, then $2v\in V_k$ and, provided that $v$ is even, $\frac v2\in V_k$. Thus
$$
\{2v:v\in V_k\}=\{v\in V_k: v\text{ is even}\}\,.
$$
Since for every $M$, we have $L_k(z,M)+L_k(-z,M)=\frac 2M\sum_{v\in V_k, v\text{ even}, v\le M}z^v=\frac 2M\sum_{v\in V_k, v\le M/2}z^{2v}=L_k(z^2,M/2)$, we conclude passing to the limit as $M\to\infty$ that $L_k(z)+L_k(-z)=L_k(z^2)$ for all $k$ and, thereby $L(z)+L(-z)=L(z^2)$.
Fifth, if $t=N_{k}^{-1}$, then $L_m(z)=0$ for $m<k$ (just because $V_m$ is $N_m$-periodic and $N_k>N_m$, but
$$
L_k(z)=N_k^{-1}\sum_{\alpha=0}^{n_k-1}e^{2\pi i\frac{2^\alpha}{2^{n_k}-1}}\ne 0
$$
(all terms are in one half-plane) and $L_m(z)$ with $m>k$ cannot change that because they are ridiculously small (at most $n_m/N_m$) compared to anything depending on $n_1,\dots, n_k$ only (it is this condition that forces the fast growth of $n_k$). Thus $L(z)\ne 0$.
