First of all, thank you Gerry Myerson for bringing this problem to my attention at West Coast Number Theory 2019.
The answer to this question is No.
We prove that under the assumptions of this problem, $|X|$ must be prime.
Step 1 : Reduction
Let $|X|>2$ and $\zeta_N=\exp(2\pi i (1/N))$.
The sequence $s_n=\sum_{x\in X} x^n$ satisfies a linear recurrence relation. By Skolem-Mahler-Lech theorem, there is an arithmetic progression $\{an+b\}_{n\geq 0}\subseteq \mathbb{N}$ with $a>0$, $b\in \mathbb{N}$ such that $$ s_{an+b}=\sum_{x\in X} x^{an+b} = 0 \ \textrm{for all} \ n\in\mathbb{N}. $$
Step 2 : Vandermonde
Let $X=\{x_1,\ldots,x_k\}$. The result from Step 1 forms a Vandermonde system $$ \begin{pmatrix} 1 & 1 & \cdots & 1 \\ x_1^a & x_2^a & \cdots & x_k^a \\ x_1^{2a} & x_2^{2a} & \cdots & x_k^{2a} \\ \vdots& \ddots& \cdots& \vdots \\ x_1^{(k-1)a} & x_2^{(k-1)a} & \cdots & x_k^{(k-1)a}\end{pmatrix}\begin{pmatrix} x_1^b \\ x_2^b \\ \vdots \\ x_k^b \end{pmatrix}=\begin{pmatrix} 0 \\ 0 \\ \vdots \\ 0 \end{pmatrix}. $$ Since $x_i\neq 0$, the Vandermonde matrix must be singular. This yields $x_i^a = x_j^a$ for some $i\neq j$. Without loss of generality, assume that $x_1^a=x_2^a$. Then we may rewrite the system as $$ \begin{pmatrix} 1 & 1 & \cdots & 1 \\ x_2^a & x_3^a & \cdots & x_k^a \\ x_2^{2a} & x_3^{2a} & \cdots & x_k^{2a} \\ \vdots& \ddots& \cdots& \vdots \\ x_2^{(k-2)a} & x_3^{(k-2)a} & \cdots & x_k^{(k-2)a}\end{pmatrix}\begin{pmatrix} x_1^b+x_2^b \\ x_3^b \\ \vdots \\ x_k^b \end{pmatrix}=\begin{pmatrix} 0 \\ 0 \\ \vdots \\ 0 \end{pmatrix}. $$ If $x_1^b+x_2^b=0$, it yields a shorter vanishing sums in the expression of $s_{an+b}=0$. So, we must have $x_1^b+x_2^b\neq 0$. Thus the above Vandermonde matrix is also singular, and we obtain another $i\neq j$ (both $\geq 2$) with $x_i^a=x_j^a$. Repeating this process, we obtain $$ x_1^a = x_2^a = \cdots = x_k^a. $$ Dividing by $x_1$, we may assume that all members of $X$ are roots of unity.
Step 3 : Vanishing sums of roots of unity
We refer to the results of this paper: T. Y. Lam, K. H. Leung, 'On the vanishing sums of roots of unity'
In this paper, the vanishing sums of roots of unity $\alpha_1+\cdots+\alpha_k=0$ is called minimal if no proper sums vanish. The characterization of the minimal vanishing sums of roots of unity is either
(1) $1+ \zeta_p+ \cdots +\zeta_p^{p-1}=0$ for prime $p$, or
(2) $(\zeta_p+\cdots + \zeta_p^{p-1})(\zeta_q+\cdots+\zeta_q^{q-1})+(\zeta_r+\cdots+\zeta_r^{r-1}) = (-1)(-1)-1= 0$ for primes $p<q<r$.
and these forms multiplied by a root of unity $\zeta_s^t$.
If we have (1), then $|X|=p$ is prime.
If we have (2), then consider $pq$-th powers. The sum in (2) after $pq$-th powers of each term, we obtain $$ (1+\cdots +1) ( 1+ \cdots +1) + (\zeta_r^{pq} + \cdots + \zeta_r^{pq(r-1)}) $$ Since $(pq,r)=1$, the sum $(\zeta_r^{pq} + \cdots + \zeta_r^{pq(r-1)})$ is a rearrangement of $\zeta_r+ \cdots + \zeta_r^{r-1}$. This yields a vanishing sub-sum of length $r$, $$ 1+\zeta_r+ \cdots + \zeta_r^{r-1}=0. $$ Note that $(p-1)(q-1)+r-1 > r$. Thus, we cannot have (2) under the inclusion-minimal assumption of this problem.