Yes, with a straightforward induction.

Let $F$ be a finite Golomb subset. Let $n\ge 1$ be the smallest element not in $\mathrm{dist}(F)$. 

Define $F_m=F\cup\{m,m+n\}$. Define $u(F)=F_m$ where $m$ is the smallest number such that $F_m$ is also Golomb. To prove this is well-defined, it is enough to show that there exists $m$ such that $F_m$ is Golomb, and actually that every large enough $m$ works. Indeed, suppose that $m$ is large enough. Take two distinct increasing pairs and show they have distinct distances. If both are in $F$, then it's OK. If one is in $F$ and the other is $(m,m+n)$, it's OK. If one is in $F$ and one is half in $F$, it's OK (provided $m> 2\max(F)$). If both are half in $F$, say $(f_1,m)$, $(f_2,m+n)$, we have $m-f_1=m+n-f_2$, i.e., $f_2-f_1=n$, contradiction.

Hence define $X_0=\emptyset$, $X_{i+1}=u(X_i)$. Then $X=\bigcup_i X_i$ works.

For instance $$X_{15}=\{1,2,5,7,15,22,38,47,65,76,120,132,154,173,$$
$$241,265,327,353,482,510,575,605,786,821,984,$$ $$1023,1151,1198,1382,1430\}.$$
(It's not in OEIS, apparently.)