For a given formal series $g(x)=\sum_{k=0}^\infty g_k x^k$ I would like to find a formal series $f(x)=\sum_{k=0}^\infty f_k x^k$ such that they satisfy the difference equation $$ f(x+1)-f(x)=g(x). $$
Is there any description of all $g(x)$ for which $f(x)$ exists?
In general, this question makes sense at a formal level near $\infty$ only, i.e. for power series involving negative powers of $x$. Setting $z:=\frac{1}{x}$, the equation becomes a so-called homological equation $$ F(\Delta(z))-F(z)=G(z) $$ where $$\Delta(z):=\frac{z}{1+z}$$ and $F(z):=f(\frac{1}{z})$, $G(z):=g(\frac{1}{z})$.
An obvious obstruction is $G(0)=0$, because $\Delta(0)=0$. Notice that $\lim_{n\to +\infty}\Delta^{\circ n}(z)=0$, so that you can use telescoping sums : $$F(z)=F(0)+\sum_{n=0}^{\infty}G\circ\Delta^{\circ n}(z).$$ This expression is well-defined at a formal level provided $G'(0)=0$ (essentially because $\Delta^{\circ n}(z)=\frac{z}{1+nz}\sim_{n\to\infty}\frac{1}{n}$). Hence the only obstruction is the value of $G(0)=G'(0)=0$.
On the other hand, if you wish to study convergence of $F$ then you need a more detailed and subtle analysis.
The equation $f(x+1)-f(x)=g(x)$ can be viewed as a "discrete derivative", hence a solution $f(x)$ can not be expected to be unique.
\begin{align} f(x+1)-f(x)&=\sum_{j\geq0}f_j\sum_{k=0}^j\binom{j}kx^k-\sum_{k\geq0}f_kx^k \\ &=\sum_{k\geq0}x^k\sum_{j\geq k}\binom{j}kf_j-\sum_{k\geq0}x^kf_k=\sum_{k\geq0}x^kg_k. \end{align} Equating coefficients, you find $$g_k=\sum_{j>k}\binom{j}kf_j.$$ When can be inverted to solve for the $f_j$'s? This is possible, for instance, if $g(x)$ is a polynomial.
The matrix involved is upper diagonal $$M_{k,j}=\binom{j}k \qquad k\geq0, \,\,\,j\geq1.$$
Look at Ramanujan Summation, also called indefinite summation. Ramanujan extended The Euler Mclaurin summation method using Bernoulli numbers and a whole stack of ideas. This is a well studied subject. In general, if your $g$ is defined in the right half plane and satisfies for $z = x + iy$ $$|g(z)| < C e^{\rho|x| + \kappa|y|}$$
where $\kappa < \pi$ and $\rho$ is arbitrary, then a unique solution exists, where $f(0) = 0$ and $f(z+1) - f(z) = g(z)$. Ramanujan's formula is rather simple
$$f(z) = C + \int_0^zg(t)\,dt + \frac{1}{2}g(z) + \sum_{n=1}^\infty \dfrac{B_{2k}}{2k!}g^{(2k-1)}(z)$$
where $C$ is a constant making $f(0) =0$ and $B_{2k}$ are the bernoulli numbers. Since this is a linear operator on $g$ it is common to write
$$\sum_z g = f$$ $$\sum_{j=1}^z g(j) = f(z)$$ $$\sum_1^z g(p)\Delta p = f(z)$$
where each draws the similarity between it and the integral.
I can give you a formula for the more restrictive case $\kappa < \pi/2$, which is a modification of Ramanujan's method.
$$f(z) = \frac{1}{\Gamma(1-z)}\Big{(}\sum_{n=0}^\infty \sum_{j=1}^{n+1}g(j)\frac{(-1)^n}{n!(n+1-z)} + \int_1^\infty\big{(}\sum_{n=0}^\infty \sum_{j=1}^{n+1}g(j)\frac{(-w)^n}{n!}\big{)}w^{-z}\,dw\Big{)}$$
which follows by Ramanujan's master theorem. All in all the subject has four names: Indefinite summation, continuum sums, Ramanujan summation, Euler-Mclaurin summation. I can't remember the names of the books, but there are some books on the calculus of finite differences that deal with this subject.
The idea goes hand in hand with Newton series as well, where in such a case it is rather trivial to produce an indefinite sum if the function is expanded in a Newton series. I.e: if $$g(z) = \sum_{n=0}^\infty a_n\dbinom{z}{n}$$ In this case it's much easier to work with $g$ bounded in the right half plane, so it's more restrictive.
Also, if $g(z)$ has nice decay properties we can always take the wondrous classical equation
$$f(z) = \sum_{k=0}^\infty g(z-k) - g(-k)$$ or
$$f(z) = \sum_{k=0}^\infty g(z+k) - g(k)$$
which was Ramanujan's motivation, but these are more volatile.
