No polynomial will suffice for $f$ since sets with the same moments up to order equal to the degree of $f$ are indistinguishable. To determine the set, moments of order up to the size of the set are needed.
If $f(x) = e^x$, then $\sum f(x_i-\lambda) = e^{-\lambda}\sum e^{x_i}$ so no information can be obtained other than $\sum e^{x_i}$. So faster than polynomial growth is also insufficient in general.
However, if $f(x)$ is monotonic and increases faster than $e^x$, the set can be reconstructed at least if it has distinct elements. For example, if $f(x)=e^{x^\alpha}$ for $\alpha\gt 1$ then $\sum f(x_i-\lambda)$ is dominated by the largest $x_i$s as $\lambda\to -\infty$. ThenSpecifically, the sum is asymptotic to $me^{(x_{max}-\lambda)^\alpha}$ where $m$ is the number of times the largest value $x_i$$x_{max}$ is in the set. From this $m$ and $x_{max}$ can be removed anddetermined. Then remove $me^{(x_{max}-\lambda)^\alpha}$ from the process repeatedsum and repeat.