I would like a reference and/or a simple proof using well-known results of the following, which I think is true. (If it's false, I'd like to know that as well of course -- and ideally a way to modify the statement to make it true.)
Let $X$ be a compact, connected, semi-algebraic set of dimension $n+1$ inside $\mathbb{R} \times \mathbb{R}^n$, and assume that $X$ contains an open $(n+1)$-dimensional ball centered at the origin. Define $$V(t) = \operatorname{Vol}\left(X \cap (\{t\} \times \mathbb{R}^n)\right),$$ where $\operatorname{Vol}$ just means $n$-dimensional Lebesgue measure. Then $V$ admits a power series expansion $$V(t) = a_0+a_1t+a_2t^2+\cdots$$ in an open neighborhood of $t=0$.
EDIT: To avoid counterexamples like the one in MattF's comment, let's assume that no line contained in the subspace $\{t=0\}$ is tangent to the boundary of $X$. This is probably not the best way of phrasing this condition -- I welcome advice here.
I'm aware of results of Lion and Rolin which has a similar flavor to this, for subanalytic sets (and with some logs I don't want in the result...). Based on my very limited knowledge of the subject, I would imagine that a proof would work by induction, and use strongly the cell decomposition of $X$. In the inductive step, I guess one is integrating the lengths of some line segments, and these are given in terms of roots of polynomials, so you use some results about analyticity of roots of polynomials as a function of the coefficients... I think I could eventually give a rigorous proof, but this seems like the kind of thing that must be well-known (again, if it's true!) to the right people. After all, the semi-algebraic category should be easier than the subanalytic one. The logic tag is because maybe this follows from a general result for o-minimal structures.