If $X$ is a Hilbert space and $A$ is an unbounded self-adjoint operator on $X$, is it necessarily that $A^k$ is self-adjoint for all positive integer $k$? (I have already known that the conclusion holds for $k$ a power of $2$)

By the functional calculus for selfadjoint operators, $f(A)$ is selfadjoint for any finite real-valued measurable function $f$; see, for example, K. Yosida, Functional Analysis, Springer, 1965.

Yes. This follows immediately from the spectral theorem and form the functional calculus of selfadjoint operators, even for a much wider range of functions.