Firstly, it may be useful to point out that Witt originally did not introduce Witt vectors for modelling $\mathbb{Z}_p$ on the basis of $\mathbb{F}_p$ or the like, his starting point was to generalize Artin-Schreier theory to $p^n$-extensions, $n > 1$ (nowadays called Artin-Schreier-Witt theory), so it is kind of Galois theory-motivated.
Thus, Witt's original motivation/philosophy when thinking about ghost components may well diverge from the 'standard perspective' most texts use nowadays.
As you specifically ask "where the ghost components that are used to define Witt vectors come from.", this is maybe what you are looking for. This is beautifully explained by Harder in his article "Witt vectors" in a very motivating way.
I just copy
"See also Harder, Wittvektoren, Jahresber. Deutsch. Math.-Verein. 99 (1997), no. 1, 18--48. An English translation of this paper has appeared in Ernst Witt, Gesammelte Abhandlungen, Springer, Berlin, 1996."
from Chandan Dalawat's posting in the discussion "http"https://mathoverflow.net/questions/512/what-is-interesting-useful-about-big-witt-vectors"
It may be totally useless, but the original german version is actually available online under http://dml.mathematik.uni-bielefeld.de/JB_DMV/JB_DMV_099_1.pdf, page 22 (warning, big file!)
Hmmmmm, so I hope maybe this is in some way useful...... maybe not.