Let $\mathbb R^4_A$, $\mathbb R^4_B$ be spacetime as seen by two inertial observers $A$, $B$ respectively, and call $f:\mathbb R^4_A \to \mathbb R^4_B$ the change of coordinates.

We assume that $f$ is a bijection that send straight lines in spacetime corresponding to the motion of free bodies to straight lines (this is to account for $A$ and $B$ being inertial).

In special relativity, by the constancy of light, one has that light cones are mapped to light cones by $f$, and Alexandrov theorem tell us that that is sufficient for $f$ to be affine.

I've read in a book (which title i can't recall at the moment, but i would guess that the following is a common statement) that in Galilean physics you still get that $f$ is linear (this time by the fundamental theorem of affine geometry) because it sends straight lines to straight lines.

That doesn't seem so obvious to me, since while in classical mechanics you assume that there's no restriction on the velocity of a body, i don't see why lines with t = const (which would correspond to bodies with infinite velocity) should be sent to straight lines unless one impose continuity or something else on $f$. Am i missing something?