No, I'll give an example of a morphism $f:X\to Y$ which is not affine although $X$ is affine.
Take $X=\mathbb A^2_k$, the affine plane over the field $k$ and for $Y$ the notorious plane with origin doubled.
We have $Y=Y_1\cup Y_2$ with $Y_i\simeq \mathbb A^2_k$ open in $Y$ and we take for $f:X\to Y$ the map sending $X$ isomorphically to $Y_1$ in the obvious way.
Then $f$ is not affine because the inverse image $f^{-1}(Y_2)$of the affine open subscheme $Y_2\subset Y$ is $X \setminus \{0\}$, the affine plane with origin deleted, well known not to be affine.