For a system $\dot x = f(t, x(t))$ with $f\in C^0(I\times \mathbb{R}^n,\mathbb{R}^n)$, the answer is: $f$ is necessarily of the form $f(t,x)=A(t)x+b(t)$, if we assume that the affine dependence of $x(t_1)$ from $x(t_0)$ holds for all pairs $t_0, t_1$ in $I$ (or, for all $t_1$ and some fixed $t_0$, hence for all). Of course, together with the implicit assumption that any IVP for this ODE has unique solution in the whole interval $I$.
Premise. Let $I$ be an open interval and let $W:I\times I\rightarrow M_n (\mathbb{R})$ a $2$-parameter family of matrices verifying the chain law:
$$W(r,s)=W(r,t)W(t,s),$$
$$W(t,t)=\mathbb{I},$$
for all $r,s,t$ in $I$. Assume that $W(t,s)$ is continuous in the pair $(t,s)\in I\times I$ and partially differentiable in the first variable.
Then it follows that $\partial_1W(t,s)W(s,t)$ is a matrix $A(t)$ independent from $s$, and that $W$ is continuously differentiable in $I\times I$, with $\partial_1W(t,s)=A(t)W(t,s)$, and
$\partial_2W(t,s)=- W(t,s)A(s)$, for all $t,s$ in $I$. In other words, $W(t,s)$ is the transition operator associated with the linear system $\dot u(t)=A(t)u(t).$ (Note that this corresponds to your situation in the special case where $c_j$ are identically zero for all $t_0$ and $t_1$).
Indeed, by the chain law we have that all $W(t,s)$ are invertible, and $W^{-1}(t,s)=W(s,t)$ for all $t,s$, so $W(t,s)$ is differentiable in its second variable too. If we derive w.r.to $t$ the identity $W(r,s)=W(r,t)W(t,s)$ we get $0=\partial_2W(r,t)W(t,s)+W(r,t)\partial_1W(t,s) $, whence multiplying conveniently on the left and on the right we see by separation of variables that both sides of $ \partial_1W(t,s) W(s,t)=- W(t,r) \partial_2W(r,t)$ only depend on $t$. If we denote the common value $A(t):=\partial_1W(t,s) W(s,t)=- W(t,r) \partial_2W(r,t)$, a continuous path of matrices, we have that $W$ solves $\partial_1W(t,s)=A(t)W(t,s)$ (and also verifies $\partial_2W(t,s)=-W(t,s)A(s)$ ), with initial condition $W(t,t)=\mathbb{1}$, that is, it is the transition operator associated with the linear system $\dot u(t)=A(t)u(t).$ (Incidentally, $W$ is actually $C^1$ by the Total Differential Theorem).
$$*$$
Now, as in your assumptions, let be given a system of ODE's, all of whose IVP have unique solutions $x:I\to\mathbb{R}^n$, and assume that for all $s,t$ in $I$ the value of the solution $x $ at time $t$ depends affinely from the initial data at time $s$. In other words, assume that for any solution $x:I\to\mathbb{R}^n$ there holds
$$x(t)=W(t,s)x(s)+c(t,s),$$
for some $2$-parameter family of matrices $W(s,t)$ and some $2$-parameter family of vectors $c(t,s)$.
Since by assumption for any $s\in I$ the values $x(s)$ can be chosen in the whole space, the above form implies that $W(t,t)=\mathbb{I}$ and $c(t,t)=0$, for any $t\in I$. Since any solution $x(t)$ is differentiable, we have that both $W(t,s)$ and $c(t,s)$ are partially differentiable in their first variable; moreover $W$ and $c$ are continuous in the pair $(t,s)$, as the continuous dependence from the initial data is in fact a general consequence of the uniqueness property of solutions of the IVP.
The above form also implies, for any $r,s,t$ in $I$ and for any solution $x$:
$$x(r )=W(r,s)x(s)+c(r,s)=W(r,t)x(t)+c(r,t)=W(r,t)\left[ W(t,s)x(s)+c(t,s) \right]+c(r,t) $$
$$ =W(r,t)W(t,s)x(s)+ W(r,t) c(t,s) +c(r,t) . $$
Again this implies two separate identities, because for any fixed $s$, $x(s)$ varies in the whole space $\mathbb{R}^n$:
$$W(r,s)=W(r,t)W(t,s)$$
and
$$c(r,s)= W(r,t) c(t,s) +c(r,t),$$
together with $W(t,t)=\mathbb{I}$ and $c(t,t)=0$, for all $r,s,t$.
By the initial remark, we have that $W$ is the transition operator of a linear system $\dot u(t)=A(t)u(t)$ corresponding to a continuous path $A(t)$ of matrices. Finally, differentiating $c(r,s)= W(r,t) c(t,s) +c(r,t)$ w.r.to $r$ we have
$$\partial_1c(r,s)= A(r)W(r,t) c(t,s) +\partial_1c(r,t)$$
$$=A(r )\left[ W(r,t) c(t,s) + c(r,t) \right] -A(r )c(r,t) + \partial_1c(r,t)$$
$$=A(r )c(r,s) -A(r )c(r,t) + \partial_1c(r,t).$$
Separating variables, this tells that $\partial_1c(r,s)-A(r )c(r,s)= \partial_1c(r,t)-A(r )c(r,t) $ is a continuous path $b(t)$ only depending on $t$.
We therefore have for all $s,t$ in $I$ and for all solution $x$
$$\dot x(t)=A(t)W(t,s)+\partial_1c(t,s)=A(t)\left[W(t,s)+c(t,s)\right]+\partial_1c(t,s)-A(t)c(t,s)=A(t)x(t)+b(t),$$
proving that the ODE is actually affine.